Question / Help Software h264 or Hardware NVENC ?

Banyarola

Active Member
I would like to know the difference between, Software h264 or Hardware NVENC in OUTPUT/ENCODER/SETTINGS?
And which is better to stream with.

Thanks.
 

Harold

Active Member
software x264 will currently ALWAYS offer a better picture at low bitrates.

In its current state, nvenc is really only suitable for recording.
 
If you really can't use x264 software because your cpu is to weak then you can use Nvenc but as @Harold has said this isn't great and shows especially in high motion games Pixelisation / Artifacting. In Low motion Nvnec can be ok.

Therefore always software x264 unless you really can't afford a performance hit.
 

Banyarola

Active Member
My PC is powerful enough. I was just curious what it was.
Also, I don't do any gaming.. Mostly webcam,music and my weather station streaming from Weather Underground.

Thanks for your opinion.
 

storrm

Member
NVENC is perfectly fine to record with (not stream) as long as you use a high bitrate. I use OBS Studio with x264 @ 3500kbps to stream and then use NVENC to record locally at 7000Kbps. This gives me the chance to edit the video later on if there's something I want to highlight, which I can then upload to YouTube. Dont be afraid to bump the NVENC bitrate even higher, it doesnt seem to hurt it much (its built for the job!). The more bitrate, the better the quality but of course you will end up with large files.
 

Ph1Lt0r

New Member
I would disagree. I use nvenc for my streams playing elite dangerous (fast moving game) and it's quite clear. 720p 60fps at a bitrate of 3500. If you want to go lower than 3500 then yes, x264 all the way.

An example from monday night. Please make sure you select source to see what it actually looks like:
https://www.twitch.tv/ph1lt0r/v/98323179

We're doing combat, so you'll see fast turns, explosions, etc. Note, language, and I was dressed as Ham Solo, so fair warning, there is chest hair.
 
Last edited:

koala

Active Member
I disagree with Harold. You can absolutely use nvenc as streaming encoder. It's good. For 2500, it gives in fact even better quality than x264 in OBS default configuration of preset "veryfast". x264 is only better if you use the more demanding preset "faster" or an even more demanding preset. Quickync, on the other hand, is definitely worse than x264 in high motion scenes. This is also true for 3500. But nvenc is really nice.

If you ask for proof, I did extensive and automated analysis of the different encoders and settings to answer for myself the same question the OP asked. I plan to post about that research some days later, but here is a preliminary extract of the bitrate 2500 videos. The important column that tells about the quality is video_avg_mse, which contains the average mse (mean square error) of all frames compared to the lossless reference video all the videos were created from. The lower the mse, the better the quality. Why it is not exactly sorted in ascended order of video_avg_mse, I will tell later - the humanly perceived quality may differ from that value.The columns mse_low contains the mse of a hand-picked frame with low motion, and the column mse_high contains the mse of a hand-picked frame from a high-motion sequence. It's always the same frame in all videos, so the values are comparable.
The table is for 1280x720p30.

enc1.png


Of course, numbers heavily depend on the game and the scenes you want to present. I prepared a reference video of 2 minutes gameplay of Guild Wars 2 with almost maximum graphics settings. 1 minute with two different low motion scenes and 1 minute with two different high motion (mass combat) scenes.
 
Last edited:

Banyarola

Active Member
Well guys, I tend to agree with Harold at this time but, guess I am just going to have to try it for myself and see what happens..
I don't do games. I do include some animations, slide shows etc. at times though.


I'm only interested in quality. I stream at 1920x1080 with good quality but if I can get better that would be nice.

Anyway, I only asked the question because I didn't know what that option was but now it's interesting to hear the different opinions.
 

koala

Active Member
Presenting all videos is over the capacity of my web space, but here are 4 of the above:

http://www.wombaz.de/uploads/2016/11/obs-codec-test/1280x720/30/

It's the 3 videos that are made with obs default settings and the x264 with "slow" preset.
The quality difference is more visible in the second half of the videos with the high motion scenes.

The png files are the same frame from all videos, one high and one low motion. The *mse.txt files are the mse values for each frame in comparison to the lossless reference video (which I don't provide here - it's 2.3 Gig), made with the psnr filter of ffmpeg.
 
Last edited:

Suslik V

Active Member
@koala Interesting testing, make sure that you run all tests at least tree times and all tests has no lag or dropped frames during recording. There is problem to start the encoding at the same time with the video input (few black frames at start for example, so not the same length of the video, maybe not the same frame to compare in case of automated analysis - how you deal with this?). Also, there is hw AMD encoder, any plans on it? For streaming important is bitrate (due to network limitations). Is bitrate has the same value for the all encoders during the timeline?

Did you make previews of the videos by
Code:
ffmpeg -debug vis_mb_type i 1.mp4 1out.mp4
pause
to see how first keyframes aligned?

I see the same frames on different time lines: once it #37-38 next file this frame is #43-44 next is #44-45 or so, how it could happen?

Overall feel of hw QSV, hw NVENC and sw x264 from the test videos not so bad. Sharp and smooth movie, who said that 30 fps and 4:2:0 are bad? I like it.
 

koala

Active Member
I did not yet fully describe my method. The encoded video is never synchronized to the original video, there are always a few junk frames at the start, or frames lost at the start. But I synchronize both, so the psnr comparison is really comparing the same frames.

I do this to synchronize the reference video to the encoded video: I picked 2 frames from the reference video (the high motion and low motion png). Then I searched for these frames in the encoded video and got the frame numbers. For example, in the reference video the frame number of the high motion png is 2513, and 2483 in the encoded video.
I substract the frame numbers of the two high motion png's (2513-2483=30) and get the offset of the encoded video to the reference video. If there are blank frames at the beginning of the encoded video: the offset will skip them. Then I use the select filter of ffmpeg to extract exactly the same frames from each video for comparison.

This is the command line for the comparison:

start := refstart + diff
end := refend + diff
[...]
runwait, ffmpeg -loglevel warning -y -hide_banner -i "%videoname%" -i "%refvideo%" -filter_complex "[0:0]select=between(n\`,%start%\`,%end%)`,setpts=PTS-STARTPTS[vid0];[1:0]select=between(n\`,%refstart%\`,%refend%)`,setpts=PTS-STARTPTS[vid1];[vid0][vid1]psnr=stats_file='%statslog%'" -f rawvideo nul

I also verified if there is always the same number of frames between the 2 reference frames: in my video there are exactly 2055 frames between them. If there were not 2055, frames were skipped or one of the frames was not found correctly. This is the proof that the same frames are being compared throughout the whole video.

Unfortunately, I cannot test AMD, because I don't have an AMD card. If someone is willing to test, I can provide the reference video and the autohotkey script that does the recording (it clicks every combination of settings in the GUI of OBS and performs analysis with ffmpeg).

What do you mean with "Is bitrate has the same value for the all encoders during the timeline?" If the video has "bitrate=2500" in its name, it has bitrate 2500 throughout the whole video. It was never actually streamed, of course, just created locally with the stream encoder settings, which would be streamed if streaming were on.

To reduce the possibility of lag or disk congestion, I put the reference video to SSD and let OBS write the encoded videos to the harddisk.
 

Suslik V

Active Member
I mean, just how bitrate fluctuates in time (maybe hw encoder not perfect for streaming in this part). Also, I think it is better to stream to some service and get file as Recording quality: same as stream (Simple Output, streaming always has one audio track, thus no need to run Advanced Output with OBS Studio). Just make sure that dropped/lag frames = 0 due to network issues. And three tries is good option - if comparison is good itself, then it will only improve your results. Anyway, video encoded with loss, so your own "feel" or the footage quality is important too.
 

koala

Active Member
@Suslik V: I should really first stream to some service, download that stream and analyze that? Honestly, I don't see the point. If you capture the data that gets sent out by OBS, it's quite the same and without errors due to network during transmit. For my stream analysis, I actually used the simple output mode and used the "Same as stream" recording quality. Should record footage identical to what would be sent out with streaming.
Meanwhile I did more recording (1280x720 60 fps, and 1920x1080 is running at the moment) and more review of the results - will report on that in a separate thread. In essence, this comparison sheet should tell you exactly what to expect from which codec at 1280x720:

stream codec comparison 1280x720 30fps 60fps.png


According to that, if I were asked what codec one should use for streaming at 1280x720, I would say this:

- if your computer is able to encode with x264 preset=faster, then use this (this is quite high demand on the CPU only for encoding, so it might interfere with your game!)
- otherwise, if you have NVIDIA, use nvenc with preset=highquality (this will not interfere with your game at all)
- otherwise, if your computer is able to encode x264 preset=veryfast (OBS' default), use this (this will somewhat interfere with your game, if your CPU is not powerful enough)
- otherwise, if you have Quicksync available, use Quicksync with preset=balanced (this will not interfere with your game at all)
- don't use any x264 preset=superfast or ultrafast, as they produce way worse output than even Quicksync

and:
- for some reason, Quicksync on Skylake CPU performs on 60fps better than Nvenc, but only for 60fps, not for 30fps.
 
Last edited:

Suslik V

Active Member
@koala, Network can add ram and bus usage to whole process. If you testing streaming abilities of the encoders - it is better to start the stream itself. About downloading back - "Same as stream" is quite enough, I think. But frame drop caused by bitrate fluctuation (if any) is important for the test itself.

Any plans to test bitrates of 1000...2000 kbit/s?
More tries of each "streaming"?

Please, post the link to your thread here too (when it's done).
 

krillep

New Member
After Twitch doesn't "limit" bitrate to 3500 anymore, especially for non-partners using single pc setup to stream with, I would never pick x264 over NVENC, unless your internet speed is horrible. NVENC combined with higher bitrate, 5k+, is just so much better due to no performance loss vs. a marginal difference in quality.
 
Top