Question / Help Nvenc Vs X264 encoder which better for my pc? need advice

M

masaki1988

Member
Hello guys. I read many threads in this forum and on redit too but not understand which is better for me. My pc config is i7 7700k 4.2, gtx 1060 6gb, 8gb gb ram. I try to compare nvenc cqp vs x264 crf and not see any diffenence. Both rate controls depend on number which youi need write. I recording video games many many years and i can t chose between tho those codecs. I try both but can t choice which is better for my target. My target is, recording high action games which is contain many object on screen and high gameplay speed (games like dynasty warriors, yakuza, god of war) with good quality (no pixete image) and smaller file sizes. And i can t choice which is better for those games. I recording from my console ps4 pro and use capture card for it. But on my cpu have weak cooler (not very weak of course but not good) and im afraid if i be recording with x264 my cpu will be too hot and may broken. But if i recording with superfast preset he use 50% of my cpu so i may not afraid overload?. If i use nvenc its pretty good with cqp but when i start recording i see fps will drop for 5-10 sconds and then its normal again. But i
remarked when i use x264 for recording fps is smooth than on nvenc. Why? I thought nvenc is better for stable fps but in my way is conversely. So i need you advice guys which encoder better for my hardware and for my target. Btw i try use hevc and i thought its better codec for now than x264. Yes its better compression video but quality is bad for high dynamic games. Maybe i can t correct setup it? tell me.
 
BK-Morpheus

BK-Morpheus

Active Member
When you compare NVENC CQP and x264 CRF, the image quality should be comparable, while the file size should be smaller on x264, because it is more efficient.
As you only want to record locally with high quality, I would recommend to use NVENC with a quality based (variable bitrate) setting like CQP...especially, because the 7700k has only 4 cores / 8 threads.
Of course, ingame performance suffers from recording, even with NVENC (because OBS and other tools need to render your scene and access the frame buffer through windows with increased PCI-express bandwidth load, whereas Nvidia Shadowplay can access the frames directly from the GPU, which is way faster end less damanding).
If you can find an older version of Nvidia Inspector, you can also deactivated the forcing of P2 power state when using CUDA or NVENC. Per default, your Nvidia card will reduce memory speed, when using CUDA / NVENC, but this can be disabled with some older versions of Nvidia Inspector: https://forums.geforce.com/default/...-causes-gpu-memory-to-downclock/post/5158554/
 
M

masaki1988

Member
BK-Morpheus said:
When you compare NVENC CQP and x264 CRF, the image quality should be comparable, while the file size should be smaller on x264, because it is more efficient.
As you only want to record locally with high quality, I would recommend to use NVENC with a quality based (variable bitrate) setting like CQP...especially, because the 7700k has only 4 cores / 8 threads.
Of course, ingame performance suffers from recording, even with NVENC (because OBS and other tools need to render your scene and access the frame buffer through windows with increased PCI-express bandwidth load, whereas Nvidia Shadowplay can access the frames directly from the GPU, which is way faster end less damanding).
If you can find an older version of Nvidia Inspector, you can also deactivated the forcing of P2 power state when using CUDA or NVENC. Per default, your Nvidia card will reduce memory speed, when using CUDA / NVENC, but this can be disabled with some older versions of Nvidia Inspector: https://forums.geforce.com/default/...-causes-gpu-memory-to-downclock/post/5158554/
Click to expand...
Thank you for reply. But i don t understand what do p2 mode for cuda? its help me gets better fps or icrease quality?. And what about cqp settings?. I set preset on high quality and profile is high BUT what about 2 pass encoding option?. Many people advice uncheck it with cqp. Yesterday i unckeck 2 pass encoding and i get good quality. But today i try check it and get lower quality than yesterday. So question is - check or uncheck this option and which level i should set in level option?.

And about hevc. On amd card its have preset quality, rate control, etc. On nvidia nothing. Only bitrate size. So, how setup rate control quality and etc with nvidia?. 10 series nvidia get full support hevc so why i can t setup it property in obs?.
 
koala

koala

Active Member
With these custom ffmpeg settings I was able to record with constant qp mode with nvenc hevc:

1538072387820.png
 
koala

koala

Active Member
Click on the screenshot and it is displayed fullscreen and can be saved, mangled, zoomed...
 
M

masaki1988

Member
So i try hevc with cqp rate and don t see any differnce in quality or file sizes than nvenc. So im not sure which encoder is better for high quality and smaller size file. Maybe i must keep with x264 crf? i don t know ((((.
 
koala

koala

Active Member
I didn't do any tests with hevc. I only puzzled out the parameters, but did not tune them. May be the qp value I used was too low. Perhaps it can be raised without sacrificing visual quality, so the file size shrinks. It is said that hevc-encoded files with the same quality as h.264-encoded files can be 40-50% smaller. One has to take a reference video and do encodings with a spread of qp values to get to the bottom of this.

It's not that the quality of hevc-encoded files is better - it's the file size that matters. You get the same quality as h.264 with 40-50% smaller files. This is the advantage. This is relevant for CBR-mode for streaming, because you can stream higher resolutions with lower bandwidth. Widespread 4k streaming isn't possible with h.264, because the bandwidth requirement is too high. With hevc it becomes possible. I'm not talking about streaming games, I'm talking about streaming movies and TV series - Netflix, Amazon, etc. They want to sell 4k streaming for their TV productions.
 
M

masaki1988

Member
koala said:
I didn't do any tests with hevc. I only puzzled out the parameters, but did not tune them. May be the qp value I used was too low. Perhaps it can be raised without sacrificing visual quality, so the file size shrinks. It is said that hevc-encoded files with the same quality as h.264-encoded files can be 40-50% smaller. One has to take a reference video and do encodings with a spread of qp values to get to the bottom of this.

It's not that the quality of hevc-encoded files is better - it's the file size that matters. You get the same quality as h.264 with 40-50% smaller files. This is the advantage. This is relevant for CBR-mode for streaming, because you can stream higher resolutions with lower bandwidth. Widespread 4k streaming isn't possible with h.264, because the bandwidth requirement is too high. With hevc it becomes possible. I'm not talking about streaming games, I'm talking about streaming movies and TV series - Netflix, Amazon, etc. They want to sell 4k streaming for their TV productions.
Click to expand...
So hevc is no good for game capture you say? but for amd gpu users in obs have hevc settings as for nvenc x264. Why? because amd gpu better for hevc?. I start thnk about change my gpu to amd because as i see (maybe i wrong) they better for encoding? or better stick with my 1066 gtx?. I bought her because i need gpu for encoding and all friends told me "amd is shit buy nvidia". Nvidia maybe good for games because i many years upgrade my computers with nvidia cards and i was satisfied playing games on max settings. But now i use my pc only for work capture video for youtube and i don t think what nvidia good for encoding videos than amd. If im wrong tell me. I will be happy listen all your advice. Btw my nvenc settings is here on screenshot.
 

Attachments

  • 1538125586197.png
    1538125586197.png
    217.4 KB · Views: 293
koala

koala

Active Member
hevc is a better codec than h.264, but its use is still kind of experimental. There is yet knowledge to build in the streaming and recording community. It is also not widespread, because streaming services currently don't support it. The browser support for hevc is still growingand not available everywhere. If you record with hevc, it's likely that your postprocessing software doesn't support it.

Because of all this, choosing h.264 is usually the better way. It's supported everywhere. It just works. And because of this, your choice of a gtx 1060 was correct, since with nvenc you have a very good hardware encoder for h.264. It also has a very good nvec encoder, in case it gets more important in the future. AMD cards don't have a good h.264 hardware encoder. It's even worse than Intel's Quicksync. The video from the amd encoder is always a bit grainy and has visible artifacts, regardless how high you push the quality settings. If this is true for the AMD hevc encoder as well, I don't know.

If I look at your screenshot, you showed the "streaming" tab. These settings are not used for recording, only for streaming. For streaming, the settings are wrong - you should use CBR for streaming and "auto" for level. But for recording, you should use CQP. Switch the tab to recording and enter the desired settings for recording.

Or, much better, switch to simple output mode and configure the encoder settings for recording in a much smaller and clear dialog. This will result in better recording quality for you. Just set "Nvenc (hardware)" as encoder in simple output mode and als quality "Indistinguishable quality". That's all you need to get a perfect recording.
 
M

masaki1988

Member
carlmmii said:
1. That's the streaming tab
2. Uncheck "Enforce streaming service encoder settings"
Click to expand...

koala said:
hevc is a better codec than h.264, but its use is still kind of experimental. There is yet knowledge to build in the streaming and recording community. It is also not widespread, because streaming services currently don't support it. The browser support for hevc is still growingand not available everywhere. If you record with hevc, it's likely that your postprocessing software doesn't support it.

Because of all this, choosing h.264 is usually the better way. It's supported everywhere. It just works. And because of this, your choice of a gtx 1060 was correct, since with nvenc you have a very good hardware encoder for h.264. It also has a very good nvec encoder, in case it gets more important in the future. AMD cards don't have a good h.264 hardware encoder. It's even worse than Intel's Quicksync. The video from the amd encoder is always a bit grainy and has visible artifacts, regardless how high you push the quality settings. If this is true for the AMD hevc encoder as well, I don't know.

If I look at your screenshot, you showed the "streaming" tab. These settings are not used for recording, only for streaming. For streaming, the settings are wrong - you should use CBR for streaming and "auto" for level. But for recording, you should use CQP. Switch the tab to recording and enter the desired settings for recording.

Or, much better, switch to simple output mode and configure the encoder settings for recording in a much smaller and clear dialog. This will result in better recording quality for you. Just set "Nvenc (hardware)" as encoder in simple output mode and als quality "Indistinguishable quality". That's all you need to get a perfect recording.
Click to expand...
Done. Ok i understand thank you for advice. I try simple mode.
 

Attachments

  • 1538131058111.png
    1538131058111.png
    236.8 KB · Views: 343
You must log in or register to reply here.
Top