Question / Help NVENC - what has the most effect on quality?

cadcTV

New Member
So, I have an Nvidia Titan Black Graphics card, which is why I use the NVENC encoder. I went off of this video guide here for quality settings, and it gave me pretty satisfactory results: https://www.youtube.com/watch?v=h08Sqg5B9As

however, I wanted to push the quality a bit further from what was suggested in this video, and I wanted to find out which setting would have the most bearing quality wise when using the NVENC encoder.

I will go over a check list of the settings I have (if you watch the video, they are essentially the same)

* Output mode: Advanced
* Encoder: ENVENC
* Enforce Streaming Service encoder settings: Checked

* Rate Control: CBR (Constant Bit Rate) [Should I try Lossless?]
* Bitrate: 6000
* Keyframe Interval: 2

*Preset: Default
* Profile: High
* Level: 4.2

* Use Two Pass Encoding: Unchecked
* GPU: 0
* B-frames: 2 (I tried 4, but yielded either equal, or slightly worse results)
---------------------

I've only actually tried testing B-frames, since some say that More B-frames = better quality, some say less B-frames = better quality, I'm not sure which is which, however I tried 4 from 2, and yielded equal to slightly worse results, so i think less B-frames = better quality?

I haven't fiddled around with more bit rate, but when I was using X-split, a slower CPU preset had more bearing on quality, but also drastically increased my CPU core temps, and also unsteady frame rates at times.

I guess what i'm trying to find out is if there is sort of like a CPU preset equivalent when using the NVENC encoder, because when I was using Xsplit with medium CPU preset, I could pump out amazing quality at a lesser Bitrate (I was using 5000 Bitrate at the time)

What would be the variable that would affect quality the most out of these settings? I was thinking maybe increasing the level, but i read this source that increasing the level, only specified to OBS how much Bitrate and decoding speed it would allow for, so i don't think this has direct bearing on quality if I increased it to a higher level: https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels
----------------------------------
My Computer Specs:


Motherboard: ASUS Rampage V Black Edition
Processor: Overclocked Intel Core i7 Extreme -5960X (8 Cores 4.2 - 4.7GHz)
Memory: 32GB - Corsair Dominator Platinum 3100MHz DDR4
Graphics Card: EVGA NVIDIA GeForce GTX Titan Black Superclocked - 6GB GDDR5
Power Supply: Corsair AX1500i Modular Power Supply (1500 Watts)
Liquid Cooling: Custom pump and pipes
Optical Drive: 14X Blue Ray / RW Combo
Solid State Drive: Samsung 840 EVO 1TB SSD
Hard Disk Drive: Western Digital Black 4TB - 7200RPM HDD
Card Reader: Internal 5.25" Bay 6-Slot Card Reader
 
Last edited:

koala

Active Member
In case you want to stream: use CBR as rate control, profile high, nvenc defaults for the rest, except the bitrate, which you set to a somewhat high value in relation to your fps and resolution. As example, see the Twitch encoding recommendations: https://stream.twitch.tv/encoding/

In case you want to record: use CQP as rate control, a CQP value of 20-15 (lower CQP means higher quality), profile high, nvenc defaults for the rest.

If you don't need to record multiple audio tracks, you can as well use simple output mode and all fields are hidden you don't ever change anyway. The defaults in simple mode are set optimal, you only need to set bitrate (for streaming) or quality (for recording). Simple output mode is not a "bad quality" mode, it's a "all settings hardcoded optimal, only required settings exposed" output mode.
 

carlmmii

Active Member
Nvenc doesn't have anything like x264 where you can tell it to allocate more processing power to give better compression. It's a very strict encoder with little in the way of customization, mainly because it's an ASIC specifically for encoding video.

Also, the Nvenc encoder you have in your Titan Black is going to produce the same exact results as the Nvenc encoder from any 1st gen Nvenc card (including something like a GT630). Same silicon. Same quality.

If you're using this to stream to twitch, then you should be using CBR at whatever bitrate you are wanting to put out there. If you're just using this for recording, then you should use CQP and set a quality level as kaola mentioned, as well as uncheck "enforce streaming service encoder settings".

As far as specific settings to use, for 1st gen Nvenc there's very little difference between high quality and default (and using high quality will normally put you over the encoding capacity if you're trying for 1080p), so you're good there leaving it default. B-frames are hit or miss as you found, but leaving it on 4 has the ability to add a little more compression ability.

The one thing I will say is set your Level back to Auto. There is absolutely no reason to change this -- the encoder will pick the appropriate level for whatever output settings you set, and this avoids the error from occurring when you don't choose the correct level.
 

cadcTV

New Member
thank you for the helpful input, I may try and see how OBS handles the X264 encoder on my end and see if i can get a preset that is slightly better than what i can achieve with the ENVEC encoder, if not, I can live with the ENVEC output, very fluid 60FPS with relatively very good quality
 
Top