Question / Help Best NVENC Settings For Local Recordings?

T

Trichael Man

New Member
I was just curious as to what the best/optimal settings are for locally recording high FPS games such as CS:GO and Overwatch. From my research online I believe that the debate comes down to using CQP or VBR as my encoder. Here are my questions:

CQP vs. VBR?
  • If so, which CQP level? I have been using 14-19 for some test recordings; 18/19 have some visible pixelations.
  • If using VBR, what should my bitrate be for:
  • 720p60?
  • 1080p60?

2-pass encoding needed?
From what I gathered, 2 pass reduces file size and increases quality, but takes longer to process the VoD once I stop recording on OBS Studio. I have also read that CQP (and maybe VBR?) does not require 2-pass encoding so I was wondering what the best option/general consensus was for that.

Re-encode via Handbrake to reduce file size? If so, what settings?
My internet is fairly poor and I plan to upload these videos to YouTube.
Settings would be:
  • Video Codec: x264
  • FPS: same as source, or 60fps? The source is already 60fps, but I was wondering if it matters which setting I pick.
  • CRF quality: which is best? I am currently using CRF = 14 right now.
Here is a sample of what I did recently, if that helps at all:
https://www.youtube.com/watch?v=cSUOTRfp2bs

OBS Studio 0.16.2
Recorded via H.264 (NVENC)
1080p60
CQP = 14
Preset = High Quality
Profile = High
Resulting File Size: 939 MB

Re-encoded via Handbrake
1080p60 CBR
x264 preset = Medium
CRF = 14
Resulting File Size: 364 MB (61% size reduction)

Computer Specs:
CPU: i7-4770k OCed to 4.0 GHz
RAM: 16GB running at 1866 MHz
GPU: EVGA GTX 760 SC w/ ACX 2GB
SSD: Samsung 850 EVO 1TB
HDD (where my VoDs are recorded to): 2TB Western Digital Green
 
Fenrir

Fenrir

Forum Admin
Honestly, you should just use simple mode and the presets (probably indistinguishable quality). If you don't understand what the settings you're changing do, you probably shouldn't be changing them.
 
T

Trichael Man

New Member
I understand your concerns, however I do find the topic of optimizing video quality fascinating, and I would like to learn more about it, so why not start on Google and here, right?

There just hasn't been much discussion on NVENC CQP specifically, but I was able to find this topic here:
https://obsproject.com/forum/resour...lity-recording-and-multiple-audio-tracks.221/

I was wondering if this QuickSync guide could apply to NVENC as well, if so CQP is the way to go:
https://obsproject.com/forum/resources/custom-parameters-of-quicksync.104/

However, most of the major event streamers I talk to recommend VBR (they use both XSplit and OBS Studio) when using NVENC to record their VoDs, but the 2 guides above recommend CQP. I just wanted to hear others' opinions regarding this matter.
 
Fenrir

Fenrir

Forum Admin
Are you sure they're talking about STREAMING? Because streaming VBR or CQP is a horrible idea.

EDIT: Nevermind, re-read and yes they are talking about recording. Carry on.
 
W

Waistless

New Member
Several things:
  • CPU encoding with CRF is a lot more efficient than NVenc, even at the ultrafast preset. Hardware encoding is great if you're also streaming with the CPU at the same time, if you aren't though, stick with CPU encoding with CRF, especially with your high end CPU. (note: for Overwatch setting the FPS cap to the same as your monitor refresh will save a lot of CPU usage)
  • CQP 14 is quite aggressive and will come close to lossless quality. If you're recording with the purpose of editing your footage then go for it. But if you're looking to go straight to uploading, you could probably bump this up for less filesize.
  • FPS Option: Shouldn't matter, but keeping to same as source should help prevent any potential jitter.
  • No harm in leaving 2-pass on from experience, either streaming with CBR or recording with CQP.
  • There's now a b-frames setting in the nvenc encoder option; better off leaving this at 2, as this is what the youtube settings guide recommends for uploads.
  • Rest of your nvenc settings are good.
  • If you have any browser sources leftover from streaming or whatever, make sure they're not in the same scene collection or make sure they are set to 60fps in their options. I've found out from experience they can cause jitter if set to a different fps than what you are recording with, even when not active.

Regarding VBR, from experience, there are edge cases where say you have a lot of small particles that might be difficult to encode (I will use the teleport cutscene from obduction as an example of this). CQP and CRF are quality targets, so this sort of thing could easily result in bitrate spikes of 100mb/s+ or so (and when youtube re-encodes most of this is getting thrown away). VBR keeps this sane, and is probably better for longer recordings. Also given that esports broadcasts aren't post-edited, this also makes VBR a suitable choice for them.

Also @Fenrir "just use simple mode" is not a helpful comment. Simple settings do not allow for any sort of fine tuning based on CPU usage vs. desired file size, when it comes to recording with CRF for instance. Plus he's here asking for advice on this subject anyway.
 
Last edited:
T

Trichael Man

New Member
Thanks so much Waistless! I'm pretty sure Fenrir thought I was just streaming at the time, but it's no big deal now :)

Questions/Clarifications:
  • Yes, I am streaming with x264 via my CPU, so NVENC (or QuickSync) is a must to also have high quality local recordings.
  • Should I leave 2-pass for the NVENC local recordings as well?
  • I picked CQP = 14 since the recommendations I've found gave me a range of 15-25, so I decided to go 1 less the minimum (odd quirk of mine)
  • I'm still a little confused on your stance of CQP vs. VBR
  • As for browser sources, just either remove them, or make sure they are set to 60fps
 
W

Waistless

New Member
  • Fair enough :)
  • I don't think it makes a difference either way, but I'll go with yes.
  • Sure thing, just keep an eye on the bitrates on the files it generates when you record a scene with a high amount of action, that will give you an idea of what to expect.
  • CQP is probably too much data for long esports streams to encode, is what I was getting at. But for personal recording I do think CQP & CRF are superior.
  • Correct.
 
T

Trichael Man

New Member
Your responses are much appreciated :)

Just a few more questions!
  • Should I set the "level" to "auto" or "4.1" manually?
  • What does the "GPU" option mean, above "B-Frames"?
  • Random tangent, should I pick 44.1 or 48 kHz as my sampling rate?
 
W

Waistless

New Member
  • I use auto, haven't tried the other settings, but reading up it has something to do with what's suitable for resolution + FPS combination, so probably best left on auto.
  • Which GPU the encoder is using, if you have multiple GPUs in your system (default is 0 as that's the starting number)
  • Youtube recommends a sampling rate of 48khz. In addition, this can be specified in your sound properties (right click volume > playback devices > properties on playback device > advanced). Probably best to also set that to 48khz for consistency.
 
S

Schauerland

Member
Quality: CQP (CQP/CRF is always Single pass, so the two pass setting can be unchecked or maybe will be ignored anyway?) - set the value to 1 for best quality (actual 0 seems to be something like "auto" and 15 is the same quality as 1 - the videos didnt get more bitrate with 1 then 15; so use 1 or 15)

Keyframe - i set it to 1 becaus the stream is faster seekable (and maybe the quality is increased idk). you can also leave it by 0 or set it to 10.

use preset high quality or high performance - i noticed no quality differences.

Profile high - this is the default for hd videos
Level auto - depends on your video resolution - i dont really know why this is even settable; this option should be removed

Two pass encoding: leve it checkd or not - should be ignored using CQP anyway.
GPU - If you have sli you can choos your card starting at 0 - if you have one card leve it as it is.
B-Frames: 2 or 4 (max) in theory it should increase qualiy and result in smaller files. idk - just leave it as it is.

With these settings filesize will be very big (depending on fps, resolution and image complexity)
this around 2-4 GB/Minute for the game "The Forest" @ 2560x1440@60fps for my example (
to have a perception of some realistic recording)
recording lossless (x264, qp=0) would result in about 10GB/Minute - Premiere can't handle that ;)

audio: in theroy compressed 44.1 320kbit aac would be a higher quality then 48.0
if you upload to youtube use 48.0 - yt will convert it to 48 anyway.

NVENC will ALWAYS result in bigger filesize at the same quality compared to x264 because software encoding is way more accurate - think for yourself, there must be a reason why NVENC is so fast ;)
 
Last edited:
M

messi

New Member
im searching for my optimal settings for my pc i have gtx 1050 / 16 go ram / i7 7700hq
i tried a lot of options but i think nvench encoder is the best for me im searching for best prest , profile , bitrate...
some one can help me plz im global elite in csgo and i want to do videos on youtube
 
You must log in or register to reply here.
Top