Encoding Overload on NVENC (New)?

[Squirtinq]

Hi,

New to the forums, so this is my first post. Read below to inform yourself on my odd problem:

So, for some reason I have been having trouble lately with encoding recordings. I have always used NVENC, both new and old, and have never had an encoding problem until recently. I will add my most current (last) log showing that it for some reason skipped frames due to lag.

Something to keep note of though is that the settings you will see in the log have always been there, even since the old NVENC, and have never been "overloaded" until recently.

I do have a GTX 2070 Super GPU and a Ryzen 9 3900x 12core/24thread CPU. I'm not sure which one I would use, but obviously the GPU would be preferable over the CPU.


Log:

https://obsproject.com/logs/jpLlF0HdYppF9640

08:02:27.083: Video stopped, number of skipped frames due to encoding lag: 38/75816 (0.1%)

EDIT: I have already ran my log through the analyzer and have changed color range to "Partial" from "Full" and made all audio samples to 48.00kHz

 

[FerretBomb]

Switch back to Partial. You only use Full if you have an end-to-end Full RGB production pipeline (and almost no one does, unless you specifically invest in one) and are only recording locally (it is hard-coded to Partial while streaming, and will cause color issues).

Switch to the Quality preset, NOT Max Quality. The 2-pass filter MQ enables is not meant for use with real-time video encoding.
Disable Lookahead and Psychovisual Tuning. They do provide a minor visual quality improvement, but when recording locally should not be relevant, and can cause encoding overload issues when none should exist. These are probably the main source of your problem.
You're using 4 b-frames... why? Unless you're using VERY low-motion video, this will be actively harmful.

Do NOT record in CBR. Use CQP or CRF, which are quality-target based encoding methods. CBR is only used while livestreaming as it is a requirement for the back-end infrastructure and CDN chunking. It's both horribly wasteful and incredibly prone to low-rate choke at the same time. Recording locally, CQP/CRF will use as much or little bitrate as needed to maintain a set image quality. No waste, no choke.

 

[Squirtinq]

Switch back to Partial. You only use Full if you have an end-to-end Full RGB production pipeline (and almost no one does, unless you specifically invest in one) and are only recording locally (it is hard-coded to Partial while streaming, and will cause color issues).

Switch to the Quality preset, NOT Max Quality. The 2-pass filter MQ enables is not meant for use with real-time video encoding.
Disable Lookahead and Psychovisual Tuning. They do provide a minor visual quality improvement, but when recording locally should not be relevant, and can cause encoding overload issues when none should exist. These are probably the main source of your problem.
You're using 4 b-frames... why? Unless you're using VERY low-motion video, this will be actively harmful.

Do NOT record in CBR. Use CQP or CRF, which are quality-target based encoding methods. CBR is only used while livestreaming as it is a requirement for the back-end infrastructure and CDN chunking. It's both horribly wasteful and incredibly prone to low-rate choke at the same time. Recording locally, CQP/CRF will use as much or little bitrate as needed to maintain a set image quality. No waste, no choke.

I already switched back to Partial. Like anyone would most likely do, they would turn on Look-Ahead and Psycho Visual Tuning for the most quality improvements. However, I will turn them off.

In the case of CQP, I use that once I'm finished recording with Handbrake to lower the file size, but also keep 90% of the quality. I usually set it at 23, but I find it pointless just to re-use the same encoding method with the same setting I'd use on OBS when it won't make a bit of difference to the file size.

What is even the recommended settings for a 1080p 60fps scenario on CQP, without having an overload?

EDIT: Not to mention CBR will have so much smaller file size after done with a recording. In my case, each minute roughly equals 100MB

 

[FerretBomb]

Like anyone would most likely do, they would turn on Look-Ahead and Psycho Visual Tuning for the most quality improvements.

If you're using CQP/CRF while recording, having these on would only result in slightly smaller file sizes. It's not worth the issues they tend to cause.

In the case of CQP, I use that once I'm finished recording with Handbrake to lower the file size, but also keep 90% of the quality. I usually set it at 23, but I find it pointless just to re-use the same encoding method with the same setting I'd use on OBS when it won't make a bit of difference to the file size.

This reencode just gets rid of the wasteful 'junk' bitrate that you've rolled in while recording by using CBR, unless you're using a better-quality non-realtime encoding preset too. If you don't use CRF/CQP while recording though, you will still have periods of bitrate choke in situations where the encode needed more to maintain image quality. This cannot be 'recovered' during the re-encode.

What is even the recommended settings for a 1080p 60fps scenario on CQP, without having an overload?

EDIT: Not to mention CBR will have so much smaller file size after done with a recording. In my case, each minute roughly equals 100MB

Resolution and framerate do not factor in. The CRF number is essentially simply how far the image quality is allowed to vary from perfection.
General use, CQP 20-23 is decently watchable with minor artifacting on close inspection. 16-20 is good to visually lossless. 12-16 generally is only used if you plan to edit and re-encode it later, to minimize reencode artifacts. Below 12 should not be used unless you know what you're doing, why you're doing it, and what problems it will create. The lower the number, the closer to 'perfect' (uncompressed) video it is, and the larger the file size will be.

If CBR results in smaller file sizes, it's because the video quality needs more and you are setting an artificially low rate, resulting in bitrate choke and image degradation. It is a sign that you have created a problem for yourself. ESPECIALLY if you plan to re-encode later. But it is your foot, and your bullet. Go for it.

 

[Squirtinq]

If you're using CQP/CRF while recording, having these on would only result in slightly smaller file sizes. It's not worth the issues they tend to cause.


This reencode just gets rid of the wasteful 'junk' bitrate that you've rolled in while recording by using CBR, unless you're using a better-quality non-realtime encoding preset too. If you don't use CRF/CQP while recording though, you will still have periods of bitrate choke in situations where the encode needed more to maintain image quality. This cannot be 'recovered' during the re-encode.


Resolution and framerate do not factor in. The CRF number is essentially simply how far the image quality is allowed to vary from perfection.
General use, CQP 20-23 is decently watchable with minor artifacting on close inspection. 16-20 is good to visually lossless. 12-16 generally is only used if you plan to edit and re-encode it later, to minimize reencode artifacts. Below 12 should not be used unless you know what you're doing, why you're doing it, and what problems it will create. The lower the number, the closer to 'perfect' (uncompressed) video it is, and the larger the file size will be.

If CBR results in smaller file sizes, it's because the video quality needs more and you are setting an artificially low rate, resulting in bitrate choke and image degradation. It is a sign that you have created a problem for yourself. ESPECIALLY if you plan to re-encode later. But it is your foot, and your bullet. Go for it.

All I needed was ranges of quality and what they're for. Thank you.