Occasional Encoding Overload

Zioth

New Member
I have been researching this issue for almost two months and still have yet to find a resolution so here I am.

GPU NVIDIA GeForce RTX 3060
Processsor i7 5820k
Ram: 32gb

Recording Format: mkv remux to mp4
Encoder: Nvidia NVENC H.264 (new)
Rate Control: CQP
CQ Level: 14, 16, 18, 20 (Have tested various values)
Keryframe Interval: 0 (These are local recordings)
Preset: Max Quality, Quality, Performance and Others. (Tested most of them)
Profile: High and Main both used in tests
Look-ahead: On and Off (tested both)
Psycho Visual Tuning: On
GPU: 0
Max B-Frames: 0, 2, 4 (Tested)

There will be times I can record for an entire hour and no encoding issues. Other times it will be 5, 10, 15 minutes I get some. I have watched my GPU in task manager and I usually am Encoding at 14-16 percent. When the encoding issue happens the Video encode drops for a split second and then back to normal. The problem does not persist constantly. Oddly enough I have tested NVIDIA NVENC H.265/HEVC (via FFmpeg) and when I watch it in the task manager my Video Encode graph runs in the 50's but I never get an encoding issue. I would use it but I am not great with all the settings plus editing the footage was rough in premier.

I am at my wits end with this. I have followed guides, turned settings on and off, always run obs as adminstrator, tested different drives to record to etc. Any help would be great in this matter.
 

FerretBomb

Active Member
Turn Psychovisual Tuning and Lookahead OFF.
Use the Quality preset, NOT Max Quality.

All three of those use CUDA cores, which tend to cause erratic/intermittent Encoding Overloaded errors on some configurations. Turning off all three resolves the issue in >99% of cases (I can only think of 1-2 where it didn't, out of hundreds to thousands where it did).
 

Zioth

New Member
Turn Psychovisual Tuning and Lookahead OFF.
Use the Quality preset, NOT Max Quality.

All three of those use CUDA cores, which tend to cause erratic/intermittent Encoding Overloaded errors on some configurations. Turning off all three resolves the issue in >99% of cases (I can only think of 1-2 where it didn't, out of hundreds to thousands where it did).
I just ran a test and still ended up with encoder lag. I looked through the log file and even ran it through the analyzer. The encoder is skipping frames because of GPU overload.
 

FerretBomb

Active Member
I just ran a test and still ended up with encoder lag. I looked through the log file and even ran it through the analyzer. The encoder is skipping frames because of GPU overload.
GPU overload would manifest as Render Delay, rather than an 'encoding overloaded' message. Post a logfile from a test streaming/recording session where the issue occurred, at least 30 seconds in length, and we can have a look.
 

FerretBomb

Active Member
5-40 frames out of 10,000-200,000 is most likely the frames lost when the encoding session was being set up and started, or shut down. 0.1% or less generally is not considered a problem, just normal variance.
 

Zioth

New Member
5-40 frames out of 10,000-200,000 is most likely the frames lost when the encoding session was being set up and started, or shut down. 0.1% or less generally is not considered a problem, just normal variance.
I figured as much. Guess I was being a bit picky lol. Appreciate the help. I did have one more question, if you have any experience with it. NVIDIA NVENC H.265/HEVC (via FFmpeg), I read it's nice for file size and quality but some of the options confuse me a bit. Do you know anything about it? Basic settings I figured out but I am new to the FFmpeg addons for obs.
 

FerretBomb

Active Member
I figured as much. Guess I was being a bit picky lol. Appreciate the help. I did have one more question, if you have any experience with it. NVIDIA NVENC H.265/HEVC (via FFmpeg), I read it's nice for file size and quality but some of the options confuse me a bit. Do you know anything about it? Basic settings I figured out but I am new to the FFmpeg addons for obs.
No, I'm primarily a livestreamer, and no sites use h.265 yet so I haven't bothered getting into it.
 
Top