Question / Help NVENC (new) Encoding Lag

[WillmsiasTheMad]

I recently upgraded to an RTX 2060 Super and was looking forward to the boost in recording quality that should've come along with using NVENC instead of x264 on my Ryzen 5 2600. NVENC, however, seems to have other plans.
No matter what I do, I always get encoding lag.
Recording League of Legends on Max Quality mode with no Look Ahead or Psycho Visual Tuning? That's around 0.1-0.2% encoding lag. Bearable, but can be annoying when the missed frames happen in series.
Recording A Plague Tale with the same settings? 0.7% (as shown in this log). Turn on Look Ahead and Psycho Visual Tuning and that jumps to 1.5% or more. This is unbearable, as it manifests as frequent, significant stutters.
The session linked to that log was around an hour long. The first 30 minutes had 0 issues. I would guess there was 0-0.1% dropped in those first 30 minutes. After that? It all of a sudden starts stuttering awfully. This makes me think that my GPU is overheating, except my framerate in game was never affected. So maybe the CPU overheated and clocked down? Nope. I just upgraded my cooler, and am running at a solid all core overclock of 4.0GHz with max temps hitting 75 degrees.

Other possibilities:

• Maybe windows started doing something in the background? Possible, but I have no way of knowing now. Would re-enabling game mode help this? Disabling it was one of the suggestions I came across when troubleshooting NVENC issues earlier in the week.
• Maybe the section of the game I got to was more CPU intensive? I highly doubt it. The stutter began at the end of one chapter when everything on screen calmed down, and continued into one of the more tame chapters of the game.

Dropping from Max Quality to Quality doesn't seem to help. The Encoder is never over 24% utilized, even when the "Encoder Overload!" message is displayed. This makes sense, as NVIDIA claims the encoder can handle 2 4K streams, or an 8K 30fps stream without overloading.

Other settings I have:

• Process priority set to High
• OBS being run as Administrator (suggested by NVIDIA in this guide)

Things to note:

• In LoL I have a performance issue where it never hits my cpu with more than a 25% load (and I've never seen any core maxed out). My GPU sits at 20-30%. Nothing I do can increase these (including my overclock... the cpu was running at 3.4-3.6 GHz before). So there's plenty of overhead for whatever OBS needs to do.
• A Plague Tale seems to use all 12 threads quite heavily, so I could see the issue being related to the first/second core of the CPU being hit too hard. The only things I can think of to fix this (if it's the issue) are to give OBS around 2 cores completely to itself, or to upgrade to a higher IPC cpu (Hello Ryzen 3000... but I was hoping to wait for 4000...).

The last thing I can try is an odd fix I found on Youtube: Open the game before OBS. I highly doubt that'll work, but I'm willing to give it a shot (also not really an option for league)... I guess I've also heard that a bad Nvidia driver could be the issue, but this has happened on the newest one (for Minecraft RTX), and the one before... I could try going older.

I would appreciate any ideas/advice/discussion on this topic.

 

[carlmmii]

Update windows to 1909, and continue running OBS as administrator.

Change off of Max Quality (this forces 2-pass encoding, which is a CUDA load).

 

[WillmsiasTheMad]

Update windows to 1909, and continue running OBS as administrator.

Change off of Max Quality (this forces 2-pass encoding, which is a CUDA load).

2-Pass is a CUDA load? I thought it wasn't (hence why it's still on)... That's likely the largest remaining issue, considering my GPU is at 100% in A Plague Tale. I'll try that. I'll also update to 1909 if any issues remain. Thanks for the quick reply.

 

[carlmmii]

The update to 1909 also means that coupled with running OBS as administrator, windows will prioritize OBS for GPU functions alongside games.

 

[FerretBomb]

This. Use Quality instead of Max Quality, disable Lookahead and Psycho-visual Tuning. Those are the three that use CUDA cores, and tend to cause problems even in a non-max-loaded system. Even with TONS of GPU available, PVT always causes render delay on my setup; the other CUDA options less-so, but still occasionally.

On a side note, I do not believe 1909 is required... IIRC it's 1903 and later, same as (related to?) the GPU selection update for laptops. OBS does still need to be running as Admin though, and OBS 24.0.3 or later (when the workaround was added).

 

[WillmsiasTheMad]

Sad to report that turning 2-pass off made no difference (here's the latest log). Still getting hit with 0.7% encoding lag with seemingly all CUDA requiring settings turned off. Next step is trying it with Windows update 1909, and if that doesn't work, I can try limiting the framerate to 60fps to reduce my GPU load... Not exactly ideal, but 60 is fine.
And if that doesn't work... I could try a preset from the Simple mode and mixing my audio into one track.

Another thing I just noticed when watching the footage, the stutter starts around 30 minutes in again. Is it possible the issue could be hard drive related? I'm recording onto an HDD instead of my main SSD. Is it possible that the hard drive slows down after 30 minutes of sustained load causing "encoding lag" since it can't write the new frames to storage quick enough?

 

[carlmmii]

Yes, that is definitely something that can happen. Try recording directly to an SSD and see if the issue clears up.

Unfortunately, because you're only seeing the issue after 30 minutes, this is a tricky problem to diagnose. But, something I would try is this:
- Set up replay buffer saving (settings->output->replay buffer) to save 1 minute of recording
- Open your stats window
- Start recording like normal, and start your replay buffer
- Once you see your encoding lag start happening in your stats window, stop recording and leave the replay buffer going
- After 1 minute and a few seconds, save your replay buffer

That should isolate the report between encoding with the requirement to save to physical media, and encoding with deferring the data transfer until later.

 

[vapeahoy]

Check that your devices run in MSI mode. Several ways to do that. For me that use lossless nvnenc, only max quality or max performance is supported, as such i always use max performance. The idea is just get it in and out of there asap in a 2pc setup. But the principle is still valid in 1pc setup, you'll prefer consistency over high quality. Install coreaac, actually dont need to install it - just some of the dll files copy to the obs folder. Possible culprit could perhaps be related to consequent explorer.exe being blocked, and thus inferring a halt process somewhere.

Most harddrives today have about 256 mb cache, some have less but 240-250mb cache on there is quite common. Generally record to ssd, then long term storage copy manually or thru a daily batch to hdd. Another option is also to record to a ram drive and then copy it after the fact to disk. This would also serve as a good way to rule out harddrive, given you have enough memory to record a bit.

 

[WillmsiasTheMad]

I did some testing over the past few days, recording furmark, prime95, and both at once. It appears that the issue was the hard drive. I just successfully recorded prime95 and furmark with 0% encoding lag when using my SSD. In the next couple days I'm going to try recording the game again to ensure that the issue is solved.

 

[WillmsiasTheMad]

Finally got around to testing in game. I can now officially confirm that the issue is completely eradicated when recording to an SSD. I assume Psycho Visual Tuning, Two-pass encoding, and look-ahead may cause some lagged frames even when recording to an SSD, so I'll keep them off. Seeing 0 frames missed due to encoding lag is pretty nice.
Thanks everyone for your help!

 

[ttv/cro_dr3ad]

This. Use Quality instead of Max Quality, disable Lookahead and Psycho-visual Tuning. Those are the three that use CUDA cores, and tend to cause problems even in a non-max-loaded system. Even with TONS of GPU available, PVT always causes render delay on my setup; the other CUDA options less-so, but still occasionally.

On a side note, I do not believe 1909 is required... IIRC it's 1903 and later, same as (related to?) the GPU selection update for laptops. OBS does still need to be running as Admin though, and OBS 24.0.3 or later (when the workaround was added).

Hi friend.
I was reading several your threads, thx a lot for helping us.
My question is Max B-frames 2,1 or 0 for FPS games stream (720p, 30/60 FPS, 2500kbps/3800kbps) ?

 

[FerretBomb]

Hi friend.
I was reading several your threads, thx a lot for helping us.
My question is Max B-frames 2,1 or 0 for FPS games stream (720p, 30/60 FPS, 2500kbps/3800kbps) ?

No problem. This is unrelated to this thread though, and probably should have its own thread opened.
Actually, someone opened one about exactly this yesterday. Here's a link:

Question / Help - b-frames best value for quality?

Good afternoon! I tested the nvenc (new) encoder for a long time to get the maximum quality of the stream and recording, but still did not make a final conclusion. I would like to hear your opinion on the use of b-frames. Increasing b-frames to 1-2 improves the graininess and image quality in...

obsproject.com

In short, zero is what I use for high-motion games. FPS games are high-motion.

 

[ttv/cro_dr3ad]

No problem. This is unrelated to this thread though, and probably should have its own thread opened.
Actually, someone opened one about exactly this yesterday. Here's a link:

Question / Help - b-frames best value for quality?

Good afternoon! I tested the nvenc (new) encoder for a long time to get the maximum quality of the stream and recording, but still did not make a final conclusion. I would like to hear your opinion on the use of b-frames. Increasing b-frames to 1-2 improves the graininess and image quality in...

obsproject.com

In short, zero is what I use for high-motion games. FPS games are high-motion.

Thx, I had no issues with 2, and checked psycho visual tuning before v25 (superior image quality over 0 and unchecked). Game I have issues with is cod modern warfare/warzone... So I'm crazy at this point, not shure is is OBS bug, nvidia driver bug or cod update...

edit:
I have issues no matter what I do, with 2 and check mark on psycho, 1h 30min is time in stream when problems start, with 0 and psycho unchecked about 2h 30 min, sometimes problems start straight away, no matter is OBS started with administrator role or without...

 

[vapeahoy]

Without b-frames you will just provide inconsistent compression and it will lead to it's own issues. Strength of encoding solution matters a lot more. Systems that tend to go drifting could have minor instability issues. I'd do some testing with booting stresstesting app and watching for erratic behaviour. Unfortunately without experience of what is normal behaviour, troubleshooting it will take some time.
What RAM is installed?

 

[ttv/cro_dr3ad]

ok i have more details now as I manage to recreate bug after around 50min of test stream
screenshoot in attacment
log file:

https://obsproject.com/logs/4oMUWi9I9CjdyJ4C

 

[Scrolls]

The update to 1909 also means that coupled with running OBS as administrator, windows will prioritize OBS for GPU functions alongside games.

I registered here just to say: Thank you for posting this. Running Streamlabs as Admin fixed my issues of the past 2 days.