OBS videos slowly become frozen

[MCBYT]

Hey! I've been having this issue for around a year now, and after hours and hours of trial and error, I couldn't fix it. It's getting pretty irritating when every video I record becomes unwatchable after 10 minutes.

Essentially, every video starts off just fine. It records at 1080p, 240fps just as I've got in the settings (before you ask, there's no reason 240FPS should cause issues - I've been recording at this quality or higher for 3+ years. I use it for resample-based motion blur and slow-mo shots) and for the first minute, recording is flawless. But after that, recording slowly begins to lose FPS. First it goes down to 60 FPS, then 30, then 10, then 5, then 1, and then to seconds per frame. Then to minutes per frame. By 3 minutes in, it's unwatchable at around 5 FPS, and after a long recording, it can get to the point where the screen won't update for half an hour. This also doesn't happen with every recording, it's only an issue with about 10% of them.

What I've tried:
Recording at 720p does not fix anything. Recording at 1440p, my native resolution, causes my GPU to max out to 100% and is unwatchable from the very start, which is also a problem I've been having for a year (again, RTX 4090 build - this shouldn't be happening) and haven't found a fix for.
My graphics drivers are updated fully.
Lowering FPS to 120 or 60 fixes nothing.
The issue occurs both with Game Capture and Display Capture.
Running OBS as administrator does not help.
Giving OBS high priority in Task Manager does not help.
A malware scan from both Windows Defender and MalwareBytes comes back clean.
The application recorded does not matter - I've recorded Minecraft, Discord, Opera GX, and my desktop, all with the same issue.

My build is:
i5-13600k
RTX 4090
64GB 5600MHZ DDR5 RAM
MSI MAG Z790 TOMAHAWK WIFI Motherboard
Intel 670p SSD (I have other drives, but this is what I record to)
My hardware should not be a problem for any reason. I've recorded at 1080p 240FPS on a Ryzen 5 3600 + GTX 1660 Super build with no problems, and on my last PC, a Ryzen 7 5800X and RTX 3080, I could get up to 1080p 600FPS without problems. Yet now, with a far more powerful PC than either of the above, OBS functions significantly worse.

Here's the current log after the issue has happened:

https://obsproject.com/logs/B8BRqvvUFyncikKv

I've noticed that the problem seems to be encoding lag, based on this snippet here:
"Video stopped, number of skipped frames due to encoding lag: 464445/474162 (98.0%)"
But there's no details that help me diagnose why this is happening.

Does anyone have any ideas on how to fix this? Any help would be greatly appreciated :)

P.S, here's some other problems I'm having that are less important - just figure I'd tack them on at the end in case they're already known and there's an easy fix.
Recording in 1440p immediately maxes out my GPU and the recording is unwatchable.
Recording in higher FPS than 240 immediately maxes out my GPU and the recording is unwatchable. I used to record in 600FPS on a 3080 build, no problem.
OBS occasionally freezes for no clear reason when in the background and needs to be restarted.
Some recordings stall on "Stopping Recording..." when ending from a hotkey, and the recording can only be properly ended by manually clicking the "Stop Recording" button in OBS.
OBS.exe can't be fully closed in Task Manager, instead "End Task" reduces its RAM usage to 12KB and trying to close it gives an "Access Denied" error. (apparently this has to do with LogiTune hooking into the program and not letting it close, I don't know a solution though)
Display capture sources freeze at random when tabbing out of games and need to be hidden and re-shown to work again.
The "Music Control" panel pops up every time I start OBS, despite me closing it each time and also not having any scenes that play music.
OBS occasionally stops responding to my Stream Deck controls and I have to restart streamdeck.exe. (this might be a Stream Deck issue rather than OBS)

 

[PaiSand]

Analyzer | OBS

OBS (Open Broadcaster Software) is free and open source software for video recording and live streaming. Stream to Twitch, YouTube and many other providers or record your own videos with high quality H264 / AAC encoding.

obsproject.com

Use only one instance of OBS.
Disable HAGS
Reduce the FPS to 60 and increase the output resolution to 2k.
Set the CQ level to 16 or 14.
Verify each plugin to be compatible with the OBS version you're using. Remove anything that is not compatible.
The GPU driver is a little too old. Perhaps you should update it. Same with OBS. Remember to backup the scene collection and the profiles, also keep a copy of the installer of the OBS version you are using now just in case. And verify the plugin compatibility with latest version.

It could be that one of the plugins isn't working as intended and causes a memory leak.
And of course a may be a bottleneck between the CPU and GPU.

 

[MCBYT]

Analyzer | OBS

OBS (Open Broadcaster Software) is free and open source software for video recording and live streaming. Stream to Twitch, YouTube and many other providers or record your own videos with high quality H264 / AAC encoding.

obsproject.com

Use only one instance of OBS.
Disable HAGS
Reduce the FPS to 60 and increase the output resolution to 2k.
Set the CQ level to 16 or 14.
Verify each plugin to be compatible with the OBS version you're using. Remove anything that is not compatible.
The GPU driver is a little too old. Perhaps you should update it. Same with OBS. Remember to backup the scene collection and the profiles, also keep a copy of the installer of the OBS version you are using now just in case. And verify the plugin compatibility with latest version.

It could be that one of the plugins isn't working as intended and causes a memory leak.
And of course a may be a bottleneck between the CPU and GPU.

Can't do anything about the extra instance of OBS, that's the one that won't shut down due to a bug.
Didn't know HAGS existed, I've turned it off and will report back.
Reducing the FPS to 60 actually let me increase resolution to 1440p, which never used to be the case - even 1440p30 recording didn't work, previously. I did a little messing around and discovered that changing my codec to NVENC AV1 instead of NVENC H.264 actually lets me record smoothly in 1440p 240FPS though, which is awesome!
All my plugins are compatible and functioning as they should - I only have Tuna and shaderfilter installed to my knowledge, both of which I use only for streaming and don't exist in the scene collection I'm using right now.
I'll try updating the GPU driver/OBS again, maybe that will help

 

[koala]

I've noticed that the problem seems to be encoding lag, based on this snippet here:
"Video stopped, number of skipped frames due to encoding lag: 464445/474162 (98.0%)"
But there's no details that help me diagnose why this is happening.

This message is not in the log you posted, at least not with 98%, but if you encounter it, it's the amount of frames skipped due to encoder overload. Every frame must be encoded within the time between frames. For 60 fps it's 16.667 ms, 120 fps is 8.3 ms, 240 fps is 4.1667 ms. If the encoder isn't able to finish its encoding within that time frame, the next frame comes in and needs to be encoded. One of both must be skipped.
So this message is a direct indicator of encoder load, which is dependent on GPU load if you use a hardware encoder. Nvenc uses almost no GPU resources, because it is a dedicated circuit, but very high GPU load affect the encoder as well.
Each hardware encoder also has its integrated limit - it is designed to encode with some given minimum speed, and this is usually 60 fps, because this fps is the common fps almost everyone uses. You try to encode with 4 times that speed, this will work only if the encoder was designed to support 4 times the expected load, for example to support 4 sessions with 60 fps each.
For exact numbers, see:

NVENC Application Note

Archive version of Video Codec SDK

docs.nvidia.com

To see how your system performed, terminate OBS after some recording and open the logfile just written. It will contain a profiler footer not visible until OBS is terminated, which will tell you how the encoder performed for each session.
An example of the profiling footer from one of my logs:

14:30:55.812: == Profiler Time Between Calls ==================
14:30:55.812: obs_hotkey_thread(25 ms): min=24.084 ms, median=25.444 ms, max=46.657 ms, 52.5123% within ±2% of 25 ms (0.027137% lower, 47.4606% higher)
14:30:55.812: obs_graphics_thread(16.6667 ms): min=14.85 ms, median=16.667 ms, max=385.15 ms, 99.8433% within ±2% of 16.667 ms (0.0783699% lower, 0.0783699% higher)
14:30:55.812: obs_graphics_thread(16.6667 ms): min=6.862 ms, median=16.667 ms, max=26.472 ms, 99.5708% within ±2% of 16.667 ms (0.21877% lower, 0.210409% higher)
14:30:55.812: obs_gpu_encode_thread(16.6667 ms): min=16.001 ms, median=16.666 ms, max=17.373 ms, 94.8718% within ±2% of 16.667 ms (2.61159% lower, 2.51662% higher)
14:30:55.812: obs_gpu_encode_thread(16.6667 ms): min=0.514 ms, median=16.666 ms, max=390.254 ms, 98.8372% within ±2% of 16.667 ms (0.664452% lower, 0.498339% higher)

Look at the obs_gpu_encode_thread entries. The median values should be exactly the frame time (for 240 fps: 4.166667 ms), and the % values should be as high as with my logs, and the last number on that line (that tells encoding was higher) should be as low as possible and not more than a few percent at worst.

 

[MCBYT]

This message is not in the log you posted, at least not with 98%, but if you encounter it, it's the amount of frames skipped due to encoder overload. Every frame must be encoded within the time between frames. For 60 fps it's 16.667 ms, 120 fps is 8.3 ms, 240 fps is 4.1667 ms. If the encoder isn't able to finish its encoding within that time frame, the next frame comes in and needs to be encoded. One of both must be skipped.
So this message is a direct indicator of encoder load, which is dependent on GPU load if you use a hardware encoder. Nvenc uses almost no GPU resources, because it is a dedicated circuit, but very high GPU load affect the encoder as well.
Each hardware encoder also has its integrated limit - it is designed to encode with some given minimum speed, and this is usually 60 fps, because this fps is the common fps almost everyone uses. You try to encode with 4 times that speed, this will work only if the encoder was designed to support 4 times the expected load, for example to support 4 sessions with 60 fps each.
For exact numbers, see:

NVENC Application Note

Archive version of Video Codec SDK

docs.nvidia.com

To see how your system performed, terminate OBS after some recording and open the logfile just written. It will contain a profiler footer not visible until OBS is terminated, which will tell you how the encoder performed for each session.
An example of the profiling footer from one of my logs:

14:30:55.812: == Profiler Time Between Calls ==================
14:30:55.812: obs_hotkey_thread(25 ms): min=24.084 ms, median=25.444 ms, max=46.657 ms, 52.5123% within ±2% of 25 ms (0.027137% lower, 47.4606% higher)
14:30:55.812: obs_graphics_thread(16.6667 ms): min=14.85 ms, median=16.667 ms, max=385.15 ms, 99.8433% within ±2% of 16.667 ms (0.0783699% lower, 0.0783699% higher)
14:30:55.812: obs_graphics_thread(16.6667 ms): min=6.862 ms, median=16.667 ms, max=26.472 ms, 99.5708% within ±2% of 16.667 ms (0.21877% lower, 0.210409% higher)
14:30:55.812: obs_gpu_encode_thread(16.6667 ms): min=16.001 ms, median=16.666 ms, max=17.373 ms, 94.8718% within ±2% of 16.667 ms (2.61159% lower, 2.51662% higher)
14:30:55.812: obs_gpu_encode_thread(16.6667 ms): min=0.514 ms, median=16.666 ms, max=390.254 ms, 98.8372% within ±2% of 16.667 ms (0.664452% lower, 0.498339% higher)

Look at the obs_gpu_encode_thread entries. The median values should be exactly the frame time (for 240 fps: 4.166667 ms), and the % values should be as high as with my logs, and the last number on that line (that tells encoding was higher) should be as low as possible and not more than a few percent at worst.

The message is in the log at "01:27:30.255", just checked.

Did some talking with a friend that's big into encoding/rendering settings and made some adjustments (for anyone in the future, make sure you're using P1-P2 instead of the default P5 quality preset), I'm able to record at 1440p 360FPS smoothly now (though I'm sticking to 240). It seems like 240 was right on the edge of what I could record before, since any FPS higher than that would cause problems - maybe that's why some recordings worked fine and others didn't? Not 100% sure. Anyways, things seem to be working better now, I'll have to see whether I encounter the problem again