RTX 3090 Encoding Overload Issues (Not Streaming, Just Recording)

S

SuperHaterBater

New Member
Hi! :)

I have a GTX 1080 TI that I screen-record with a lot. I use it to record my desktop and to watch what I'm doing on various programs. I use the lossless setting in OBS when going to "Simple" instead of advanced. I record in my native desktop resolution of 3840 x 2160 and record to display capture (yes, I've read that's a no-no here, but I've been doing it for years with my 1080 TI and it's doing great still and never had issues).

Now my problem is that now I have a new computer with an RTX 3090 it says "Encoding Overload" even just sitting on the desktop for a single second.. The 1080 Ti has absolutely no issue using the exact same settings, but this RTX 3090 can't even record a still screen in lossless... I am honestly at a loss at what's going on with the GPU, but I know you guys are mad wizards and probably know something I don't! :)

Thanks guys! :)

The log is here

 
Last edited:
S

SuperHaterBater

New Member
Please don't respond with separate settings to use in "advanced" mode that aren't the "simple" mode fully lossless setting. I can see that it still works perfectly to screen record on that setting in 3840 x 2160 on my 1080 Ti so the problem has got to be something other than that! :) I should be able to screen record the same quality on my 3090 as I can on my 1080 Ti, so it is saddening when people are telling me the solution is to record in a lower quality..

The CPU is a 5950X, GPU is a 3090.

For comparison, the computer with the GTX 1080 Ti that I screen record with daily is using an i9 7980XE and I use it in 3840 x 2160 with the lossless setting in "simple" mode just the same.

Both computers are very comparable in CPU-power, but one has a ridiculously strong GPU compared to the other and for some reason the stronger computer can't screen record... There has to be a wrong setting somewhere in Windows or something that's causing the better computer to be unable to screen-record in full-lossless, but I am fully at a loss.

Thank you all!
 
Last edited:
S

SuperHaterBater

New Member
On the older computer with the GTX 1080 Ti I'm running an old Samsung 970 Pro and it does fine. On the newer computer I'm running a Western Digital SN850 I believe, so a much quicker NVME and it is still having this issue. So confused... I also tried setting the output folder to a Samsung 970 Pro I had laying around but it still had this issue, so it's not the drive that's being encoded to that's the problem.
 
S

SuperHaterBater

New Member
rockbottom said:
I don't have that issue writing to a 980 Pro. Perhaps it is the drive not able to keep up, bit-rate is high.
Click to expand...
The drive I use regularly for this is a Samsung 970 Pro. The bitrate of that drive is way way slower than a Western Digital SN850. If it is working with a GTX 1080 Ti with the slower drive, it isn't the drive. Also, I tested it with both drives with the newer GPU and computer setup and it still does not work.
 
S

SuperHaterBater

New Member
PaiSand said:
Try using hardware encoder (NVENC) instead of software encoder (ffmpeg).
Click to expand...
I do not believe NVENC is available as an option when using the "Simple" mode "fully lossless" setting.

If it is, I can not see where it would be in the "simple" mode settings to enable it. I do not believe NVENC is capable of encoding fully lossless.
 
S

SuperHaterBater

New Member
rockbottom said:
I'm running a 12900k/3090 & tested your settings...
Click to expand...
Exactly. It should work on this no problem. My 1080 Ti is running it no problem. That's exactly what I'm saying. Was hoping one of the OBS wizards would have a solution. Maybe a change in Windows or something that needs to happen.
 
R

rockbottom

Active Member
The SN850 should be OK but maybe something's not quite right. The drive could be bad, bad drivers, bad configuration, the thing is overheating. Record to a different drive. A 3 minute recording produced a file larger than 48GB.....
 
koala

koala

Active Member
Is this a custom built computer or one ready built from some shop? in case it's custom built, or you made hardware changes after purchase yourself, you might have introduced some issue.

Make sure the GPU is running with x16 pci-express speed. Use gpu-z to verify. Usually, this speed is exclusive to only one pci-express slot on the motherboard, where the gpu has to be located in a single gpu setup. Usually, it's the slot nearest to the CPU.
Make sure the ssd you're recording to is working properly. Run a disk benchmark to verify its speed. Should be over 1000 MB/sec.
Check power supply, cabling, overheating.
 
S

SuperHaterBater

New Member
rockbottom said:
The SN850 should be OK but maybe something's not quite right. The drive could be bad, bad drivers, bad configuration, the thing is overheating. Record to a different drive. A 3 minute recording produced a file larger than 48GB.....
Click to expand...
I did the same with a Samsung 970 Pro that I normally use for this that has much less throughput. It's a PCI-E generation behind the other...
 
Harold

Harold

Active Member
In simple mode, the "lossless" encoder uses the utvideo encoder, not any h264, h265 or av1 based encoder, and will run entirely in software on the CPU.
 
R

rockbottom

Active Member
Looks like the 970 Pro more than holds it's own against the x850. It's actually faster than the x850 in sustained writes, even the EVO is faster, both are gen 3.

1716580808200.png

1716580930138.png
 
S

SuperHaterBater

New Member
rockbottom said:
Looks like the 970 Pro more than holds it's own against the x850. It's actually faster than the x850 in sustained writes, even the EVO is faster, both are gen 3.

View attachment 104080

View attachment 104081
Click to expand...
I honestly don't know what you're trying to prove. An SN850 is PCIE Gen 4. Both are NOT Gen 3. I said this. It isn't even in the same ballpark and anybody who works in computer technology or even dabbles in computers knows that... Beyond this, it doesn't do anyone any help to argue about it when I said I tried it with a Samsung 970 Pro already and it doesn't work on the RTX 3090 but does work on the GTX 1080 Ti...
 
S

SuperHaterBater

New Member
Harold said:
In simple mode, the "lossless" encoder uses the utvideo encoder, not any h264, h265 or av1 based encoder, and will run entirely in software on the CPU.
Click to expand...
This is what I thought. I use this setting on my 1080 Ti computer and never have an issue, but maybe the explanation is because it's not GPU-bound so the 1080 Ti isn't a part of the equation then.. I think we may be getting somewhere!

Is there specific CPU settings to look up in BIOS that could affect this then?
 
S

SuperHaterBater

New Member
rockbottom said:
The numbers don't lie. Start troubleshooting your rig. OBS is not your issue.
Click to expand...
I know OBS is not the issue. I came on the OBS forum to see if anyone else had run into the same issue and knew of a fix in the BIOS or some type of system change that needed to be done. Honestly you need to chill out on the snark. If your intention is just to say "figure it out", then you don't need to post anything on here.
 
R

rockbottom

Active Member
I already told you to try recording to a different drive. How did that go? Yeah I did but it's in another system.

koala & Harold provided some steps above. We're not here to help you fix your hardware.
 
You must log in or register to reply here.
Top