4K 60fps lossless recording - not doable on my hardware?

kamild_

Member
Hello,

The important bits of my hardware are as follows:
CPU: AMD Ryzen 7 5800X3D
RAM: GoodRAM IRDM PRO 32GB DDR4 3600 MHz CL18
GPU: ZOTAC GeForce RTX 3070 Ti Trinity OC 8GB (deshroud)
SSD in PC: ADATA SX8200 Pro M.2 2TB (PCIe 3.0)
SSD in a USB NVMe enclosure: ADATA LEGEND 710 2 TB (PCIe 3.0)

I would like to make a recording of a game (it's Beat Saber but running in desktop mode). I want the captured file to be in the highest quality possible - it will not be shared on any video service such as Youtube, it'll be stored locally and maybe shared on Discord after converting it to AV1. My first attempts involved messing with advanced output settings, but after reading up on some threads in this forum saying to do otherwise, I went back to simple output mode and started simple:
1717847335305.png1717848288585.png
Trying to record with these settings yesterday resulted in a lot of "Encoder overload" messages and noticeable stuttering on the recordings. When I uploaded the log to the OBS Analyzer page, I followed the advice and:
  • disabled HAGS
  • made OBS launch as admin
  • removed the display capture source from my scene
  • limited game framerate via v-sync
  • changed my color format, because having it set to BGRA seemed to completely break NVENC encoding, showing me encoder timeouts (I set it to I444).
I did not follow the color range advice because as mentioned, the recording won't be uploaded to any streaming site.
Today's next 3 attempts are covered by this log.

Attempt #1: Recording was going pretty well, until after about 2 minutes my PC has started slowing down to the levels I haven't seen before - it rapidly slowed down until it was completely frozen with looping audio for about 10 seconds. As soon as the system started to recover, I stopped recording. I monitored the system via Task Manager while the recording was happening, and it seemed like resource usage was well within control - CPU at about 30%, 6 GB out of 32 GB RAM taken, GPU 3D load at about 60%, Video Encode at about 30%, VRAM usage at about 2,5 GB out of 8 (values written from memory, could have been slightly different). At that time I was recording to my USB SSD NVMe drive, so I thought this might have been the cause.

Attempt #2: I switched the output folder to my main SSD instead of USB one. It didn't go as bad as attempt #1, but there were still severe periods of the entire system freezing. That's when I noted my SSD usage was at about 90-100%, and "Average response time" was spiking to about 200-300 ms. So I guess my storage is just too slow?

Attempt #3: I switched the recording quality from "Lossless" to "Indistinguishable" and set the encoder to HEVC - since the recording wouldn't be lossless anymore, I wanted to maximize the quality. I have left the encoder preset unchanged (P1: Fastest). I'm guessing this setting is actually more resource hungry, because within next several seconds I was already seeing "Encoder overload" messages, and the recording was severely stuttering.

Is it at that point that I get comfortable with the idea that my PC isn't enough to capture at this resolution and framerate, or is there anything else I can do? My hardware seems to be functioning properly, no issues outside of OBS. Temperatures are well within the norm, the system is up-to-date.
 

rockbottom

Active Member
Your system is capable of 2160p/60 but your setup is a mess. Something seems wrong with the encoder, I noticed even when you are in Simple Mode your Keyframe Interval is being set @ -1, it should be 250. Stream FX appears to be stale, I can't find any info on the version you have installed & I don't see any version that has been vetted with v30.1.2. If it's old & not the current version, get rid of it.

Follow the directions already provided by the Analyzer.

Disable ESET

As far as Lossless goes, you need a real stout SSD. If it's not a "Pro" drive with DRAM, lossless may not be possible.

Try these settings after the above is completed.

19:47:28.975: encoder: NVIDIA NVENC HEVC (FFmpeg)
19:47:28.975: rate_control: CQP
19:47:28.975: bitrate: 0
19:47:28.975: cqp: 16
19:47:28.975: keyint: -1 > 2 Seconds
19:47:28.975: preset: p5
19:47:28.975: tuning: hq
19:47:28.975: multipass: qres
19:47:28.975: profile: main
19:47:28.975: width: 3840
19:47:28.975: height: 2160
19:47:28.975: b-frames: 2 > 0
19:47:28.975: psycho-aq: 1 > Disabled
19:47:28.975: GPU: 0
 

rockbottom

Active Member
So your SSD is a Pro drive but it's Gen 3 & it has really poor performance in that type of workload. Speed drops low enough that you will have trouble with Lossless after a few minutes of recording.

1717854697698.png
 

Suslik V

Active Member
When you using 4:4:4 color space (I444) with NVENC - you forcing OBS to use slower method of encoding (with additional RAM>VRAM transfer):
Code:
...
13:14:45.973: [obs-nvenc] nv12/p010 not active, falling back to ffmpeg
Here OBS expects NV12 (default format, 4:2:0 space) for HW based encoder. Textures in NV12 format OBS can deliver to encoder without changes while it remains inside the VRAM (where the game was rendered and where your hardware encoder is located).

By looking into full log you may see where bottleneck is (performance profiler data will be written to the log on OBS exit). So, find the log that was created by OBS after you've had recording and already exited from OBS.

My other thoughts:
For 3840x2160@60, 8-bit per color component, no chroma subsampling or RGB - you need 1 492 992 000 bytes/s (up to 12 Gbit/s) transfers between video card and RAM, and about half of this for saving to disk. So, with NV12 and NVENC you will have better chances to succeed. But if you need only 4:4:4, then software encoders may work better for you. Some hardware encoders capable of processing CRF=0 (aka "lossless") but subsampling (4:2:0) may still take place, it is kind of "semi-lossless" in my opinion.

Modern SSD drives can cache its data up to tens of GB but then you still may notice drop in writing speed.
 
Last edited:

rockbottom

Active Member
I had to dig into that review but here is the 2TB version. It performs much better but still may drop a bit too low. Any other NVMe's in the system to test?

1717855597871.png
 

kamild_

Member
Thank you for the responses.

I don't have other NVMe drives in my disposal other than the Crucial P1 1TB, which from experience I know is even slower than the ADATA I currently have. I have also removed StreamFX before the today's attempts. I'll try toggling off ESET, I somehow didn't think about that.

Also thanks for the more detailed information about color spaces. If going 4:4:4 can really create such bottlenecks, I'll switch back to 4:2:0 and judge whether or not I'm fine with the quality change on that, and if I'm not - perhaps I'll get a faster SSD.

Unfortunately, software encoding doesn't seem to be possible at this resolution and with this CPU - I tried x264 and AV1 on fastest presets, as well as some Apple ProRes codecs and FFV1 in the advanced mode. All of them aren't keeping up, though maybe there's some tweaking possible to get them to work...
 

rockbottom

Active Member
The performance level of the 970 Pro is more than enough for lossless. You can see in the graph that it just maintains it's speed at all times. Look for similar or better performance from the next drive you purchase.

Stick with NVENC.
 

rockbottom

Active Member
5 minute Lossless test on my system using I444. No issues, are you sure you want to record Lossless? You're probably going to want a 4TB drive....

I'm recording to a 980 Pro.


General​

Unique ID :235277822600368030916447091282575444067 (0xB100DC41D6DB01A4C0EE132CD0A54863)
Complete name :D:\2024-06-08_12-02-16.mkv
Format :Matroska
Format version :Version 4
File size :27.6 GiB
Duration :5 min 9 s
Overall bit rate :767 Mb/s
Frame rate :60.000 FPS
Writing application :Lavf60.3.100
Writing library :Lavf60.3.100
ErrorDetectionType :Per level 1


Video​

ID :1
Format :HEVC
Format/Info :High Efficiency Video Coding
Format profile :Format Range@L5.1@Main
Codec ID :V_MPEGH/ISO/HEVC
Duration :5 min 9 s
Bit rate :749 Mb/s
Width :3 840 pixels
Height :2 160 pixels
Display aspect ratio :16:9
Frame rate mode :Constant
Frame rate :60.000 FPS
Color space :YUV
Chroma subsampling :4:4:4
Bit depth :8 bits
Bits/(Pixel*Frame) :1.505
Stream size :27.0 GiB (98%)
Default :No
Forced :No
Color range :Full
Color primaries :BT.709
Transfer characteristics :BT.709
Matrix coefficients :BT.709


Audio​

ID :2
Format :PCM
Format settings :Little / Signed
Codec ID :A_PCM/INT/LIT
Duration :5 min 9 s
Bit rate mode :Constant
Bit rate :2 304 kb/s
Channel(s) :2 channels
Sampling rate :48.0 kHz
Bit depth :24 bits
Stream size :85.0 MiB (0%)
Title :Track1
Default :No
Forced :No
 

rockbottom

Active Member
I changed up sources & NVENC on my 3090 is hitting 90% utilization using I444, too high. Your 3070TI will be lagging frames in the same scenario. NV12 it's closer to 70% utilized, leaving plenty of headroom.
 

kamild_

Member
Storage space is fortunately not a big issue because I expect my recording to be 3 minutes maximum. The HEVC lossless recordings I did fully capture but stuttered a fair bit were about 22 GB.
I guess I'll have to re-check my Video Encode usage, but I swear when I had it on I444 and on the "Lossless" simple preset, it was hovering at around 30-40%.
 

rockbottom

Active Member
@rockbottom can you try simple output mode with software lossless encoders (both I444 and BGRA)?

I444 is A-OK, better performance than NVENC, file is MASSIVE. But, BGRA lags, need to drop to 30FPS. Also with BGRA the encoder will fail on some start attempts.


General​

Complete name :D:\2024-06-08 14-37-27.avi
Format :AVI
Format/Info :Audio Video Interleave
Format profile :OpenDML
Format settings :PcmWaveformat
File size :77.0 GiB
Duration :5 min 0 s
Overall bit rate :2 201 Mb/s
Frame rate :60.000 FPS
Writing application :Lavf60.3.100


Video​

ID :0
Format :ULH4
Codec ID :ULH4
Duration :5 min 0 s
Bit rate :2 200 Mb/s
Width :3 840 pixels
Height :2 160 pixels
Display aspect ratio :16:9
Frame rate :60.000 FPS
Bits/(Pixel*Frame) :4.420
Stream size :77.0 GiB (100%)


Audio​

ID :1
Format :PCM
Format settings :Little / Signed
Codec ID :1
Duration :5 min 0 s
Bit rate mode :Constant
Bit rate :1 536 kb/s
Channel(s) :2 channels
Sampling rate :48.0 kHz
Bit depth :16 bits
Stream size :55.0 MiB (0%)
Alignment :Aligned on interleaves
Interleave, duration :21 ms (1.28 video frame)
 

kamild_

Member
Try these settings after the above is completed.

19:47:28.975: encoder: NVIDIA NVENC HEVC (FFmpeg)
19:47:28.975: rate_control: CQP
19:47:28.975: bitrate: 0
19:47:28.975: cqp: 16
19:47:28.975: keyint: -1 > 2 Seconds
19:47:28.975: preset: p5
19:47:28.975: tuning: hq
19:47:28.975: multipass: qres
19:47:28.975: profile: main
19:47:28.975: width: 3840
19:47:28.975: height: 2160
19:47:28.975: b-frames: 2 > 0
19:47:28.975: psycho-aq: 1 > Disabled
19:47:28.975: GPU: 0
I set the color range back to NV12 and tried these settings, except I dropped CQP to 10. No issues throughout the recording, Video Encode usage hovered at around 80%. I don't know where I got that 30% figure from. I tried CQP at 1 for fun, but it just slightly delayed the effect lossless recording had on my system, and Video Encode usage was hovering at around 90%, often jumping to ~98%. I'll stick to CQP at 10, I think quality is sufficient for me.

Thank you once again for the help everyone :)
 
Top