Recent Encoder Overload Issues

[Loebrah]

How's it going! Recently I have run into some issues with OBS while Streaming to YouTube & Recording ARK: Survival Evolved and I was hoping to either find out the best settings I am capable of running. Below I will list my computer specs and my current OBS settings as well as a log to take a look at. I appreciate anyone who takes the time to help me figure out what is going on and what I can do to fix this issue!

Computer Specs
GPU: Nvidia Geforce RTX 3080
CPU: AMD Ryzen 7 5700x
RAM: 32 GB

My recording path is set to an external 1TB HDD and maybe I should switch that to record to my actual C drive but I am not sure how much that changes things. I do limit my FPS on ARK to 120fps and I keep the settings around MID/HIGH ranges. Sometimes the stream seems to go fine for an hour or two then the encoding issue gets worse and worse. Also I stream to YouTube-RTMPS but somewhere I had read maybe to try YouTube-HLS I just don't know the difference there.

OBS Settings
Base Canvas: 2560x1440
Output Resolution: 1920x1080 ( Could changing base canvas to 1920x1080 help with eliminating rescaling? )
Downscale: Lanczos
FPS: 60

Output for Streaming
Video Encoder: NVIDIA NVENC H.264
Rate Control: CBR
Bitrate: I have tried ranges from 6000 to 7500 (20MB Download)
Keyframe Interval: Tried 0 & 2 (What is the difference?)
Preset: P5 (I've thought about lowering this to P4 just haven't tried it yet)
Tuning: High Quality
Multipass Mode: I've kept this at single but also tried Two Passes (Quarter Resolution)
Profile: High
Look Head: OFF
Psycho Visual Tuning: ON
GPU: 0
Max B-Frames: 2

Recording Settings
Record As MKV
Same Video Encoder
Rate Control: CQP
CQ Level: 18
Rest of the settings match stream

I have streamed WoW with maxed graphic settings and never had an issue with encoder overload. I do know ARK is a more demanding game to run so I most likely will have to tune either OBS settings or graphic settings but I figured I would ask first before I go changing anything! If there is anymore info that you need I will try to reply ASAP so I can get this solved with everyone's help! Thanks again for taking the time to look at this.

 

[PaiSand]

Base resolution is the resolution of your main display.
Output resolution is the one you aim to stream/record to.

Do not use advance output. No need for it.
Simple run the Auto-configuration Wizard and apply the settings it gives. Do not change them. Restart OBS (!important)
Now, notice you're on simple output mode. Keep it like this. Test as is.
Try changing the the quality presets in here to see if your hardware is able to accomodate it. Test. Test. Test.

 

[Lawrence_SoCal]

My recording path is set to an external 1TB HDD and maybe I should switch that to record to my actual C drive but I am not sure how much that changes things.

External HDD typically have much lower throughput that direct motherboard attached storage. Personally I record to my NVMe SSD C:\, remux when done, and then move the video to a HDD for archiving... so I don't need to worry about Disk I/O contention... even though my HDD should be able to handle my recording throughput just fine... in my case, not worth it to me to risk a Recording
but if drive I/O overloaded, OS would start buffering, etc... not encoder overload ... so I doubt this ais an issue for you... but I'd still adjust anyway to give your CPU (and USB Root Hub, if USB connected HDD) less to deal with

OBS Settings
Base Canvas: 2560x1440
Output Resolution: 1920x1080 ( Could changing base canvas to 1920x1080 help with eliminating rescaling? )

To avoid/minimize re-scaling, I have my Canvas and Resolution set to 1080p even though my main monitor is also 1440p. However, in my case, I'm using Window (not Display) Capture, and size my app window to be 1080 pixels tall
Assuming you want to use full resolution and Display/Screen capture, and you don't want to stream at that resolution, then re-scaling required YouTube re-encodes regardless. I'm not a gamer, so not an issue I've looked into, but have you researched or tested streaming at 1440p (assuming you have sufficient upload bandwidth), and seeing result after YouTube re-encodes and re-scales?

Downscale: Lanczos

Why Lanczos? just curious

In my use case, I do use Advanced Output settings, as I stream at one bitrate, but want to keep a higher quality local recording. But I'm NOT running a game, so CPU & GPU utilization both on low-end, and no CUDA implications with Look Head & Psycho Visual Tuning

 

[Loebrah]

External HDD typically have much lower throughput that direct motherboard attached storage. Personally I record to my NVMe SSD C:\, remux when done, and then move the video to a HDD for archiving... so I don't need to worry about Disk I/O contention... even though my HDD should be able to handle my recording throughput just fine... in my case, not worth it to me to risk a Recording
but if drive I/O overloaded, OS would start buffering, etc... not encoder overload ... so I doubt this ais an issue for you... but I'd still adjust anyway to give your CPU (and USB Root Hub, if USB connected HDD) less to deal with


To avoid/minimize re-scaling, I have my Canvas and Resolution set to 1080p even though my main monitor is also 1440p. However, in my case, I'm using Window (not Display) Capture, and size my app window to be 1080 pixels tall
Assuming you want to use full resolution and Display/Screen capture, and you don't want to stream at that resolution, then re-scaling required YouTube re-encodes regardless. I'm not a gamer, so not an issue I've looked into, but have you researched or tested streaming at 1440p (assuming you have sufficient upload bandwidth), and seeing result after YouTube re-encodes and re-scales?


Why Lanczos? just curious

In my use case, I do use Advanced Output settings, as I stream at one bitrate, but want to keep a higher quality local recording. But I'm NOT running a game, so CPU & GPU utilization both on low-end, and no CUDA implications with Look Head & Psycho Visual Tuning

I use the Lanczos downscale because that was what I had found suggested. From my research there doesnt seem to be much more system usage between the next step down but using Lanczos is double for rescaling clarity. I haven't tested going one step down to see if there is much of a difference yet.

I know you said you're not a gamer but would using game capture or display capture be better overall? I've always had my game as a individual game capture rather than screen capture and have never tested to see if it would use less resources. As for streaming in 1440p the only thing I have done was recorded in 1440p but the file sizes were a bit to large for how long I was recording and if I didn't cut them up or tranfer them right away I would lose refordings from my M2 being full. I am a little confused with what you are talking about with YouTube re-encoding. YouTube recieves what ever signal I send then wether it is 1080p or 1440p and then re-encodes with either avc1 or vp9 but that wouldn't have anything to do on my end other than what signal I am sending.

The mutiple streams I have done with base canvas at 2560x1440 and Output Resolution 1920x1080 have been clear and smooth but would setting both settings to 1920x1080 require less system resources since there would be no rescaling with me still gaming at 2560x1080?

 

[Lawrence_SoCal]

Old school (initial) SSDs would crash (sometimes brick) if regularly kept (near) full. To work around silly (PEBKAC) users, mfgs started adding some extra buffer space... still, filling a SSD to over 90% negatively impacts performance and wear-levelling... strongly not advisable.. you are asking for trouble

At one time, streaming at 1440p to YouTube was recommended as a why to bypass some re-encoding issues at YouTube (and force higher resolution/bitrate... ie, nicer looking videos.) Not being a gamer, I didn't follow and don't know if still current.

With my use case, and NVENC GPU utilization being and remaining low, I Record at one value (higher, for future editing/other), and Stream at another. This is 2X the encoding workload, but my system can handle it (and still barely get to 10-20% utilization)... so not an issue for me.

As for GPU resource utilization... that's a question a gamer would ask/pursue/test, sorry not really relevant for me so I don't have an authoritative answer... and I suspect it is not a single/clean answer... instead, I suspect its a 'it depends' type answer and will vary based on application, image dynamics, and more.
Then if you have base canvas and output at 1440p, (all else being similar) then streaming and recording bitrate will be higher, and lower local hardware utilization with avoidance of re-scaling. So which do you have more of? sort of.. think multi-variant calculus... figuring it out is doable, with patience. Sorry I can't give you an 'easy answer'.. maybe someone else can?