Having GPU Hardware Encoding Overload during streaming

[GameCubeGuy]

I've been experiencing encoding overloads during my streams as of late. I've tried several fixes on the internet but they don't seem to do the trick. I've already tried lowering my streaming quality from 1080p 60fps to 720p 60fps and the issues still persists. Even when i try lowering the bitrate from 6000kbps to 4500kbps its still an issue. Not entirely sure what's going, I know AMD gpus don't have the greatest encoder but i didn't expect it to cause this kind of issue. I've used lesser hardware and hadn't had this kind of issue before. I have noticed my vram continuously go up, all the way to the full 12gb in my GPU (RX 6750XT). Some of the fixes i've tried implementing include turning off hardware decoding on all of my 1080p video scenes, turning off browser source hardware acceleration (that's why it says that in the log), unloading images/videos/music when not in use, turning off hardware acceleration for my stinger transition, preloading the stinger into ram. Those are the fixes that i can remember, so if anyone has any ideas on how to help that would be greatly appreciated.

Log: https://obsproject.com/logs/L6WmOu3FPzCJRF8V

P.S: I also tried to use AMF/FFmpeg option commands to try and see if it would help, the issue was still occuring before i added them.

HighMotionQualityBoostEnable=false EnableVBAQ=false RateControlPreanalysisEnable=0 BReferenceEnable=true AdaptiveMiniGOP=false RateControlSkipFrameEnable=false EnablePreAnalysis=true PASceneChangeDetectionEnable=false PATemporalAQMode=1 PAFrameSadEnable=true

 

[qhobbes]

Lower the resolution on your camera.
If you can, change the refresh rate on your gaming monitor to 60, cap your games at 60, enable v-sync/freesync.
Limit the Max Video Tracks to 3 or less.

 

[GameCubeGuy]

Lower the resolution on your camera.
If you can, change the refresh rate on your gaming monitor to 60, cap your games at 60, enable v-sync/freesync.
Limit the Max Video Tracks to 3 or less.

So far i've lowered the camera resolution and lowered the resolution of the stream output to 720p. CPU usage is lower and the encoder overloads aren't as long though in the past i haven't had issues of the CPU being fully utilized or nor the GPU during my tests in the past. So far the encoder overload usually occurs around the 25 minute to 1 hour mark into the test, or whenever I close the game (testing the stream with Resident Evil 4 remake.) I'm thinking its because my vram usage just keeps going up, especially with games that just use at least 1gb. The log before was the test i did with 720p 60fps 4500kpbs output. I still have to test with the refresh rate to my monitor but i don't have high hopes.

https://obsproject.com/logs/TQ2f0YXNrjrv9gQy

 

[GameCubeGuy]

So far i've lowered the camera resolution and lowered the resolution of the stream output to 720p. CPU usage is lower and the encoder overloads aren't as long though in the past i haven't had issues of the CPU being fully utilized or nor the GPU during my tests in the past. So far the encoder overload usually occurs around the 25 minute to 1 hour mark into the test, or whenever I close the game (testing the stream with Resident Evil 4 remake.) I'm thinking its because my vram usage just keeps going up, especially with games that just use at least 1gb. The log before was the test i did with 720p 60fps 4500kpbs output. I still have to test with the refresh rate to my monitor but i don't have high hopes.

https://obsproject.com/logs/TQ2f0YXNrjrv9gQy

Alright i think i found a possible fix. Still gonna do further fixing but i was worried enabling hardware decoding on my scene sources would overload my GPU. Thankfully in 30min tests i've done its been able to handle itself. Will report if its a solid fix.

 

[rockbottom]

Rendering lag is GPU overload. If the rendering lag is eliminated the encoding lag will most likely go away too.

Disable HDR on your E22 monitor & run both monitors @ 60HZ. Test

Note: If HDR is already disabled, look at the Color Pixel Settings in the AMD Control Panel, link below.

17:41:57.045: Output 'rtmp multitrack video': Number of lagged frames due to rendering lag/stalls: 41 (0.3%)
17:41:57.045: Video stopped, number of skipped frames due to encoding lag: 41/12573 (0.3%)
17:41:57.047: ==== Streaming Stop ================================================
17:41:57.066: Video stopped, number of skipped frames due to encoding lag: 41/12574 (0.3%)
17:41:57.080: Video stopped, number of skipped frames due to encoding lag: 41/12575 (0.3%)
17:41:57.098: warning: 2 frames left in the queue on closing

17:35:10.648: name=SB220Q
17:35:10.648: pos={0, 0}
17:35:10.648: size={1920, 1080}
17:35:10.648: attached=true
17:35:10.648: refresh=75 > 60
17:35:10.648: bits_per_color=8
17:35:10.648: space=RGB_FULL_G22_NONE_P709
17:35:10.648: primaries=[r=(0.653320, 0.336914), g=(0.322266, 0.611328), b=(0.153320, 0.060547), wp=(0.313477, 0.329102)]
17:35:10.648: relative_gamut_area=[709=1.020525, P3=0.752302, 2020=0.539726]
17:35:10.648: sdr_white_nits=80
17:35:10.648: nit_range=[min=0.500000, max=270.000000, max_full_frame=270.000000]
17:35:10.648: dpi=96 (100%)
17:35:10.648: id=\\?\DISPLAY#ACR06AB#7&3674592b&1&UID264#{e6f07b5f-ee97-4a90-b076-33f57bf4eaa7}
17:35:10.648: alt_id=\\.\DISPLAY1
17:35:10.648: output 1:
17:35:10.648: name=E22
17:35:10.648: pos={1920, 0}
17:35:10.648: size={1920, 1080}
17:35:10.648: attached=true
17:35:10.648: refresh=75 > 60
17:35:10.648: bits_per_color=10 > 8
17:35:10.648: space=RGB_FULL_G22_NONE_P709
17:35:10.648: primaries=[r=(0.639648, 0.330078), g=(0.299805, 0.599609), b=(0.139648, 0.059570), wp=(0.313477, 0.329102)]
17:35:10.648: relative_gamut_area=[709=1.011584, P3=0.745711, 2020=0.534997]
17:35:10.648: sdr_white_nits=80
17:35:10.648: nit_range=[min=0.500000, max=270.000000, max_full_frame=270.000000]
17:35:10.648: dpi=96 (100%)
17:35:10.648: id=\\?\DISPLAY#SPT08D5#7&3674592b&1&UID268#{e6f07b5f-ee97-4a90-b076-33f57bf4eaa7}
17:35:10.648: alt_id=\\.\DISPLAY2

https://www.amd.com/en/resources/support-articles/faqs/DH3-021.html

 

[GameCubeGuy]

Rendering lag is GPU overload. If the rendering lag is eliminated the encoding lag will most likely go away too.

Disable HDR on your E22 monitor & run both monitors @ 60HZ. Test

Note: If HDR is already disabled, look at the Color Pixel Settings in the AMD Control Panel, link below.

17:41:57.045: Output 'rtmp multitrack video': Number of lagged frames due to rendering lag/stalls: 41 (0.3%)
17:41:57.045: Video stopped, number of skipped frames due to encoding lag: 41/12573 (0.3%)
17:41:57.047: ==== Streaming Stop ================================================
17:41:57.066: Video stopped, number of skipped frames due to encoding lag: 41/12574 (0.3%)
17:41:57.080: Video stopped, number of skipped frames due to encoding lag: 41/12575 (0.3%)
17:41:57.098: warning: 2 frames left in the queue on closing

17:35:10.648: name=SB220Q
17:35:10.648: pos={0, 0}
17:35:10.648: size={1920, 1080}
17:35:10.648: attached=true
17:35:10.648: refresh=75 > 60
17:35:10.648: bits_per_color=8
17:35:10.648: space=RGB_FULL_G22_NONE_P709
17:35:10.648: primaries=[r=(0.653320, 0.336914), g=(0.322266, 0.611328), b=(0.153320, 0.060547), wp=(0.313477, 0.329102)]
17:35:10.648: relative_gamut_area=[709=1.020525, P3=0.752302, 2020=0.539726]
17:35:10.648: sdr_white_nits=80
17:35:10.648: nit_range=[min=0.500000, max=270.000000, max_full_frame=270.000000]
17:35:10.648: dpi=96 (100%)
17:35:10.648: id=\\?\DISPLAY#ACR06AB#7&3674592b&1&UID264#{e6f07b5f-ee97-4a90-b076-33f57bf4eaa7}
17:35:10.648: alt_id=\\.\DISPLAY1
17:35:10.648: output 1:
17:35:10.648: name=E22
17:35:10.648: pos={1920, 0}
17:35:10.648: size={1920, 1080}
17:35:10.648: attached=true
17:35:10.648: refresh=75 > 60
17:35:10.648: bits_per_color=10 > 8
17:35:10.648: space=RGB_FULL_G22_NONE_P709
17:35:10.648: primaries=[r=(0.639648, 0.330078), g=(0.299805, 0.599609), b=(0.139648, 0.059570), wp=(0.313477, 0.329102)]
17:35:10.648: relative_gamut_area=[709=1.011584, P3=0.745711, 2020=0.534997]
17:35:10.648: sdr_white_nits=80
17:35:10.648: nit_range=[min=0.500000, max=270.000000, max_full_frame=270.000000]
17:35:10.648: dpi=96 (100%)
17:35:10.648: id=\\?\DISPLAY#SPT08D5#7&3674592b&1&UID268#{e6f07b5f-ee97-4a90-b076-33f57bf4eaa7}
17:35:10.648: alt_id=\\.\DISPLAY2

https://www.amd.com/en/resources/support-articles/faqs/DH3-021.html

Gotcha. I'll test these out later when i get the chance and report back. I checked my windows 11 settings and HDR isn't supported on either monitor anyway. Am i supposed to enable the custom color or have it disabled because its been disabled the whole time if that was a potential issue. Also i'll try testing with 60hz on my E22 monitor since that's the monitor i use to display OBS during streaming.

 

[rockbottom]

The Pixel Color setting is near the bottom. It's probably set to 10--bit, change it to 8-bit Full RGB.

Run both monitors @ 60. Forget 75HZ even exists in your settings when using OBS. FWIW, I only run 30, 60, 120, 240 & 360 when using OBS.

 

[GameCubeGuy]

Yeah this is the only option that showed for that 10 bit pixel format that you were talking about, unless i'm wrong. I'll try to find another guide in case its somewhere else. Will report back on further tests. Just about to test with the 60 hz refresh rate.

 

[GameCubeGuy]

View attachment 112671
Yeah this is the only option that showed for that 10 bit pixel format that you were talking about, unless i'm wrong. I'll try to find another guide in case its somewhere else. Will report back on further tests. Just about to test with the 60 hz refresh rate.

I did find this in the display settings, but its already set to 8, and 8 is the only option anyway.

 

[prgmitchell]

You may want to try turning off enhanced broadcasting in settings>stream and testing again.

 

[rockbottom]

The default monitor is the SB220Q so the 8-bit setting is correct, it was already there per your log above. Double check the settings for the the E22 (Monitor #2), that's the monitor that was running 10-bit color and you are correct, it's not an HDR monitor.

 

[GameCubeGuy]

The default monitor is the SB220Q so the 8-bit setting is correct, it was already there per your log above. Double check the settings for the the E22 (Monitor #2), that's the monitor that was running 10-bit color and you are correct, it's not an HDR monitor.

Got it. I'll do further tests to see if that will alleviate things.

 

[GameCubeGuy]

https://obsproject.com/logs/teShu7jpYr5AxAA7

^This was the test where i had my E22 monitor with the 8 bit color setting and with enhanced broadcasting off. I didn't come across any notifications for encoding overload the entire 1 and 22 minutes i tested it, though it was showing a decent of rendering and encoding lag based on the obs stats i had up during the test (Test done in 1080p 60fps 6000kpbs.)

https://obsproject.com/logs/5yJO6O7FEz29hw10

^This test was was done with 720P 60fps 4500kpbs. Lower CPU usage and the rendering lag appears just slightly lower i believe. I don't think i come across any encoding overload at all though the lag does appear at times. I'd rather have the lag than a straight up overload at least.

 

[prgmitchell]

https://obsproject.com/logs/teShu7jpYr5AxAA7

^This was the test where i had my E22 monitor with the 8 bit color setting and with enhanced broadcasting off. I didn't come across any notifications for encoding overload the entire 1 and 22 minutes i tested it, though it was showing a decent of rendering and encoding lag based on the obs stats i had up during the test (Test done in 1080p 60fps 6000kpbs.)

https://obsproject.com/logs/5yJO6O7FEz29hw10

^This test was was done with 720P 60fps 4500kpbs. Lower CPU usage and the rendering lag appears just slightly lower i believe. I don't think i come across any encoding overload at all though the lag does appear at times. I'd rather have the lag than a straight up overload at least.

Overload and lag are the same thing in this context, one causes the other. Now that you've sent a log without enhanced broadcasting on the fix is pretty obvious though, go to settings>output and change your preset from "quality" to "speed".

 

[rockbottom]

There's still some rendering lag in both logs, 0.2% & 0.3%. The log with the enhanced broadcasting had some nasty audio lag. This indicates that your system was overloaded during that test.

Disable these, they're not needed & can waste resources.
17:41:17.378: Game Bar: On
17:41:17.378: Game DVR: On

Anyway, there's still lots of SE/Java Script errors in the log. One or more of the those browser sources could be causing some pain. Try removing them one at a time.

17:42:42.133: ==== Streaming Start ===============================================
17:44:26.271: warning: Could not update timestamps for discarded samples.
17:44:26.298: warning: Could not update timestamps for skipped samples.
17:45:57.807: User switched to scene 'Main'
17:46:04.414: warning: Could not update timestamps for skipped samples.
17:46:20.372: Max audio buffering reached!
17:46:20.372: adding 917 milliseconds of audio buffering, total audio buffering is now 960 milliseconds (source: Desktop Audio)
17:46:20.372:
17:46:20.394: Source Desktop Audio audio is lagging (over by 176261.84 ms) at max audio buffering. Restarting source audio.

18:12:24.746: Source Desktop Audio audio is lagging (over by 87449.67 ms) at max audio buffering. Restarting source audio.


23:14:58.033: - scene 'Main':
23:14:58.033: - source: 'Background' (image_source)
23:14:58.033: - source: 'Mic/Aux' (wasapi_input_capture)
23:14:58.033: - filter: 'Noise Suppression' (noise_suppress_filter_v2)
23:14:58.033: - source: 'Game Capture' (game_capture)
23:14:58.033: - filter: 'Color Correction' (color_filter_v2)
23:14:58.033: - source: 'Chat' (browser_source)
23:14:58.033: - source: 'Blerp' (browser_source)
23:14:58.033: - source: 'EmoteWall' (browser_source)
23:14:58.033: - source: 'Alert' (browser_source)
23:14:58.033: - source: 'mouse' (image_source)
23:14:58.033: - source: 'keyboard' (image_source)
23:14:58.033: - source: 'camera' (dshow_input)

Try removing this one first.
17:41:18.888: [obs-browser: 'Blerp'] Error: [Report Only] Refused to load the script 'https://obs.blerp.com/cdn-cgi/speculation' because it violates the following Content Security Policy directive: "script-src 'none'".
17:41:18.888: (https://obs.blerp.com/pwn/ir6tshpld9cs:0)

 

[GameCubeGuy]

There's still some rendering lag in both logs, 0.2% & 0.3%. The log with the enhanced broadcasting had some nasty audio lag. This indicates that your system was overloaded during that test.

Disable these, they're not needed & can waste resources.
17:41:17.378: Game Bar: On
17:41:17.378: Game DVR: On

Anyway, there's still lots of SE/Java Script errors in the log. One or more of the those browser sources could be causing some pain. Try removing them one at a time.

17:42:42.133: ==== Streaming Start ===============================================
17:44:26.271: warning: Could not update timestamps for discarded samples.
17:44:26.298: warning: Could not update timestamps for skipped samples.
17:45:57.807: User switched to scene 'Main'
17:46:04.414: warning: Could not update timestamps for skipped samples.
17:46:20.372: Max audio buffering reached!
17:46:20.372: adding 917 milliseconds of audio buffering, total audio buffering is now 960 milliseconds (source: Desktop Audio)
17:46:20.372:
17:46:20.394: Source Desktop Audio audio is lagging (over by 176261.84 ms) at max audio buffering. Restarting source audio.

18:12:24.746: Source Desktop Audio audio is lagging (over by 87449.67 ms) at max audio buffering. Restarting source audio.


23:14:58.033: - scene 'Main':
23:14:58.033: - source: 'Background' (image_source)
23:14:58.033: - source: 'Mic/Aux' (wasapi_input_capture)
23:14:58.033: - filter: 'Noise Suppression' (noise_suppress_filter_v2)
23:14:58.033: - source: 'Game Capture' (game_capture)
23:14:58.033: - filter: 'Color Correction' (color_filter_v2)
23:14:58.033: - source: 'Chat' (browser_source)
23:14:58.033: - source: 'Blerp' (browser_source)
23:14:58.033: - source: 'EmoteWall' (browser_source)
23:14:58.033: - source: 'Alert' (browser_source)
23:14:58.033: - source: 'mouse' (image_source)
23:14:58.033: - source: 'keyboard' (image_source)
23:14:58.033: - source: 'camera' (dshow_input)

Try removing this one first.
17:41:18.888: [obs-browser: 'Blerp'] Error: [Report Only] Refused to load the script 'https://obs.blerp.com/cdn-cgi/speculation' because it violates the following Content Security Policy directive: "script-src 'none'".
17:41:18.888: (https://obs.blerp.com/pwn/ir6tshpld9cs:0)

Yeah i'm not sure why it still says xbox dvr and xbox game bar are still on, i already have them disabled in my settings. Gonna try disabling them in the registry.

 

[GameCubeGuy]

I just did a 3 hour stream at 1080p 60fps to test my limits. I did lower certain scenes to 720p 60fps, turned off enhanced broadcasting off, and i turned off xbox dvr off through the registry. Didn't have any encoding overloads whatsoever, though the analyzer does say i still have max audio buffering so i'll look into those other changes recommended later.

https://obsproject.com/logs/rvdj1ErKhrUOyX5v

 

[prgmitchell]

I just did a 3 hour stream at 1080p 60fps to test my limits. I did lower certain scenes to 720p 60fps, turned off enhanced broadcasting off, and i turned off xbox dvr off through the registry. Didn't have any encoding overloads whatsoever, though the analyzer does say i still have max audio buffering so i'll look into those other changes recommended later.

https://obsproject.com/logs/rvdj1ErKhrUOyX5v

If you aren't hearing any audio issues you can ignore it....the analyzer isn't really something that always needs attention, it is just meant to point you towards a solution when you are having issues.