Question / Help Sudden "Encoding Overloaded" on EVERYTHING

[Juliannb]

Greetings, everybody!
A friend streamer and I are both having issues with OBS 21.1.2, we're encountering this issue when he streams and I record: Encoding overloaded.
Starting with my fellow streamer first, he's having this issue whenever he streams highly demanding games, such as Overwatch, Dead by Deadlight and Far Cry 4, just to name a few, on his brand-new PC. Here's his build: https://pcpartpicker.com/list/bXVNcY
As you can see, he can clearly do gaming and streaming on the same computer without any issues, and he did! He was streaming just fine, but then all the sudden, the encoder would overload without any reason. I don't have any log files or anything, but I helped him with the settings. The encoder he uses is H264/AVC Encoder (AMD Advance Media Framework), preset Twitch, at 720p 30 FPS, rescaled from 1440x900. He has two monitors, which he used on an old build that was half as powerful and could stream without this issue, and now it seems impossible to have a fluent experience whatsoever.
Now, my issue seems to be random and sometimes it makes no sense. I have a quite similar build as he does: https://pcpartpicker.com/list/D68Tmq
As it's shown in the log, I have been using these settings for quite a while and they haven't given me any issues... that was until last week. I was getting random encoding overloads on games that demand little to nothing, and sometimes in games like Overwatch, I wouldn't suffer any performance problems, although I still have some other random cases, and EVEN WITHOUT ANY SOURCES! I recorded a blank screen and it would overload! I changed the resolution, the framerate, but no matter what I'd do, the overloading would still happen.
I tried to look up this issue, but no such luck. I thought to record using x264 encoding, but with my demand... I can't get it to work at all, that's why I switched to H264. I also tried H265, and the issue would happen in less successions (and also, I don't want to use it, for editing purposes). I even tried an older version of OBS... I'm running out of ideas.
At the moment, I'm not using the latest AMD drivers, because they could cause issues with my PC, I'm sticking to the AMD Radeon Software 18.3.4 which works fine for all my daily needs.

And that's about it. If there's any information you need from me, please, let me know and I'll try to provide it as soon as I can.

Thanks in advance!

 

[Patrick Ewalt]

You really should consider updating your drivers to 18.6.1 even in the 18.4 series major fixes were made to the AMD AMF backend in the drivers that are very performant and could alleviate some of your issues, I'm assuming you are both using the same driver build probably. Also try to run OBS as an Administrator whenever possible as it can help rule out other issues with contention or access rights particularly for some capture.

ALSO of big note: DO NOT USE MONITOR CAP & GAME CAPTURE WITHIN SAME SCENE! Even when the eyeball isn't clicked the source is running, this will cause major conflicts and random dropouts. Delete the monitor capture, in fact create a new collection for just the monitor capture on it's own, keep it out of your game capture collection if you can help it. Particularly now since the latest windows update, having both of those sources in the same collection caused me much grief until I just cleaned out my scenes and collections and now things are champ. Let me know if you still need help or have further questions, will do my best!

On another note, you had network issues, you may wish to run some network tests to determine your bandwidth as well las any latency/jitter or packet loss. the log indicates high levels of packet drops to the network...

Oh yeah after you get it cleaned up, restart OBS, you don't have to, but I like working with it fresh and clean after making such changes.

 

[Juliannb]

I know I should, but take a look at this thread I opened at AMD's forum: https://community.amd.com/message/2865982#comment-2865982 (it is such a mess, even the AMD staff have the same issue, but luckily they're working on it)

Hah, my bad, I was so frustrated with the whole situation I forgot to delete one of them. I can't say "well, this happened even before that" because the log speaks for itself, so once I'm back to my PC, I'll (try to) recreate the problem and send the log. So I shall stay away from Game Capture? Roger that, I'll keep that in mind.

Wait, how did the stream test get into the log? I closed and opened OBS to get a new log... Well! That's awkward, hehe. Still, my streaming issues are not the main point here, but I appreciate your concern about it.

Sure thing! Thank you very much for your message, Patrick! Like I said before, I'll remove the Game Capturer source and try to recreate and generate another log file later this day.

 

[Patrick Ewalt]

Julainnb, No problem! If you notice at AMD forums, I'm highly ranked there as welll, same name and avatar if you search me you'll find lots and lots of posts. :) If you need anything you can reach out to me on my discord! https://discord.gg/9CgFMaS

 

[Deleted member 121471]

Did you accidentally enable SLI/crossfire option in your "game capture" source properties?

Also, the previous user suggested not keeping both in the same scene, you're supposed to use game capture to cap your games, it has the lowest performance hit out of all capture methods.

 

[Juliannb]

Alright, I removed the Display Capture source, then I recorded Castle Crashers again and... the encoder would still overload.
Then I checked out what Volfield said and, for whatever reason, the SLI/Crossfire option was ON... I have no idea why, I only have one GPU on my PC. Either way, I disabled it and the moment I hit record, I'd overload the encoder, again... I haven't tried with the Display Capture source alone yet, just because I fear it's going to happen again.
I attach today's log. The first recording was with the SLI/Crossfire option enabled on the Game Capture source, the second one is disabled. In fact, I'll attach a screenshot of the current settings on the Game Capture source just in case.

Thanks so much again for the comments!

EDIT: Well, this is what happens when I'm close minded: Castle Crashers seems to be the only one with this issue, but I'm going to keep on trying different games. The Binding of Isaac Afterbirth † seems to work just fine with Game Capture. I'll keep trying other games, but I think we got this.

 

[Patrick Ewalt]

Sounds good, let us know. Yeah some games, just....some games. I've been testing all day with different games and it's just funny how some can run amazing and capture, and some of them run like crap even though they have like no fidelity or harcore graphics. Sometimes it's just poor optimization or methods that don't like each other in terms of output from the game and then capture into OBS. Good luck, let us know!

 

[jimmy213]

I have this problem too. I opened OBS to capture some footage and the final clip is bad (I think it's called frame skipping). Here is an example.
I'm using the latest version of OBS, Windows 10 and GPU drivers (18.6.1).
Yes, my PC is pretty weak but here's a clip a few months ago (using the same presets and settings, I haven't changed anything).

 

[Patrick Ewalt]

I see the Encoder stall, the cause is not clear, I really believe we are seeing all these problems due to last windows update and whatever changed in DX or WDDM, something is affecting this...

 

[boothemav2]

Juliannb, you said you're using Windows 10. Have you tried telling Windows 10 using the task manager in "Details" to give higher CPU importance to OBS or, or trying to set the game(s) giving you issues, plus OBS to be given higher CPU importance? This is just an idea that I'm having after seeing a YouTube video lowering the CPU importance of the game(s) that's having issues.

I have this same issue and I have what I call a decent/good PC for gaming and streaming, parts below. I also get random blue screens while streaming. Last one happened after OBS crashed and I reopened it while playing Overwatch.
CPU: AMD Ryzen 7 1700
GPU: AMD RX 580
RAM: 16GB DDR4