Hi, I've just upgraded to a new Ryzen 1800x system, specifically with recording and streaming in mind. I've had some issues with Overloaded Encoder errors, regardless of what bitrate or quality settings I was choosing, until I came across the problem. It may be an issue with Ryzen, maybe Windows, maybe OBS, I have no idea, so this might be a little long-winded, but I want to include all my troubleshooting steps in an attempt to provide some usable information, or to help others having the same experience.
All of this was tested recording Battlefield 1 on Win 10 pro 64bit, with an 1800x clocked to 3.8ghz on all cores and 16gb of 3200mhz ram. I haven't had a chance to test this with other games yet, but I assume the results would be similar. Also here is my log from this current session I took the screenshots from:
https://obsproject.com/logs/NWtzzd4Kq1oP7jum
When I first started having issues, changing encoder settings made very little difference. After playing with these for a while, I decided to look into other possibilities. I checked my CPU usage after starting to record, and I noticed that only 4 threads were being used by OBS for encoding. Here is my CPU usage from before and after starting to record. You can clearly see the 4 threads which are being used.

After noticing this, I decided to have a play round with CPU affinity to see if I could manually set cores. So I set BF1 to my first 8 threads, and OBS to my last 8 threads. As you can see, this evens out the CPU usage and causes OBS to use the last 8 threads pretty evenly.

This issue now is, when I change Windows' focus back to BF1, OBS stops using all but one thread, and I'm sure you can guess what happens to my recordings. Once I focus back on other programs on my desktop, the CPU usage evens out to the 8 threads again.

Now when I go back to affinity settings and untick the thread which is being maxed out (number 10 in this case), OBS is then able to go back to using the remaining 7 threads for encoding, and I can even select additional threads for it to use with no further issues, as long as that offending thread is left unticked. It's not the same thread every time either...next time I start OBS it may be a different thread, but the results are the same and repeatable.

So there is an issue here regarding thread utilization. This could be caused by a number of things, but I wanted to put this information up in the hopes that it helps someone, or may help the OBS team find an issue somewhere. If anyone would like some more details or wants me to test anything else, let me know.
Thanks.
All of this was tested recording Battlefield 1 on Win 10 pro 64bit, with an 1800x clocked to 3.8ghz on all cores and 16gb of 3200mhz ram. I haven't had a chance to test this with other games yet, but I assume the results would be similar. Also here is my log from this current session I took the screenshots from:
https://obsproject.com/logs/NWtzzd4Kq1oP7jum
When I first started having issues, changing encoder settings made very little difference. After playing with these for a while, I decided to look into other possibilities. I checked my CPU usage after starting to record, and I noticed that only 4 threads were being used by OBS for encoding. Here is my CPU usage from before and after starting to record. You can clearly see the 4 threads which are being used.


After noticing this, I decided to have a play round with CPU affinity to see if I could manually set cores. So I set BF1 to my first 8 threads, and OBS to my last 8 threads. As you can see, this evens out the CPU usage and causes OBS to use the last 8 threads pretty evenly.



This issue now is, when I change Windows' focus back to BF1, OBS stops using all but one thread, and I'm sure you can guess what happens to my recordings. Once I focus back on other programs on my desktop, the CPU usage evens out to the 8 threads again.

Now when I go back to affinity settings and untick the thread which is being maxed out (number 10 in this case), OBS is then able to go back to using the remaining 7 threads for encoding, and I can even select additional threads for it to use with no further issues, as long as that offending thread is left unticked. It's not the same thread every time either...next time I start OBS it may be a different thread, but the results are the same and repeatable.


So there is an issue here regarding thread utilization. This could be caused by a number of things, but I wanted to put this information up in the hopes that it helps someone, or may help the OBS team find an issue somewhere. If anyone would like some more details or wants me to test anything else, let me know.
Thanks.