Question / Help Slow x264 preset no longer works

[blender]

I have a 2700x CPU and RX 5700 graphics card. When I had an nVidia card, I was able to use slow x264 preset with no issues at 900p/6000Kb/s. My CPU load would go up to like 80%, but it worked pretty good.

But now that I have RX 5700, I get 80% skipped frames due to encoding lag at same resolution and same 6000Kb bitrate. And my CPU stays around like 30% during the encoding with all the missed frames. Encoding works fine with medium preset though.

I miss the slow encoding because it improves quality at low bitrates, why doesn't it work anymore?

 

[Deleted member 121471]

Post the logfile of a streaming or recording session.

https://obsproject.com/forum/threads/please-post-a-log-with-your-issue-heres-how.23074/

 

[blender]

Post the logfile of a streaming or recording session.

https://obsproject.com/forum/threads/please-post-a-log-with-your-issue-heres-how.23074/

I added it to my original post.

 

[Narcogen]

22:07:58.408: Loading up D3D11 on adapter AMD Radeon RX 5700 (0)
22:07:58.421: D3D11 loaded successfully, feature level used: b000
22:07:58.421: D3D11 GPU priority setup failed (not admin?)

Does running as admin affect OBS CPU priority?

 

[blender]

22:07:58.408: Loading up D3D11 on adapter AMD Radeon RX 5700 (0)
22:07:58.421: D3D11 loaded successfully, feature level used: b000
22:07:58.421: D3D11 GPU priority setup failed (not admin?)

Does running as admin affect OBS CPU priority?

How do I check this?
I tried running OBS as admin and the log said: "D3D11 GPU priority setup success", but I still got 55% skipped frames.

 

[Narcogen]

That's GPU priority. I'm wondering what CPU priority you have OBS set to in Windows, and whether that is affected by running as admin, as GPU priority is.

 

[Deleted member 121471]

This is the first time I ever heard of a non-overclocked 2700x handling 900p@60FPS@slow preset without issues, on a single computer setup, unless games or other software light on CPU load somehow allow it to work well, which would still surprise me.

Unless the AMD drivers installed or NVIDIA ones removed are causing some weird issue, changing graphic cards should have no significance whatsoever on CPU encoder load, assuming everything else is the same.

It's also a sort of streamer myth that "slow" encoding is the unicorn of picture perfect settings, much better than anything else. In reality, it's very marginally better than "medium" in terms of image quality, in the low single digit % of improvement, at the cost of a much higher CPU performance hit.

Personally, after following @Narcogen suggestion, I'd also change to the following:

1) Output category

Rate Control: CBR
Bitrate: 6000 Kbps
Keyframe Interval: 2
CPU preset: Medium
Profile: High
Tune: None, Animation or Film (needs testing)
x264: The only setting that sometimes helps here is "threads=10" or "threads=12" but it requires testing. This command helps guarantee some threads are always available for every other process currently active. Not strictly necessary, just an option.

2) Video category

Output resolution: 1600x900 or 1280x720 (cleaner image due to dividing evenly with 1440p base canvas)
Downscale Filter: Bicubic (consistent) or Lanczos (best but can be too sharp in some cases)

 

[blender]

That's GPU priority. I'm wondering what CPU priority you have OBS set to in Windows, and whether that is affected by running as admin, as GPU priority is.

In OBS advanced settings and in Windows task manager it says priority is normal, I've always left it at normal. Should I change it to something else?

This is the first time I ever heard of a non-overclocked 2700x handling 900p@60FPS@slow preset without issues, on a single computer setup, unless games or other software light on CPU load somehow allow it to work well, which would still surprise me.

Unless the AMD drivers installed or NVIDIA ones removed are causing some weird issue, changing graphic cards should have no significance whatsoever on CPU encoder load, assuming everything else is the same.

It's also a sort of streamer myth that "slow" encoding is the unicorn of picture perfect settings, much better than anything else. In reality, it's very marginally better than "medium" in terms of image quality, in the low single digit % of improvement, at the cost of a much higher CPU performance hit.

Personally, after following @Narcogen suggestion, I'd also change to the following:

1) Output category

Rate Control: CBR
Bitrate: 6000 Kbps
Keyframe Interval: 2
CPU preset: Medium
Profile: High
Tune: None, Animation or Film (needs testing)
x264: The only setting that sometimes helps here is "threads=10" or "threads=12" but it requires testing. This command helps guarantee some threads are always available for every other process currently active. Not strictly necessary, just an option.

2) Video category

Output resolution: 1600x900 or 1280x720 (cleaner image due to dividing evenly with 1440p base canvas)
Downscale Filter: Bicubic (consistent) or Lanczos (best but can be too sharp in some cases)

Yes, not all my games would run on the slow preset when I had nVidia card. But Overwatch and some other old games would run fine... going up to 85% cpu load though. It was just strange to me why the gpu changed caused slow not to work anymore.

I think I'll just stick to medium, since slow isn't worth it like you said. Thank you for your informative explanation.

In your recommended settings you said you use high profile, why is that? I read online to use main because high doesn't work on some devices, is this true?

 

[Deleted member 121471]

I think I'll just stick to medium, since slow isn't worth it like you said. Thank you for your informative explanation.

In your recommended settings you said you use high profile, why is that? I read online to use main because high doesn't work on some devices, is this true?

It's incredibly rare to find people still using devices that expect "main" profile. On the other hand, "high" profile allows for slightly better compression with negligible/non-existent performance hit on the encoder and is widely supported.

On my post, I forgot to mention to switch your colour space to "709", as it is properly supported now.

 

[blender]

It's incredibly rare to find people still using devices that expect "main" profile. On the other hand, "high" profile allows for slightly better compression with negligible/non-existent performance hit on the encoder and is widely supported.

On my post, I forgot to mention to switch your colour space to "709", as it is properly supported now.

So the settings you recommended ended up screwing something up lol.

These are the changes I made: profile from main to high, tune from none to film, color space from 601 to 709. I also tested threads=12, but it ended up giving 80% skipped frames due to encoding lag so I took it off.

After I messed with those settings, I started getting skipped frames due to encoder lag at settings that ran flawlessly before. Going back to my old settings doesn't even fix it, its like OBS glitched up. Now I keep getting skipped frames due to encoder lag and encoder overload notifications at quality settings that ran flawlessly before in same levels in same games.

I end up having to drop my resolution to 720p60 to avoid skipped frames/encoder overload. Once again I have 2700x.

For example, I am attaching log with a game running at 1080p60 Medium preset and getting 1.3% encoder lag skipped frames. This game would run flawlessly before at these settings! Going back to main/none/601 doesn't even fix it, it still gets up to 2% encoder skipped frames. It still gets about 0.3% encoder lag skipped frames at 900p, which was a breeze before. Other games are hit harder, for example, up to 3% skipped frames due to encoder lag at setting that ran flawelessly.

 

[Deleted member 121471]

Profile and colour space have minimal impact on encoder performance.

Resolution, FPS and CPU preset are the settings you need to tinker with until you find a sweet spot you are happy with. Tune is the very last setting you can try for that extra fine tuning but it's strictly optional, requires testing per game and it's only useful if your encoder can actually handle the extra load.

 

[carlmmii]

It was just strange to me why the gpu changed caused slow not to work anymore.

Simple. If you've upgraded your GPU, then your game isn't going to be limited as much by the GPU, meaning the game will be allowed to render more frames, which takes up more CPU load.

This can also happen if you turn on more CPU-intensive graphics settings. The only way to really have a CPU usage equivalent to before would be to limit your framerate in-game down to what you were getting before, with the exact same graphics settings... but that's pretty much the opposite of the entire purpose of upgrading your GPU.

Just out of curiosity, what was the Nvidia GPU you had before?

 

[blender]

Simple. If you've upgraded your GPU, then your game isn't going to be limited as much by the GPU, meaning the game will be allowed to render more frames, which takes up more CPU load.

This can also happen if you turn on more CPU-intensive graphics settings. The only way to really have a CPU usage equivalent to before would be to limit your framerate in-game down to what you were getting before, with the exact same graphics settings... but that's pretty much the opposite of the entire purpose of upgrading your GPU.

Just out of curiosity, what was the Nvidia GPU you had before?

I had GTX 1060 6GB. It is true that I did turn up the graphics, although I always play with 60fps cap.

However, one of the games where I was easily able to run slow preset at 900p60, I did not increase graphics in it, so the in game settings are completely the same and I'm getting 80% skipped frames from encoder lag while CPU load is only 40%. Another game where I was able to use slow however, the graphics remained the same, but fps went from 40 to 60.

Also, when ever I'm getting massive skipped frames due to encoder lag, my CPU isn't maxing it... its staying at 40% load while getting 40-80% skipped frames.

Also, I am now getting skipped frames from encoder lag at medium preset, and my CPU isn't maxing out either, it's like 60% load.

 

[blender]

Profile and colour space have minimal impact on encoder performance.

Resolution, FPS and CPU preset are the settings you need to tinker with until you find a sweet spot you are happy with. Tune is the very last setting you can try for that extra fine tuning but it's strictly optional, requires testing per game and it's only useful if your encoder can actually handle the extra load.

Thanks again for your help!

I have a couple questions if you can answer. So are you saying film tune increases encoder load? Also, I noticed adding bframes=0 really improves performance (Mixer says to add that). Does setting bframes to zero reduce quality by a lot, is the default bframes set to 3 at medium preset?

 

[Deleted member 121471]

Thanks again for your help!

I have a couple questions if you can answer. So are you saying film tune increases encoder load? Also, I noticed adding bframes=0 really improves performance (Mixer says to add that). Does setting bframes to zero reduce quality by a lot, is the default bframes set to 3 at medium preset?

Yes, it's only an optional setting, not really required at all but it does increase load. Usually, you find the settings where you're don't experience encoder lag and only after that, maybe try different tunes or just be satisfied with "None". In my experience, the benefit of tunes is more obvious in bitrate-starved scenes.

I think B-frames are set to 3 on all presets from very fast to medium but I could be wrong on this. They help more with compression than absolute quality, if I recall correctly, which is what you want for streaming, due to bitrate constraints imposed by each streaming service.

 

[blender]

I'm bumping this thread of mine because I did testing and uploaded test videos.

So I found an antique GTX260 (12 year old card) and put it in instead of my RX5700. This is the what I recorded at the following settings:
-1600x900 game and encoding resolution / no scaling
-60fps VSync game, 60fps OBS
-6000Mbits
-Slow Preset
-High Profile
-Film Tune
-Either 0% or 0.1% Skipped frames due to encoder lag (can't remember exactly)

And this is what I recorded when I put my RX5700 back in, with everything at EXACTLY IDENTICAL (including ingame graphics) settings:
-53% skipped frames due to encoding lag

My CPU is stock 2700x by the way. And testing with my previews GTX1060 had same results... no skipped frames due to encoding lag even at slow preset at 900p. I tested all this for months in various games.

I'm currently trying to get rid of my RX5700 and buy an NVIDIA card, because this is just garbage. I tested a bunch of games, and all have same results... encode flawelessly on NVIDIA and garbage with AMD. Other easy to run games give me 80% skipped frames, so TF2 wasn't even the worst affected.