OBS branch with AMD VCE support.

[Vieron82]

Hi!

My R9 380 findings:

The VCE engine of Tonga is perfectly capable to record in 1080p60, try it yourself with some window capture of a fluid video on youtube for example.
Unfortunately the hardware is not capable to maintain this fluidity, quality encoding when the card is under heavy load (gaming).

- Driver issue?

- Is Tonga truly has a dedicated hardware VCE engine?
If yes, i'd say there is some architectural limitation, bottleneck (bandwidth constraints, frame buffer access latency).

If you have high fps then try to use vsync.
To record at 1080p60 when the fps is higher than 60 is quite hopeless...

I made a little test.
I opened the AMD Overdrive panel, and started to record in OBS VCE.
In 720p@30fps the clock speed of the Gpu and the Memory went to maximum (1010/1380 MHz), but the Gpu was idle
In 1080p@60fps the same happened with the clock speeds, but the Gpu activity went to 30%.
So i think there is no dedicated x264 encoding Hardware, or it is not working in Windows 10 with 16.1 Crimson driver.
By the way, i can record Subnautica when it runs over 100 Fps with 60 fps setting in OBS VCE, with no problen, until the Gpu have enough spare power to do the encode...
I am using Latency Constrained VBR, with 5000 kbit/s max bitrate in 720p@30fps and 30000 kbit/sec in 1080p@60fps.

Edit:
I have an Asus STRIX R9 380 OC 2GB.

 

[dping]

Nope, it killed my windows, because it also removed the USB and Sata drivers, so i have to revert to the last restore point.

fuck. so sorry. I should have asked.

 

[dping]

I made a little test.
I opened the AMD Overdrive panel, and started to record in OBS VCE.
In 720p@30fps the clock speed of the Gpu and the Memory went to maximum (1010/1380 MHz), but the Gpu was idle
In 1080p@60fps the same happened with the clock speeds, but the Gpu activity went to 30%.
So i think there is no dedicated x264 encoding Hardware, or it is not working in Windows 10 with 16.1 Crimson driver.
By the way, i can record Subnautica when it runs over 100 Fps with 60 fps setting in OBS VCE, with no problen, until the Gpu have enough spare power to do the encode...
I am using Latency Constrained VBR, with 5000 kbit/s max bitrate in 720p@30fps and 30000 kbit/sec in 1080p@60fps.

Edit:
I have an Asus STRIX R9 380 OC 2GB.

Just because its usage goes up doesn't mean that it isn't dedicated hardware. try host which will capture the encoding buffer on the host memory.

 

[dping]

Does anyone have experience fixing atikmpag.sys BSOD?
Sometimes the OBS replay buffer will crash I then kill obs.exe in process explorer, start up OBS again then turn on replay buffer which triggers a BSOD.

disable steam hardware decoding if you have a GCN 1.2 might help

 

[Andt42]

I get crash report. I got these before and they are back. I was playing Ghost Recon Phantoms. I don't know is it the game or what, because I use this AMD settings only in that game. Other games work with x264 setting.

My CPU is FX-8350 and GPU is HD 7870XT.

 

[Elensar286]

Hi!

My R9 380 findings:

The VCE engine of Tonga is perfectly capable to record in 1080p60, try it yourself with some window capture of a fluid video on youtube for example.
Unfortunately the hardware is not capable to maintain this fluidity, quality encoding when the card is under heavy load (gaming).

- Driver issue?

- Is Tonga truly has a dedicated hardware VCE engine?
If yes, i'd say there is some architectural limitation, bottleneck (bandwidth constraints, frame buffer access latency).

If you have high fps then try to use vsync.
To record at 1080p60 when the fps is higher than 60 is quite hopeless...

You seem to be correct. The recording is perfectly fluid on desktop or basic 3D applications, but any recent game just drops frames like nothing else. I'm running VBR 1080p60 32000kbps on my R9 380 G1 Gaming with 8320@4.7GHz, and with my settings in Fallout I average about 70FPS. The resultant video after frame doubling and drops ends up being about 40fps, which is a huge pain since my processor doesn't have an encoding block like Intel has Quicksync, so I really don't have any way to record 1080p60 even though my system can handle it.

 

[ZeroWalker]

Done many tests on r9 380.
And well basically i am very disappointed.

The FPS drop is huge even with 30fps recording,
if i use x264 and the CPU isn't limited there is basically no FPS drop at all.
I assumed a hardware encoder would have virtually no restrain on the System at all aside for extreme cases.
But as this is, i get much more out of using x264, even though that kills my CPU.

Worth noting is that it seems that drivers do improve.
When i did some tests with MSI Afterburner with VCE on Halo CE,
i think that if i used Crimson 16.1 Hotfix on 4k -> 1920x1200 i dropped to about 60-70fps at a certain spot.
But with Catalyst 15.1.1 it was somewhere around 40-50. If i remember the test correctly.
Worth noting is that i had 15.1.1 installed but i used the 16.1 dll files to force that onto that specific game.

Will do a retry soon just to see so i am not lying here.

EDIT:

With 16.1 Hotfix on 4k at 1920x1200 i recorded and my FPS dropped from about 160 to 60.
And with 15.1.1 on same settings my FPS dropped from about 160 to 40-43.

The drop is enormous, and VSR is to blame much for, using Gedosato produces quite less performance loss.
The improvement in recording is quite bit though,
so unless something is wrong with 15.1.1 then there seems to be some driver related issue.
One can always hope that issue is huge and the software sucks, and then they will fix it and we get a super boost;P

Worth noting is that even Lossless encoding with MSI Afterburner causes quite an FPS drop,
didn't test both, but on 16.1 Hotfix it's from about 160 to about 120 or something.

 

[dping]

I get crash report. I got these before and they are back. I was playing Ghost Recon Phantoms. I don't know is it the game or what, because I use this AMD settings only in that game. Other games work with x264 setting.

My CPU is FX-8350 and GPU is HD 7870XT.

did you change

Done many tests on r9 380.
And well basically i am very disappointed.

The FPS drop is huge even with 30fps recording,
if i use x264 and the CPU isn't limited there is basically no FPS drop at all.
I assumed a hardware encoder would have virtually no restrain on the System at all aside for extreme cases.
But as this is, i get much more out of using x264, even though that kills my CPU.

Worth noting is that it seems that drivers do improve.
When i did some tests with MSI Afterburner with VCE on Halo CE,
i think that if i used Crimson 16.1 Hotfix on 4k -> 1920x1200 i dropped to about 60-70fps at a certain spot.
But with Catalyst 15.1.1 it was somewhere around 40-50. If i remember the test correctly.
Worth noting is that i had 15.1.1 installed but i used the 16.1 dll files to force that onto that specific game.

Will do a retry soon just to see so i am not lying here.

EDIT:

With 16.1 Hotfix on 4k at 1920x1200 i recorded and my FPS dropped from about 160 to 60.
And with 15.1.1 on same settings my FPS dropped from about 160 to 40-43.

The drop is enormous, and VSR is to blame much for, using Gedosato produces quite less performance loss.
The improvement in recording is quite bit though,
so unless something is wrong with 15.1.1 then there seems to be some driver related issue.
One can always hope that issue is huge and the software sucks, and then they will fix it and we get a super boost;P

Worth noting is that even Lossless encoding with MSI Afterburner causes quite an FPS drop,
didn't test both, but on 16.1 Hotfix it's from about 160 to about 120 or something.

you're trying 4k on a 380 and streaming and expecting results great? I dont get you. anyway. definately try the old trick of using host instead of dx 9 or 11

 

[ZeroWalker]

I am simply doing 4K as the game i am testing is super old and doesn't take much usage.
Doing it in 4K makes it more like a normal game GPU usage.
Otherwise i will run around with 200fps+ all the time.

Will try, my last tests on this version of AMD VCE has not gotten me much of anywhere,
i get complete stutter and drops even during low usage.

EDIT:

Okej tried now.

4K to 1920x1200 basically fails.

1920x1200 normal works but it seems to drop about 28% of the frames compared to about 1.25% on OBS Studio.
Both had about same FPS drop, which i think was from like 200fps to 130fps where i did the test.

MSI Afterburner had a much severe hit, but it seems to be the Hooking causing an issue as well as even lossless drops a lot while OBS barely takes no hit if i use lossless.

I used Host btw.

 

[Megedi]

Hello, I have a few questions to improve the quality of my stream with OBS and AMD VCE.
How did the "GOP Size" affect the quality?
What settings for AMD VCE still have an effect on the quality when streaming with CBR setting?

My current settings are 3500kbit bitrate, 30FPS, CBR, Quality Preset in AMD VCE, GOP size 30, IDR 60, Min QP 18, Max QP 41, QP Delta 23-23-23-4.

Thank you for your help.

 

[ZeroWalker]

Mgedi,

I am no AMD VCE expert not about Streaming, but i can name a few things.

GOP Size it probably something you don't need to think much about, let it be at about the same as your Framerate or Default.

As for the QP settings, those can be ignored if you are using CBR, VBR etc,
it only works if you use Constrained QP (CQP), which completely ignored bitrate and only goes by a quality factor (which is the QP numbers).


IDR is probably not doing much either.
B-Frames, i don't know the Default but if your GPU uses them i think 3 is a good settings, at least in standard encoder terms it should be.

Then it's probably most about the Quality Preset, but as you have it to Quality there's not much to do there.

In my test the Quality compared to x264 superfast is quite bad on AMD VCE.
On bright and overall detailed spots it's doing fine, you don't really notice much quality loss (though it's there).
However for flat areas with similar colors or dark spots, then you can really see the blocking, it's kinda melting around.

 

[dping]

Mgedi,

I am no AMD VCE expert not about Streaming, but i can name a few things.

GOP Size it probably something you don't need to think much about, let it be at about the same as your Framerate or Default.

As for the QP settings, those can be ignored if you are using CBR, VBR etc,
it only works if you use Constrained QP (CQP), which completely ignored bitrate and only goes by a quality factor (which is the QP numbers).


IDR is probably not doing much either.
B-Frames, i don't know the Default but if your GPU uses them i think 3 is a good settings, at least in standard encoder terms it should be.

Then it's probably most about the Quality Preset, but as you have it to Quality there's not much to do there.

In my test the Quality compared to x264 superfast is quite bad on AMD VCE.
On bright and overall detailed spots it's doing fine, you don't really notice much quality loss (though it's there).
However for flat areas with similar colors or dark spots, then you can really see the blocking, it's kinda melting around.

Min/max qp is not ignored as it uses this as its cbr/vbr quality range. so it will attempt to use min QP for still/simple image and fall back to max qp when movement/complexity is in the scene

 

[ZeroWalker]

Min/max qp is not ignored as it uses this as its cbr/vbr quality range. so it will attempt to use min QP for still/simple image and fall back to max qp when movement/complexity is in the scene

Oh didn't know that, thanks.

 

[BioGenx2b]

@dping, how can I keep OBS VCE bitrate from wobbling above my max setting? I stream at 2500kbps and spikes to 2800 or or even 3000+.

Also, games that use a lot of VRAM are causing stuttering on OBS (out of memory I guess?) so I have to turn down some options so the stream isn't choppy at times, even lowering my resolution! Anything that can help with that specifically?

I'm running an R9 290X (Sapphire Tri-X OC) and an FX-9370 stock.

 

[dping]

@dping, how can I keep OBS VCE bitrate from wobbling above my max setting? I stream at 2500kbps and spikes to 2800 or or even 3000+.

Also, games that use a lot of VRAM are causing stuttering on OBS (out of memory I guess?) so I have to turn down some options so the stream isn't choppy at times, even lowering my resolution! Anything that can help with that specifically?

I'm running an R9 290X (Sapphire Tri-X OC) and an FX-9370 stock.

Change min qp to 22 or 23. this helped in my testing.

 

[BioGenx2b]

Change min qp to 22 or 23. this helped in my testing.

The Witcher 3 is a good example, I can't turn everything to ultra or OBS framerate starts to dip. The game is solid though. I'll give that suggestion a shot when I get home tonight.

Thanks!

 

[Elensar286]

Changing the engine type to host and disabling Direct3D Interop has made my videos considerably smoother for my 380, but I guess it would put slightly more load on the CPU.

However, it would appear OpenGL titles now get extreme recording corruption, so this is only really applicable to DirectX games.

 

[dping]

Changing the engine type to host and disabling Direct3D Interop has made my videos considerably smoother for my 380, but I guess it would put slightly more load on the CPU.

However, it would appear OpenGL titles now get extreme recording corruption, so this is only really applicable to DirectX games.

Glad to hear that.
openGL or D3D doesn't change which engine you can use. AMF engine is mainly where the buffer goes.

but you are saying with opengl titles you get corruption with host AMF engine or all modes?

 

[Elensar286]

Glad to hear that.
openGL or D3D doesn't change which engine you can use. AMF engine is mainly where the buffer goes.

but you are saying with opengl titles you get corruption with host AMF engine or all modes?

Ah, looks like I was completely wrong. Regular OpenGL games seem to be working, just the rhythm game 'Osu!' appears to corrupt. Could be due to the ridiculously high frame rate. It's not really a game I play often, so it's not a problem.

 

[ZeroWalker]

Can anyone try a GPU intenstive game with OBS and MSI Afterburner and compare.

My results are that MSI Afterburner usually have a higher impact on the Game Framerate,
but it captures at that given framerate.

While OBS has lower impact but drops A Lot of frames compared.

(With MSI Afterburner i had best results with Direct3D9, even though Host had lower impact the resulting file had much less fps).