OBS branch with AMD VCE support.

R

rhole

New Member
Here are my settings. I have even tried 3500k. Results are quite brilliant. With no stuttering over time.

Video tab - http://imgur.com/cjdHTGN
VCE tab - http://imgur.com/t6jDF8P

I use a Radeon 7970.

In the VCE tab set the I Pic Period and GOP size to the same as the capture Frame Rate. I have no idea what the IDR period is yet.

I set my preset to Slower since I assume when using x264 this would produce the best quality. However it probably has no effect on the VCE encoder.
I find that lowering the Speed Vs Quality in the VCE tab seems to lower spikes in bitrate. Since it currently doesn't stick to the bitrate you set. Try lowering it if you are still having issues as I honestly don't see that much difference in quality between the settings. 60 produces spikes up to 12Mbit for me at 6k. While my upload can take it (20Mbit) it is certainly not acceptable for steaming. 50 lowered the spikes to around 9k. This is only what I observed from a couple of test sessions. So could be put down to coincidence. More testing is certainly needed.
 
D

dping

Active Member
rhole said:
More testing is certainly needed.
Click to expand...

thanks for giving me a starting point. That is probably what it is then, I will set it to 60 with a 3k bitrate and see what i can get. my upload is around 6Mb so I could assume the blocks are from dropped frames that cause the blocks when it tries to send more frames than my upload will allow.
 
D

dping

Active Member
rhole said:
Here are my settings. I have even tried 3500k. Results are quite brilliant. With no stuttering over time.

Video tab - http://imgur.com/cjdHTGN
VCE tab - http://imgur.com/t6jDF8P

I use a Radeon 7970.
More testing is certainly needed.
Click to expand...

Since We have almost identical cards (I have a R9 280X) I have some settings for you to try. Now this is by memory so please forgive me if I get a few off. First, Your settings were years ahead of mine since I was still playing a lot with no success. So ty for that. I tried your default settings but felt I could push out a little more quality at the 3.1k bitrate and below was my results with the highest quality in a FPS with fast moving scenes.

For my tests I used 3.1k bitrate and a 4.6kbuffer (this is mainly use to my limited upload speed) but I cant see needing over 4k bitrate with this quality (i.e. 720p @ 48fps)

I used your settings minus CABAC then raised the MFT to 90% (can do 100% but not noticeable difference in quality and some minor studdering). CABAC raises encoding usage quite a bit, and decoding a little on the client side. is supposed to raise quality and more effecient but I have only found it to limit the MFT quality.

from what I understand CABAC is called "lossless" compression, but the lossless is always based on the input quality (i.e. MFT quality).

I played with 8-16 reference frames which gave me outstanding quality (16 being the max) and settled on 12 which gave The best quality that I could push as 3 just just to little.

I played with IDR period and found it best at either 12 or 1. 12 is just a max of 4 IDR frames a second (IDR is used for scene changes but tend to lower quality if used a lot. A setting of 1 uses IDRs on every detected scene change which could be accidentally overused during fast moving scenes.

ToDo:
Play more with I Pic Period
Disable SATD?
Play more with CABAC and MFT quality
GOP size of 2x fps (i.e. 96) https://obsproject.com/forum/threads/how-to-change-your-keyframe-interval-for-twitch.6481/
 
Last edited:
OliverMD

OliverMD

Member
@dping: Short question. Why aren't you just using openCL?
Just like you I've been using MFT at first since CL didnt quite wanted to run as i did.
Changed to CL a few days ago finally after Jack told me and holy moly i love the performance boost.
Even more since i don't have to fight around with any custom settings or anythign and always get top quality.

Is there any difference that MFT is better for streamer or something?
 
D

dping

Active Member
@OliverMD Using OpenCL for NV12 conversion fails to initialize. Also, checking the box for using custom settings says something about YUV444 error, yet unchecked, the custom settings seem to work.

I just figured it was an alpha build and OpenCL didn't work. using 10 Aug4 2014 64bit build btw.
 
OliverMD

OliverMD

Member
dping said:
@OliverMDUsing OpenCL for NV12 conversion fails to initialize. Also, checking the box for using custom settings says something about YUV444 error, yet unchecked, the custom settings seem to work.
Click to expand...
Interesting. When i use openCL i can't have "Custom Settings" checked or else it fails to initialize and won't work.
But if i have it unchecked it works perfectly fine. Running on the up2date version 2014-07-20

dping said:
says something about YUV444 error
Click to expand...
You know you have to uncheck MFT and only have CL checked right? ^^
When i check both MFT and CL boxes it tells "YUV444 colorspace is not supported by VCE" too.
 
R

rhole

New Member
dping said:
Since We have almost identical cards (I have a R9 280X) I have some settings for you to try. Now this is by memory so please forgive me if I get a few off. First, Your settings were years ahead of mine since I was still playing a lot with no success. So ty for that. I tried your default settings but felt I could push out a little more quality at the 3.1k bitrate and below was my results with the highest quality in a FPS with fast moving scenes.

For my tests I used 3.1k bitrate and a 4.6kbuffer (this is mainly use to my limited upload speed) but I cant see needing over 4k bitrate with this quality (i.e. 720p @ 48fps)

I used your settings minus CABAC then raised the MFT to 90% (can do 100% but not noticeable difference in quality and some minor studdering). CABAC raises encoding usage quite a bit, and decoding a little on the client side. is supposed to raise quality and more effecient but I have only found it to limit the MFT quality.

from what I understand CABAC is called "lossless" compression, but the lossless is always based on the input quality (i.e. MFT quality).

I played with 8-16 reference frames which gave me outstanding quality (16 being the max) and settled on 12 which gave The best quality that I could push as 3 just just to little.

I played with IDR period and found it best at either 12 or 1. 12 is just a max of 4 IDR frames a second (IDR is used for scene changes but tend to lower quality if used a lot. A setting of 1 uses IDRs on every detected scene change which could be accidentally overused during fast moving scenes.

ToDo:
Play more with I Pic Period
Disable SATD?
Play more with CABAC and MFT quality
GOP size of 2x fps (i.e. 96) https://obsproject.com/forum/threads/how-to-change-your-keyframe-interval-for-twitch.6481/
Click to expand...

Thanks for the extra info here. I have copied the IDR period and disabled CABAC. Set MFT to 90 and also set bitrate to 3.1k and buffer to 4.6 just to see what kind of quality can be got from this. Will be testing for the rest of the night.

I was under the impression MFT was faster than OpenCL? I set MFT and notice no loss in FPS so I am a little mythed at the report of someone using OpenCL with faster results. Also, if we cannot have custom settings then wouldn't this compromise the quality anyway? I currently find MFT to be stable for me.
 
T

toxify

New Member
OliverMD said:
Interesting. When i use openCL i can't have "Custom Settings" checked or else it fails to initialize and won't work.
But if i have it unchecked it works perfectly fine. Running on the up2date version 2014-07-20

You know you have to uncheck MFT and only have CL checked right? ^^
When i check both MFT and CL boxes it tells "YUV444 colorspace is not supported by VCE" too.
Click to expand...

i've tried with both ways you said always get encoder failed to initialize
 
D

dping

Active Member
OliverMD said:
Interesting. When i use openCL i can't have "Custom Settings" checked or else it fails to initialize and won't work. Running on the up2date version 2014-07-20

You know you have to uncheck MFT and only have CL checked right? ^^
When i check both MFT and CL boxes it tells "YUV444 colorspace is not supported by VCE" too.
Click to expand...

I do realize I cant have both MFT and OCL checked. I get some sort of error any time Custom settings is checked (any mode) and/or OpenCL is checked (whether custom is checked or not). the only mode that works for me is MFT with Custom unchecked.

I will revert to 07-20. So is 07-20 like stable and 08-10 unstable as in terms of linux builds? :)

No, I know, its all Alpha as ATM.
 
D

dping

Active Member
@jackun Any thoughts on the above with build from 2014-08-10? I'm using a VCE 1.0 GPU (R9 280X) cant seem to enable OpenCL to NV12 instead of MFT; MFT works fine as long as custom settings disabled. I'll have to get home and pastbin the logs, exc.
 
L

langtugl

New Member
First, i would like to say thank you very very much jackun, without ur hard word i'm unable to streaming my game.
But i have a question.
I have seen tobiwan dota 2 stream, 1080p 50fps everage details at just 3000 kbps, no or little block when moving camera, or another local stream 1080p 30fps at 1500 kbps, little little block. Seem they encode using CPU.
But i'm trying setting obs with 3000kbps 1080p 30fps, details are good but to many block when moving camera, immobile area like HUD is more details than 2 stream above.
So the question is the quality of encode using GPU vs using CPU? How can i reduce detail to reduce block while moving camera?
I try reduce resolution but seem not affective.
Sorry for my English.

P/S:like me, 07-20 build seem stable than 08-10 build.
 
bwat47

bwat47

New Member
@dping

I have a 280x and opencl to nv12 works fine here with the 8-10 build

I also have the box for use custom settings checked and it seems to work
 
R

rhole

New Member
Just a quick idea for those with the Error about the encoder not able to be initialised. Make sure the Ref Frames is not more than 4 or 5 when custom settings are enabled. This seems to give that error.
 
You must log in or register to reply here.
Top