Question / Help Any good AMD settings that replicate NVENC Low Lat-High Quality?

[SpectreKid]

Hi,

I am currently using an AMD card (RX 560) and I want saw many others use Low Lat High Quality settings on NVENC H.264, but I can't seem to replicate their settings with their same bitrate and get good or similiar quality to theirs.

If you didn't understand what I just said, this is an example.

Someone who has NVENC H.264 streams at 5500 bitrate on OBS, and enables Low Latency High Quality preset and Two-pass encoding. I stream at the same bitrate (5500 bitrate), but with AMD AMF options on, but without the Low Latency High Quality preset enabled, and without Two-pass encoding enabled (mainly because I cant figure out how to enable and the reason why I am asking a question in this thread). The problem is, I seem to get inferior quality, although the bitrate is the same, but on the AMD settings.

Is there any way to enable Low Lat High Quality and Two-pass encoding for AMD encoder, or any way to replicate the 5500 bitrate with Two-pass encoding, Low lat high quality preset and no rescale output on AMD AMF encoder? I see other people with Low lat high quality preset and two-pass encoding on NVENC and their 5500 bitrate or even 6MBPS streams look so good, but whenever I try to replicate their settings I get inferior quality by alot! PLEASE HELP!!!

- Best Regards,

OBS USER SPECTREKID

 

[koala]

Unfortunately, the AMD hardware encoder isn't able to deliver the same quality as NVENC, x264 or even Quicksync. The output looks always somewhat grainy or cloudy, regardless how high you tune up the bitrate or quality settings. This is not true for the other encoders. If you don't care about file size, they all allow to produce quality you aren't able to distinguish between the original and the recording.

If you want to judge yourself, this comparison might help: https://obsproject.com/forum/threads/comparison-of-x264-nvenc-quicksync-vce.57358/post-252221

 

[SpectreKid]

koala, thanks for the feedback! However, I do care about file size and actually is my weakest part: I know 5500 bitrate wont give me good record quality, but at least i can get stream quality. I have the Quicksync option available, so should I use that?

I did some testing with QSV and didn't see much difference: it was still pixelated. Idc about grain; only about pixellation. Should I go with QSV instead?

(P.S I don't touch x264 as I dont want to ruin any more performance, as my GPU is as average as can be)

THX koala, and pls reply!!

 

[SpectreKid]

Unfortunately, the AMD hardware encoder isn't able to deliver the same quality as NVENC, x264 or even Quicksync. The output looks always somewhat grainy or cloudy, regardless how high you tune up the bitrate or quality settings. This is not true for the other encoders. If you don't care about file size, they all allow to produce quality you aren't able to distinguish between the original and the recording.

If you want to judge yourself, this comparison might help: https://obsproject.com/forum/threads/comparison-of-x264-nvenc-quicksync-vce.57358/post-252221

(SORRY FOR THE RE-TEXT BUT I DIDNT REPLY AND I REPLIED MYSELF SO SORRY PLS DONT BAN)

koala, thanks for the feedback! However, I do care about file size and actually is my weakest part: I know 5500 bitrate wont give me good record quality, but at least i can get stream quality. I have the Quicksync option available, so should I use that?

I did some testing with QSV and didn't see much difference: it was still pixelated. Idc about grain; only about pixellation. Should I go with QSV instead?

(P.S I don't touch x264 as I dont want to ruin any more performance, as my GPU is as average as can be)

THX koala, and pls reply!!

 

[koala]

If you have Quicksync available, use Quicksync with a bitrate of 5500 and target use. You don't have much alternatives anyway, so this is the only option left for you. Use the defaults for the other Quicksync options.

On paper, x264 is better if you use a preset better than veryfast, i. e. faster or fast or medium, but this is only better if your CPU can cope with that. In most cases this will only stress the computer in a way that result in lags, bad gameplay for the captured game and lost frames ("encoding overload"), so the video will get worse than with Quicksync. Only if you have the fastest available CPUs, you can use x264 without these adverse effects. So if you have Quicksync, definitely use Quicksync. As far as I see, it is a better hardware encoder than the AMD encoder. There is another issue with the AMD encoder: its performance depends on some GPU resources. If the GPU is fully loaded by the game, the AMD encoder will be overloaded and lose frames. This will not happen for Quicksync or Nvidia's NVENC.

 

[SpectreKid]

If you have Quicksync available, use Quicksync with a bitrate of 5500 and target use. You don't have much alternatives anyway, so this is the only option left for you. Use the defaults for the other Quicksync options.

On paper, x264 is better if you use a preset better than veryfast, i. e. faster or fast or medium, but this is only better if your CPU can cope with that. In most cases this will only stress the computer in a way that result in lags, bad gameplay for the captured game and lost frames ("encoding overload"), so the video will get worse than with Quicksync. Only if you have the fastest available CPUs, you can use x264 without these adverse effects. So if you have Quicksync, definitely use Quicksync. As far as I see, it is a better hardware encoder than the AMD encoder. There is another issue with the AMD encoder: its performance depends on some GPU resources. If the GPU is fully loaded by the game, the AMD encoder will be overloaded and lose frames. This will not happen for Quicksync or Nvidia's NVENC.

Ok koala thx but I still have a couple questions if you don’t mind.

#1: How will Quicksync be better than AMF? Is there a different way do encoding the video to make it better?

#2: What should Keyfram interval be?

#3 what is a sync depth?

Those are the questions for now but thx man you have helped so much!!!

 

[SpectreKid]

If you have Quicksync available, use Quicksync with a bitrate of 5500 and target use. You don't have much alternatives anyway, so this is the only option left for you. Use the defaults for the other Quicksync options.

On paper, x264 is better if you use a preset better than veryfast, i. e. faster or fast or medium, but this is only better if your CPU can cope with that. In most cases this will only stress the computer in a way that result in lags, bad gameplay for the captured game and lost frames ("encoding overload"), so the video will get worse than with Quicksync. Only if you have the fastest available CPUs, you can use x264 without these adverse effects. So if you have Quicksync, definitely use Quicksync. As far as I see, it is a better hardware encoder than the AMD encoder. There is another issue with the AMD encoder: its performance depends on some GPU resources. If the GPU is fully loaded by the game, the AMD encoder will be overloaded and lose frames. This will not happen for Quicksync or Nvidia's NVENC.

Also CBR or VBR for 5500 bitrate recording for CBR? I will compress later if it is display capture but mostly it would be gameplay so that’s that. (Sry for making to separate replies but I forgot to ask this in original reply sry. But thx again you’re helping and you’re a great help!!!) :)

 

[koala]

Please don't ask for every single option. Keep all options at their default, unless you know you need to change some option, and you know what the impact will be. All options have sane defaults, you don't need to change. Only ever change what you need to change, only if you know what the result will be, and only if you really need the different result.

For streaming, use CBR, not VBR. If you don't stream but record, use CRF, CQP or ICQ. VBR has no widespread use case, as far as I know.

And for the question about how Quicksync will be better than AMF - I don't understand the question. I posted a comparison table with a bunch of identical videos, all encoded with different encoders and different resolution and fps. You can see it for yourself which video and which encoder is better than another. A more statistical approach with a rating table can be found on the first post of that thread, and as far as I remember some more tables of that kind were distributed over that thread.

 

[SpectreKid]

Please don't ask for every single option. Keep all options at their default, unless you know you need to change some option, and you know what the impact will be. All options have sane defaults, you don't need to change. Only ever change what you need to change, only if you know what the result will be, and only if you really need the different result.

For streaming, use CBR, not VBR. If you don't stream but record, use CRF, CQP or ICQ. VBR has no widespread use case, as far as I know.

And for the question about how Quicksync will be better than AMF - I don't understand the question. I posted a comparison table with a bunch of identical videos, all encoded with different encoders and different resolution and fps. You can see it for yourself which video and which encoder is better than another. A more statistical approach with a rating table can be found on the first post of that thread, and as far as I remember some more tables of that kind were distributed over that thread.

Sorry for asking about every option, but i jsut need to know what async depth is. I never really understood the input for it.

For the question about AMD vs QSV I think I will drop that question. You seem to already have answered my question already.

Another question though. What is ICQ? I think I have seen it before in Quicksync settings before, but never really used it. ICQ seemed "exotic," and I never really touch things unless they are common like CBR, VBR, CQP, ABR, or CRF. (My sanity doesnt go that far)

Thanks again koala for helping me with my settings. I will report back my experiences and see if they are good or not. THX AGAIN BRO

 

[koala]

ICQ is a rate control of Quicksync, similar to CRF on x264. It's available only on somewhat newer or selected Intel iGPUs.

 

[SpectreKid]

ICQ is a rate control of Quicksync, similar to CRF on x264. It's available only on somewhat newer or selected Intel iGPUs.

Ok thx bro! Btw I just mentioned you about AMD settings on another thread (even tho that’s off topic.) I’ll report back what I get and quality! You help much bro thanks