AMD GPU Encoding forces 4:2:0 color range no matter the settings

[CanCeralp]

Hello,
I have been using OBS Studio with H265 with my AMD 6700XT for a while now. Recently I wanted to up my settings a little bit more. I wanted to record with 4:4:4 at full range so I could color grade my gameplay videos better. However the "video" section in the Advanced options work only for x264 (CPU recording). AMD options of H265 and H264 force 4:2:0 partial no matter what I try.
My computer screen is set to RGB(Full) in AMD Radeon settings. I also tried YCbCr 4:4:4 but didn't change anything.

Here are the possibilities I tried:
1) NV12 - 709 - Partial --> Things are close to what is on screen. But the color is in 4:2:0 partial format. It causes banding issues and harder to apply LUTs, etc.
2) NV12 - 709 - Full --> This is what I think at least should be working. But the result is lots of black and white crashes with extreme contrast. I thought Color range should be equal to monitor's color setting but OBS disagrees with me.
3) I444 - 709/sRGB - Full --> the result is the same as number 1. 4:2:0 Partial.
4) RGB - 709 - Full --> encoding overloaded. And even the computer got frozen once. I don't know why.
5) RGB - sRGB - Full --> Range is finally full with no black/white crashes and things seem OK. But the Color Format is still in 4:4:4.

Long story short I have failed to understand 2 things:
1) Why can't I record at 4:4:4 format?
2) Why doesn't full range work, or gives crashed contrast even though my screen is also set to RGB Full?

Any help is appreciated. Thank you.

One more additional question. What is Color space? I can understand the format is basically how many bits are saved per pixel and Color range determines a color can have either 256 or 220 values, but what does Color space do? Is it a conversion function like Gamma, or Tonemapping?

Thank you, again.

 

[koala]

I cannot comment on your issue with the AMD encoder not producing 4:4:4 material (I don't have an AMD GPU), but I can explain what color space and color format are.

For performance and space reasons, textures are often not stored with their red, green, blue parts in 3 bytes but with a format that stores luma (brightness), chroma (color) and saturation. Luma is stored with the native resolution while chroma and saturation is stored with half resolution, because the human eye isn't able to see color information with the same resolution as just brightness. This is called YUV or NV12 format. Almost all video meant for consumption by an end user is stored in this format.

This format cannot store the same colors as (r, g, b). Some combinations of r, g, b cannot be represented in YUV format. What colors are represented and what are left out is defined with the Color Space. Color space 709 is the standard for digital video (TV), while sRGB is the industry standard for newer non-TV digital video. The difference is just the coefficients in the conversion formula, nothing special. Performance is the same.

If it comes to OBS, RGB format isn't hardware accelerated by the GPU, so you got encoding overload with 4) and 5).

If it comes to color range (partial or full), the standard for video is partial. Partial is a reduced color set to 235 colors instead of the full 256 colors per channel. This is also done for space saving. Some production tools don't properly mark full material as full and partial as partial, and some media players don't recognize full/partial info but instead play back everything as partial, even if it is full, so you sometimes get wrong colors if you create full material (washed out or hardened contrast).

By the way, with x264 and nvenc (Nvidia GPUs) it's definitely possible to create 4:4:4 material by setting color format to i444 (or RGB), so it's not a bug in OBS in general.

Keep in mind, regardless of what you create with OBS or with your postproduction tools, if you upload your product to some video service such as Youtube, and this video service recodes it (like Youtube), you will always get NV12 color format with partial color range. Even if you uploaded carefully crafted 4:4:4 full material. So it is often easiest with no visual degradation to just do everything as NV12 partial in the first place.

 

[CanCeralp]

I cannot comment on your issue with the AMD encoder not producing 4:4:4 material (I don't have an AMD GPU), but I can explain what color space and color format are.

For performance and space reasons, textures are often not stored with their red, green, blue parts in 3 bytes but with a format that stores luma (brightness), chroma (color) and saturation. Luma is stored with the native resolution while chroma and saturation is stored with half resolution, because the human eye isn't able to see color information with the same resolution as just brightness. This is called YUV or NV12 format. Almost all video meant for consumption by an end user is stored in this format.

This format cannot store the same colors as (r, g, b). Some combinations of r, g, b cannot be represented in YUV format. What colors are represented and what are left out is defined with the Color Space. Color space 709 is the standard for digital video (TV), while sRGB is the industry standard for newer non-TV digital video. The difference is just the coefficients in the conversion formula, nothing special. Performance is the same.

If it comes to OBS, RGB format isn't hardware accelerated by the GPU, so you got encoding overload with 4) and 5).

If it comes to color range (partial or full), the standard for video is partial. Partial is a reduced color set to 235 colors instead of the full 256 colors per channel. This is also done for space saving. Some production tools don't properly mark full material as full and partial as partial, and some media players don't recognize full/partial info but instead play back everything as partial, even if it is full, so you sometimes get wrong colors if you create full material (washed out or hardened contrast).

By the way, with x264 and nvenc (Nvidia GPUs) it's definitely possible to create 4:4:4 material by setting color format to i444 (or RGB), so it's not a bug in OBS in general.

Keep in mind, regardless of what you create with OBS or with your postproduction tools, if you upload your product to some video service such as Youtube, and this video service recodes it (like Youtube), you will always get NV12 color format with partial color range. Even if you uploaded carefully crafted 4:4:4 full material. So it is often easiest with no visual degradation to just do everything as NV12 partial in the first place.

Thank you for the clear and nice explanations. However, I have more questions.
It is obvious AMD GPUs' own behavior is forcing OBS to record in 4:2:0 so a hardware recording of 4:4:4 format is not possible with a Radeon GPU. AMD's own recording software, Relive, also records in 4:2:0.

However, I still can not understand why I get a contrast mismatch when I capture at full 0-255 range? My screen is set to 4:4:4 Full range RGB, technically the games are also working at 4:4:4 Full RGB format, so OBS settings also should be full range, right? I re-did all these test I said in the OP with settings my screen to YCbCr 4:4:4 as well, but I still get a contrast mismatch when OBS is set to Full range. All the advises I found on the internet says OBS color range should match the screen's and games' color range. In my case, this formula doesn't work.

BTW, I want my videos to have extra color information for archiving purposes. I occasionally make videos for Youtube as well, but I already export them with partial color range. The extra color information is helpful when color grading, applying effects, chromakeying or placing transparent overlays on the videos. When the source materials have reduced color information with a reduced color range, playing with the colors becomes problematic.

Basically, I can settle for 4:2:0 for a high performance high resolution capturing of a game, but it would be nice to have full range on the color information. So, if you guys can help me with that, I'd appreciate it.

 

[koala]

How did you test your material so you came to the conclusion your material isn't full range? There are media players that don't properly play back full material. If you use sophisticated media players, make sure you don't have any postprocessing active within the media player that may change/distort the colors. Also make sure you don't have any function active within your gpu driver that "enhances" video. Nvidia does have such a function, may be AMD as well.

 

[CanCeralp]

First I noticed the problem on my texts with naked eye, then I started researching. I tested with two methods:
1) Mediainfo software.
2) Here, in this forum there an excellent guide. I used the charts from there and double confirmed that the videos were indeed 4:2:0.

AMD Radeon Settings has such video enhancing feature, BTW. But they are disabled. Plus, NV12 + 709 + Partial option of OBS matches the desktop color tone 99%. If video enhancing software was the problem, it would have effected those videos, too and none of the settings would give accurate results.

So, there is must be some other explanation.

 

[Suslik V]

There is no 4:4:4 surfaces (but RGB) in the AMD AMF SDK. This means there is no 4:4:4 subsampling for AMD HW-encoders as for today. The RGB is very large, thus it is not for the H.264, H.265 etc. No wonders in this world. The 4:2:0 is forced because other - not supported.

 

[CanCeralp]

There is no 4:4:4 surfaces (but RGB) in the AMD AMF SDK. This means there is no 4:4:4 subsampling for AMD HW-encoders as for today. The RGB is very large, thus it is not for the H.264, H.265 etc. No wonders in this world. The 4:2:0 is forced because other - not supported.

Thank you. This means there is no point in trying anymore for an AMD hardware recorded 4:4:4 footage. Do you, by any chance, know why my recordings with full color range are broken in contrast?

 

[Suslik V]

File example needed (original), about 1 sec long, with 1280x720_color_range_test_chart.png in it. The 709+NV12+Full, 1280x720 base canvas and output (no rescale output), encoder standalone (NOT same as stream), should be set in OBS.

If I'll be unable to answer, all were answered above - wrong player or not supported.

 

[CanCeralp]

Here is Radeon Settings

Spoiler

Here is OBS settings

Spoiler

Here is the color chart I used, the canvas is 2560x1440 but the image itself is 1280x720 unscaled (100%)

Spoiler

And the videos:

Spoiler

4 videos from Google Drive they captured excatly what you see in the color chart screenshot

I recorded with x264 and AMD HEVC codecs. Both of them have full and partial samples, thus 4 videos total. Partial videos look excatly like the screenshot above. Full videos look darker with crashed contrast. All 4 videos are 4:2:0.

 

[koala]

For me, all 4 videos look 100% identical. Contrast, brightness, colors all identical, and all look also identical to the original *.png color chart I have on disk. I tried Media Player Classic as media player as well as ffplay.exe. You created flawless videos.

If there is the expected tiny color difference between partial and full I didn't check, because this would require comparing the difference with some painting app.

However, with the integrated Windows media playing function, I do see wrong contrast. Best visible on the two black bars in the bottom left corner. They look distinct (one is really black and one is dark grey) with MPC and with ffplay, while both are identical black with Windows media playback function on the 2 full videos, so I assume the Windows playback function cannot correctly play back this full material.
As far as I see, this means your media player is at fault and your videos are correct, and this means OBS did correctly record them.

And because of all that, I continue to recommend recording as partial.

 

[CanCeralp]

For me, all 4 videos look 100% identical. Contrast, brightness, colors all identical, and all look also identical to the original *.png color chart I have on disk. I tried Media Player Classic as media player as well as ffplay.exe. You created flawless videos.

If there is the expected tiny color difference between partial and full I didn't check, because this would require comparing the difference with some painting app.

However, with the integrated Windows media playing function, I do see wrong contrast. Best visible on the two black bars in the bottom left corner. They look distinct (one is really black and one is dark grey) with MPC and with ffplay, while both are identical black with Windows media playback function on the 2 full videos, so I assume the Windows playback function cannot correctly play back this full material.
As far as I see, this means your media player is at fault and your videos are correct, and this means OBS did correctly record them.

And because of all that, I continue to recommend recording as partial.

Interesting. Thank you. I am using VLC and it is capable of correctly playing the sample videos in the color charts folder. But somehow the videos I make with OBS are played with wrong contrast.

More interestingly, when I drag those videos into Kdenlive, the video editor I use, they still have wrong contrasts. Even it can't grab the correct color information from the videos for some reason.

 

[Suslik V]

You need 1280x720_color_range_test_chart.png !!! (not the chroma_subsampling).

Some video editors has wrong previews (speed-upping optimizations) but export may be OK. It's all about your own luck.

 

[52rpm]

i have amd gpu and my records and stream look washed :(

 

[CanCeralp]

I have downloaded another player called MVP and it properly displays everything as they should be. So the problem was never about the recordings, nor OBS. VLC simply treats everything as Partial and forces Full color range video to a contrast crash. Kdenlive also assumes the file is at partial color range but once I manually tell it that the video is "Full Range" with a checkbox, things work as they should.

To summerize the story, I learnt that there is no way to force 4:4:4 on AMD's hardware video capturing codec, however recording at full 0-255 color range is perfectly possible. Only problem is, when VLC sees 709, it assumes the video is at 16-235 range. No big problem as I record my games at 4K (high resolution should mitigate the lack of 4:4:4 in a game's recordings) and always re-encode them after a couple editing.

Thank you everyone for your help!

And, 52rpm, I'm sorry I can't help you with your problem because I have zero knowledge about streaming. I only know that OBS always forces the streaming to 4:2:0 Partial range. So, something must be mismatching with your screen/game color settings. You might try changing adding a LUT file to your stream or maybe fiddle with your monitor settings. I'm sure the pros here will help you much more than I could, if you start another topic about your problem. Or they may even have found a solution to your problem amongst the existing ones.