color quality loss due to chroma subsampling

[chrcoluk]

Hi guys, been fiddling with this for a while, I finally have my recordings looking ok for the quality I want, but colors remain a niggling issue.

Quick bit of info.

OBS version 25.0.8 64bit
Source PS4 Pro configured to use 4k output, I know rendering is lower resolution.
Capture device elgato HD S60+ (note the + it seems this is rare out in the wild not a single other soul on the internet has talked about using this, basically its a 4k HDR version of the HD S60)
Capture device settings 1080p 60fps color YUY2 (4:2:2), sadly for some unknown reason XRGB is "not" an option, only normal l420 and crappy NV12 as alternatives so YUY2 is best one.
In video, canvas and base both set to 1080p to match capture resolution.
In advanced, currently NV12, I have also tried l444 more on that below.
Finally output recording is x264, CRF 18, max bitrate 30000 (above this causes frame drops) 4.2 profile. Also fastest cpu (fast causes some frame drops, medium a lot of frame drops even with my 9900k)

So from my testing, on the Final Fantasy 7 remake equipment screen, this screen shows lots of numbers and text and is a good test to look at sharpness and
clarity.

It has White, Red and Blue, text and digits.

White and Blue are high sharpness and detail when viewing preview in OBS and in recordings.

Red depends.

YUY2 in capture settings, Reds have blockyness but the detail inside of the numbers is reasonable, so just the blockyness.
NV12 ir l420, its blocky and also inconsistent detail, inside the numbers, this looks aweful and makes me wonder how 4:2:0 ever became a thing. I suppose not an issue I noticed until looking at actual red numbers, but when I did notice it, its bad.
In recordings with NV12 in advanced settings (encoded at 4:2:0), the red numbers become blurry, this masks the blockyness, but also looks really odd alongside sharp white and blue characters.
In recordings with l444 in advanced settings, making it encode with "predictive 4:4:4" Its a duplicate of what I see in the preview, so no "further" loss of detail.

So the first question is does OBS support 4:2:2 output, I expect the answer is no but just want confirmation. If it doesnt is there any plans to add it?
Second question any idea why I cannot pick XRGB on the capture device? its advertised as supporting HDR, which seems odd if it has to subsample.
Third Question, I know 4:4:4 uses more bandwidth, when using CRF does this reduce quality in other ways to compensate or just increase filesize instead? I am noticing recordings are "not" bigger with it on which makes me wonder if quality is compromised. I feel like 4:4:4 is wasting me encoding bandwidth as the source is 4:2:2, so 4:2:2 would be perfect but OBS doesnt seem to support it.

Finally for those wondering why I am not outputting the ps4 at 1080p, the game is visually less sharp both live on the monitor and in OBS vs 1080p ps4 output, even though I am capturing at 1080p only. I even had blurry text on monitor with it in 1080p mode (that doesnt happen without hdmi passthru, so somehow its messing with 1080p output scaling I think). Also this is for local recording only I have no concerns for streaming on this project.

:)

 

[Suslik V]

does OBS support 4:2:2 output...?

No. It does not (v25.0.8).

...XRGB on the capture device?

Try to look for answers in:

Guides (OBS Studio)

Guides for OBS Studio

obsproject.com

there some guides written specially for Elgato devices. Also firm has YouTube channel with brief info about its products features.

I know 4:4:4 uses more bandwidth, when using CRF does this reduce quality..?

What you set as Output Mode? Simple or Advanced? I hope encoder is "x264" not the "(Use stream encoder)"? Otherwise the color planes conversion 4:2:0 may turn on automatically. How you playback your videos? You can try mpv player or latest build of VLC Player for example.
Related test charts and pre-recorded movies can be found here:

OBS Studio: Color Space, Color Format, Color Range settings Guide. Test charts.

Introduction. For higher compression rate, video encoders usually use methods of representation of each video frame as 3 separate channels: one for luminosity and two for color. This allow to compress color info separately from the luminance...

obsproject.com

 

[chrcoluk]

I playback using MPC-HC. which I find much superior to VLC in its color control and other metrics. VLC is just so buggy for me I stopped using it.

I set output to advanced.

Encoder is x264.

Sadly that second link you posted gives no answer to my question, youtube, twitch etc, have no relevance as the recordings are for local usage so concerns of online services converting to 4:2:0 doesnt matter. I dont see anything in the document about whether 4:4:4 steals quality from other metrics of the video. I asked the question because it is stated 4:4:4 uses more bandwidth but yet my filesizes dont increase on CFR when using it, so it must be getting this bandwidth from somewhere? I can see in the encodings 4:4:4 is working as its not removing color detail as the 4:2:0 encoding does.

 

[Suslik V]

whether 4:4:4 steals quality from other metrics of the video..?

Compressed with the loss of quality (CRF more than 0)? Yes. CRF = Constant Rate Factor. There is number of restrictions for encoding of H.264 (for example "level" etc.) that determines output file size. It is big question if the source current bitrate get limits by encoding restrictions or by poor source itself (like static image). Lossless size of the 4:4:4 almost always larger than 4:2:0.

Uncompressed data bandwidth and compressed data bandwidth are different things.

General rule: more pixels = more data to CPU through the RAM = more CPU work.

Color Range explicitly sets the limits on number of bits that are encoded. More bits at the same bitrate = less "visual" quality of the encoding. Thus I recommended Partial range.

What OBS Studio do - is preparing uncompressed color planes (4:2:0 or 4:4:4 sizes) and feeds them to encoder. Thus, encoder only process the data and doesn't resizes chroma planes internally.

You may run few FFpmeg encoding tasks to see how all this internal encoding stuff works (form build to build). "Lossy" encoding is highly depends on perception. Lossless encoding is different thing.

 

[chrcoluk]

Hi, you seem to be getting confused, I am not talking about CRF, I am not talking about pixel count (resolution).
I am not talking about color range.

I am talking about chroma subsampling.

All documentation I have read states that 4:4:4 subsampling has significant extra cost in terms of bitrate for supposed little benefit, so for this reason 4:2:0 is common.

In my recordings of FF7 Remake, I can indeed see most of the time 4:2:0 is not noticeable from 4:4:4 in terms of color detail.

However when the HP count goes critical, the color for the numbers changes to red, and the subsampling becomes extremely noticeable. Because this is a solid red colour.

My capture device is unable to capture at 4:4:4 (at least not with obss) so for that reason I can only compare 4:2:2 to 4:2:0 on recordings, although I have done a screenshot on the ps4 to show a 4:4:4 example.

The question was why am I not seeing a increased filesize when recording using predictive 4:4:4 instead of 4:2:0. All the documentation suggests that should be happening, which is what has made me wonder if quality from elsewhere is been lowered to keep to the same bitrate as 4:2:0, or is there a miracle happening and 4:4:4 gives me color detail with zero cost on filesize?

Interestingly when I made the below comparison image I can indeed see a loss of quality on 4:4:4, so I guess that answers my question. The first 2 images, have the same CRF, the same resolution, the same x264 profile, the only single change, is changing NV12 to i444 in OBSS advanced settings which changes x264 from yuv420 to yuv444, the 3rd image is a 4k screenshot so is nothing like apples to apples.

When watching the recordings, the white text, blue text etc. looks high quality and sharp, these images are zoomed in a lot, but the red is noticeable hence the comments, but I can see now that the 4:4:4 encoding option has some regressions so I will for sure keep encoding at 4:2:0, and maybe if I ever get a 4:4:4 capture device I might reconsider then. :)

Video: MPEG4 Video (H264) 1920x1080 60fps [V: h264 high L4.2, yuv420p, 1920x1080 [default]]
Video: MPEG4 Video (H264) 1920x1080 60fps [V: h264 high 4:4:4 predictive L4.2, yuv444p, 1920x1080 [default]]

--update--

Added second pic showing, extra images, labeled. So in the end the yuv420 actually isnt that bad a result, interesting.

--update 2--

Looks like the native images are 4:2:0, if I load up the image in irfanviewer it reports 2x2 subsampling. I did notice that if I zoomed in enough the red has less detail. So I dont think I can show a 4:4:4 image, the red numbers do look sharper on my monitor, I assume if I took a picture with my phone that would subsample as well, as its widespread practice lol. But interesting that if the base resolution is high enough it mitigates the affect of subsampling.

 

[chrcoluk]

Final update (sorry cant edit above post)

The 1080p native image is 2x2 subsampling as reported by irfanviewer. (bottom image)
The 4k native image (3rd image) is reported as no subsampling in irfanviewer, so the 4k screenshot may well be 4:4:4: but sadly the 1080p screenshot is for sure 4:2:0 so hard to get proper comparisons.

 

[Suslik V]

Enlarging/reducing image resolution may has its own subsampling, wrong gamma applied, half pixel-shift (for texels) etc. For example, some TV-boxes always renders menus in FullHD (no matter what output resolution is), thus internal scaling takes place.

 

[DayGeckoArt]

I know this is an old thread but yes 4:4:4 would get rid of the artifacts because color data is being captured at full resolution.Text would also look true to the original image, without the halo effect you get from the subsampling. OBS does support 4:4:4 but encoding depends on your computer. My Macbook Air M2 cannot hardware encode 4:4:4 but my Windows PCs with Nvenc and Quicksync can encode 4:4:4

But it looks like 4:4:4 could require a different capture device. I have a Blackmagic Intensity 4K PCIE card and it does capture 4:4:4