NVENC HEVC and H.264 different contrast or am I doing something wrong?

[joedynamite]

It is very possible I'm doing something wrong, or don't have something configured properly so please tell me if the problem is me, haha. I was trying to record I444 video for editing later and noticed some differences in HEVC and H.264 output contrast. I've used the loading screen in Mario Kart 8 Deluxe to determine if I have RGB range settings correct. It has very low contrast, light grey icons on a white background. If I ever have Full/Limited ranges incorrectly set it will be all white, but when properly matched it will show the light grey icons.

I had everything matched up on Full (since I was trying I444 recording) with H.264 and I correctly get the grey images in the background in the recording. But as soon as I change to HEVC, leaving every other setting the same, I get an all white background in the recording. Am I trying to do something HEVC doesn't support? Is it due to the HEVC codec or compression somehow? Do I have something else misconfigured? These screenshots are of the recordings being played back in VLC.

OBS 28.1.2, pretty fresh install, one scene, one video capture source.
Nintendo Switch running into AVerMedia Live Gamer 4K.
Nintendo Switch set to 1080p Full Range RGB.
AVerMedia Live Gamer 4K source set to Custom, 1080p, match output fps, XRGB, Rec. 709, Full Color Range.
OBS Video settings tab 1080p, no scaling, 59.94fps.
OBS Advanced settings tab I444, Rec. 709, Full Color Range, SDR White 300 nits, HDR nominal peak level 1000 nits.
OBS Recording Output settings CQP 20, 2s keyframe, P2, High Quality, Two Pass Qres, main, 2 max bframes.

Using an RTX 4080 for encoding (just got it hooked up a few days ago), I don't think I tried this on my previous card, a GTX 1080 Ti, and don't really want to swap it back out. I would also try recording a PC game to take the AVerMedia out of the equation but I don't know of a scene in any PC games that has low contrast like this for testing.

Here is a log file https://gist.github.com/joedynamite/f6528d9ac7f4ffe8cce9d11a27a457ba
Here are two short recordings, should be the last two recordings in the log above https://www.dropbox.com/scl/fo/zr2x586okqugxadxxwkd4/h?dl=0&rlkey=dncwzl7r2qdmvqm70dheo4zxm
^ weird that Dropbox generated thumbnails both have the correct contrast

 

[koala]

Both videos contain the grey background, if I download and play them with Media Player Classic, as well as if I watch them directly in the browser. Both contain 4:4:4 content. If you're not able to see it on your system, your media player isn't correctly displaying it.

This issue with media players not correctly displaying "full" color range content is one of the major reasons why it's not recommended to record "full". If you switch and record with "full" content, it's assumed you know of the implications.

However, if I use VLC, the background is missing as well, and VLC is known as media player that has a good video handling and support "full" content. It might be the slightly damaged hevc video. If I start it in the browser or with Media Player Classic, I see a garbled OK button as if the video data of the first video frame is defective, then it changes to the white frames with the grey images. In VLC, the initial garbled frames with the OK button are entirely skipped and you only see the plain white background with the figures at the bottom.

If you cut the video for showing the issue here, it may be you didn't cut at a key frame, so the initial frames are orphaned predictive frames with incomplete image info.

 

[joedynamite]

Thanks for the response. I hadn't used Media Player Classic before, but just downloaded it to try out. As for cutting the video, I just started and stopped recording in OBS to get those recordings, I didn't use another program or anything. Should OBS always start and stop recording on keyframes?

You mentioned a possibly damaged file, Is there any other way to detect if the HEVC file is damaged or are there ways to attempt to repair it? It seems to happen _every_ time I try to record HEVC and never when I record H.264, any reason one setting might produce damaged files and not the other?

I guess I can also fall back to limited range too, I mostly used that in the past but I was trying to follow Xaymar's guide to higher quality recordings and it says to use full https://www.xaymar.com/guides/obs/high-quality-recording/

 

[koala]

I don't know why your hevc recordings have a broken start. My own recordings don't have this. According to your log, you didn't record with i444 but with rgb, may be this is the cause. Rgb isn't recommended, because it is not hardware accelerated and very slow and resource demanding.

If you should go back to standard configuration or not depends on your use case. If you have a real need for full range recording, for example because you need a lossless source for postprocessing or the consumers of your final product require full range, you need this. But if you just want some recording of random stuff with reasonably best quality, full color range isn't the relevant thing that will give you best quality. It's the general encoder settings instead: a quality based rate control with a quality parameter that results in exactly the quality you need - lossless, or indistinguishable from lossless with smaller file size, or even smaller file size but slightly visual compression artefacts.

 

[joedynamite]

Thanks so much for all the explanation. I guess I need to understand all of these settings a bit better and that's my fault. I thought setting the capture card to XRGB was how to utilize 4:4:4 subsampling, and I thought I444 was the only way to record without it losing data and dropping down to 4:2:0. So perhaps my combination of settings is just incorrect. Just seemed strange that switching between H.264 and HEVC made a noticeable difference in output to me. Thanks again!