Recording in RGB color space broken

[Lojalfan]

The recording is totally broken when using RGB color space and libx264rgb. The whole screen is green basically.
The same settings worked fine on OBS 27 and the only thing I did was upgrade to 28, there was some other guy who had the same problem on a Mac. I didn't test using "lossless" in simple output mode, just x264.

Example:

Log: https://obsproject.com/logs/43D61yZAH-fYc-TK

 

[Lojalfan]

Yeah lossless in simple output mode works fine, problem is I don't have a large enough SSD to record 4K so it has to be x264rgb.

 

[ravewulf]

I also have this issue and the root cause is OBS isn't using libx264rgb at all even though the dropdown says it should be.

Instead, it is sending the RGB video data to the regular (non-rgb) libx264 encoder which then treats it as if it were YUV without actually doing the color conversion and leading to the green mess seen above.

 

[DayGeckoArt]

Do other encoders work with RGB? I learned in 2021 that RGB doesn't work and you just get NV12... So I switched to 444

 

[ravewulf]

I've tried ffvhuff, utvideo, and ffv1 which all work but create significantly larger files than libx264rgb, though they can be recompressed after

 

[ravewulf]

Currently, OBS's version of libx264rgb maps RGB video like this and marks it as YUV instead of RGB
R -> V
G -> Y
B -> U

If you're familiar with AviSynth, you can use this statement to reverse the process

MergeRGB(ShowV,ShowY,ShowU)

I'm not sure if OBS's broken libx264rgb encoder is lossless or not so it would be best to use a different codec for RGB recordings until this is fixed if that's important to you.

 

[ravewulf]

Looks like there is already bug report on GitHub https://github.com/obsproject/obs-studio/issues/7345

The build of ffmpeg that OBS 29 uses is broken in this regard so we'd need to wait for OBS to update it or otherwise try and see if we can use the official ffmpeg dlls with OBS (not something I particularly want to try at the moment as I'm busy with other things) for a true fix.

A workaround mentioned in the bug report is to add

colorspace=0

to the Video Encoder Settings box in OBS, which will at least correctly flag the resulting h.264 video as RGB.