Strange color/ Problem OBs

[nuunty]

I have a monitor with HDR
I can't set up the color form and space in obs.
If the color space is: sRGB, Rec 709, Rec 601, it doesn't matter what color format I set, the picture will be the same (dark colors, with transparent windows: Screenshot 1). With these settings, you can start broadcasting and video.
If the color space is: Rec.2100(HLG,PQ), color format P010 OR I010, the picture becomes normal (Screenshot 2). But it gives the error Nvenc Error: Failed to perform 10-bit encoding in this encoder when starting the broadcast and recording video. In other color formats, when I use Rec 2100, the error "OBS does not support 8-bit Rec 2100 output"

 

[Suslik V]

Color Range is a limit and way to save bandwidth for signal (no matter colored or black-n-white).
Color Space is a standard and way to display colors.
Color Format is a precision and way to store colors.

You video card may not support encoding with high precision colors (more than 8-bit per color component), thus you see encoding error in OBS application.

Please post a log with your issue! Here's how...

OBS Studio creates a log file each time you run it which contains very useful diagnostic information. Without a log file, it's much harder for any support volunteers to figure out problems and your post may be ignored. To upload a log file, go to the Help menu > Log Files > Upload Last Log...

obsproject.com

 

[koala]

You need to first decide if you want to record HDR or SDR. HDR is somewhat new and not yet established with best practice, so you're mostly on your own.

To choose between SDR and HDR in OBS, you select the corresponding color format. A 8 bit format is SDR, a 10 bit format is HDR. According to what I know of best practice, you should use NV12 for SDR and P010 for HDR.
As color space select rec.709 if you chose SDR and rec.2100 (PQ) for HDR.

As encoder for SDR you can choose whatever encoder you like.
As encoder for HDR you have to choose a 10 bit capable encoder. That's the error you get. You have to choose HEVC or AV1 as encoder. H.264 doesn't support HDR, only SDR.