RGB Color Format warning implies it's a good setting for recording

[smbhax]

If you set Settings > Advanced > Color format to "RGB," a warning appears at the bottom of the OBS window:

Warning: Color formats other than NV12 are primarily intended for recording, and are not recommended when streaming. Streaming may incur increased CPU usage due to color format conversion.

^ That seems to imply that RGB is a good choice for recording, maybe superior to NV12. According to a response in this thread -- https://obsproject.com/forum/threads/obs-recording-gets-blurry-on-the-right-side.127231/ -- and my own experience applying its advice to switch back to NV12 from RGB (I too had the right-side blurry video problem under RGB), it is likely that in many cases, RGB is definitely *not* the correct choice for recording, and will cause quality problems if used instead of NV12.

Perhaps the warning text could be adjusted so that it doesn't imply that RGB is a good choice for recording. I'm annoyed with myself for having recorded unnecessarily blurred videos for years now on the RGB setting, thinking it was the special secret choice for cool recording people, in part due to how I'd interpreted that warning message.

Similarly, perhaps a "(Recommended)" note or something added to the Color Space "709" and Color Range "Partial" options just below?

 

[DayGeckoArt]

I registered just to reply to your post! Been lurking for a little while.

RGB is the a good choice for recording, and I consider it the best choice. Why? Because it's recording the data straight off the screen, without any processing to convert to I444 or I420 or NV12. So encoder usage is lower.

To put it another way, I420 is lower quality but also takes more processing to encode, and I444 is the same quality but also requires more processing to encode.

The only problem is if you have an older GPU or CPU with an older version of Nvenc or Quicksync without RGB support

 

[FerretBomb]

As far as I am aware, even the newest generation NVENC does not support RGB, and is forced to fall back to software x264 when RGB is selected.
I444 is strongly recommended if you need no-subsampling recordings and are using a hardware encoder, as the encoding load is already removed from the CPU in that case, and at least with NVENC will not affect in-game performance either.

The warning is mostly there as (last I was made aware) OBS is hard-coded to livestream in NV12, even if RGB is selected, which will cause color skew issues in addition to the performance hit RGB carries.

 

[DayGeckoArt]

As far as I am aware, even the newest generation NVENC does not support RGB, and is forced to fall back to software x264 when RGB is selected.
I444 is strongly recommended if you need no-subsampling recordings and are using a hardware encoder, as the encoding load is already removed from the CPU in that case, and at least with NVENC will not affect in-game performance either.

The warning is mostly there as (last I was made aware) OBS is hard-coded to livestream in NV12, even if RGB is selected, which will cause color skew issues in addition to the performance hit RGB carries.

I have 3 computers that I've successfully used RGB with:
Alienware R8 with RTX 2060S recording 4K - HWinfo64 shows about 45% NVENC usage
Dell workstation with brand new Quadro T400 recording 4K - HWinfo64 shows about 45% NVENC usage
Dell Micro with i7 4th gen CPU with Quicksync recording 1080p - Haven't monitored QSV usage

I haven't seen any color skew issues either. I set it to REC709 and full color range. I attached screenshots of my settings

 

[DayGeckoArt]

That shows H.264 but I've also used NVENC-HEVC successfully, records totally smooth 30fps 4K

 

[koala]

I haven't seen any color skew issues either. I set it to REC709 and full color range. I attached screenshots of my settings

OBS settings are one, but are you sure your recorded material is really really RGB? If I use your settings and look with MediaInfo what actually has been written to the resulting video, I get YUV 4:2:0 material, no RGB material, not even 4:4:4:

As far as I know, this is because h.264 nvenc simply doesn't support RGB.

Nvenc h.265 (hevc) does support it, but still downgrades to 4:2:0

 

[DayGeckoArt]

OBS settings are one, but are you sure your recorded material is really really RGB? If I use your settings and look with MediaInfo what actually has been written to the resulting video, I get YUV 4:2:0 material, no RGB material, not even 4:4:4:

View attachment 78294

As far as I know, this is because h.264 nvenc simply doesn't support RGB.

Nvenc h.265 (hevc) does support it, but still downgrades to 4:2:0
View attachment 78295

Interesting... Mine shows the same thing. I tried making two videos with NVENC HEVC, one with RGB and the other 420, of a test pattern. Amazingly, the I420 looks better. The RGB one shows the artifacting the test pattern is supposed to create but the I420 one doesn't https://www.rtings.com/tv/learn/chroma-subsampling

 

[DayGeckoArt]

I then tried adding using the parameter to force RGB for FFMPG but it had no effect

 

[DayGeckoArt]

And I tried I444 which does get the result I want, no chroma subsampling! Thanks, now I know not to use RGB and instead use I444

 

[koala]

Yes, that's my experience as well. RGB isn't well supported in consumer video formats, if at all. Using i444 has the nice property that it is hardware accelerated within the GPU (and the OBS compositing), while RGB isn't.

By the way, in your nvenc_hevc settings screenshot you're using the wrong parameter format, so pix_fmt is ignored. It's ignored by OBS anyway, because nvenc_hevc doesn't support this parameter in ffmpeg/nvenc (call "ffmpeg -h hevc_nvenc").

If you want better quality, you must not limit the bitrate to measly 15000. Use rc=vbr cq=24 as encoder settings. This enables vbr instead of cbr mode and requests a quality not distinguishable from the original. This will use dynamic bitrate - down to below 100 kb/s for static images, up to the given bitrate on high motion scenes.

 

[FerretBomb]

Just as a note, post the logfile from test recordings, not settings-screenshots. Screenshots are next to useless.
A logfile will contain your settings, as well as actual useful diagnostic information like the internal message when a recording setting is not supported by the encoder, and what settings the recording is instead falling back to use.

Also not sure if ffmpeg reverses this, but when using CQP, lower is higher quality (and larger filesizes). The CQ value is how far the encoder is allowed to deviate from perfect, uncompressed frames. 24 is pretty far; you can see artifacting at 22. 16 is about the line where visually-lossless hits. Would not use VBR or CBR for recording.

 

[Tomasz Góral]

RGB is only in 24bits or 32bits with alpha channel 8bit value per pixel.
yuv is in 420 is 12bits per pixel or 422(typing for tv) 16bits per pixel or 444. Of course there’s more surface.
yuv have hardware converter in graphics card (since 15 years, last card without converter is intel 845 about which I know), obs draw on RGB surface and send copy to yuv surface to encoding.

 

[DayGeckoArt]

Just as a note, post the logfile from test recordings, not settings-screenshots. Screenshots are next to useless.
A logfile will contain your settings, as well as actual useful diagnostic information like the internal message when a recording setting is not supported by the encoder, and what settings the recording is instead falling back to use.

Also not sure if ffmpeg reverses this, but when using CQP, lower is higher quality (and larger filesizes). The CQ value is how far the encoder is allowed to deviate from perfect, uncompressed frames. 24 is pretty far; you can see artifacting at 22. 16 is about the line where visually-lossless hits. Would not use VBR or CBR for recording.

Thanks, I tried the FFMPEG set to nvenc_hevc and cq=16 as the only parameter... That does create VBR video. I set the recording bitrate to 30Mbps and it varied between 5Mbps and 30Mbps

BTW this is all stuff I tried to research about a year ago when I first started using OBS and Streamlabs, and there's just not info out there. Seems like pretty much everyone only cares about streaming rather than settings for recording. I don't get it-- who is watching all these streams?

For example there's this thread from 2017 with no replies https://obsproject.com/forum/threads/recommended-settings-for-ffmpeg-nvenc-hevc-encoding.78433/

 

[DayGeckoArt]

Yes, that's my experience as well. RGB isn't well supported in consumer video formats, if at all. Using i444 has the nice property that it is hardware accelerated within the GPU (and the OBS compositing), while RGB isn't.

By the way, in your nvenc_hevc settings screenshot you're using the wrong parameter format, so pix_fmt is ignored. It's ignored by OBS anyway, because nvenc_hevc doesn't support this parameter in ffmpeg/nvenc (call "ffmpeg -h hevc_nvenc").

If you want better quality, you must not limit the bitrate to measly 15000. Use rc=vbr cq=24 as encoder settings. This enables vbr instead of cbr mode and requests a quality not distinguishable from the original. This will use dynamic bitrate - down to below 100 kb/s for static images, up to the given bitrate on high motion scenes.

So the box that says Video Encoder Settings (if any) can have the paratmeters like cq=24? Where do you put "ffmpeg -h hevc_nvenc"?

Where can I find a list of settings that can go in that box with the exact syntax?

 

[DayGeckoArt]

Koala, I found this thread where you showed rc=vbr

I added rc=cqp to my parameters but I get this error in the log:

22:16:46.564: Loaded scenes:
22:16:46.564: - scene 'Dell 4K Main Screen':
22:16:46.564:     - source: 'Dell 4K' (monitor_capture)
22:16:46.564: - scene 'HP Side Screen':
22:16:46.564:     - source: 'HP1200p' (monitor_capture)
22:16:46.564: - scene 'Dell 4K with Camlink':
22:16:46.564:     - source: 'Dell 4K' (monitor_capture)
22:16:46.564:     - source: 'Camlink' (dshow_input)
22:16:46.564: - scene 'Camlink Only':
22:16:46.564:     - source: 'Camlink' (dshow_input)
22:16:46.564: ------------------------------------------------
22:16:46.611: adding 42 milliseconds of audio buffering, total audio buffering is now 42 milliseconds (source: Mic/Aux 2)
22:16:46.611:
22:16:46.670: Camlink: data.GetDevice failed
22:16:46.670: Camlink: Video configuration failed
22:20:35.218: Settings changed (outputs)
22:20:35.218: ------------------------------------------------
22:20:37.759: Failed to set rc=cqp
22:20:38.035: ==== Recording Start ===============================================
22:20:58.944: Output 'adv_ffmpeg_output': stopping
22:20:58.944: Output 'adv_ffmpeg_output': Total frames output: 625
22:20:58.944: Output 'adv_ffmpeg_output': Total drawn frames: 636
22:20:58.945: ==== Recording Stop ================================================
22:21:03.483: ==== Shutting down ==================================================

 

[koala]

Where do you put "ffmpeg -h hevc_nvenc"?

This is from the standalone ffmpeg.exe you need to download extra. It's the command line for getting all parameters for hevc_nvenc. It can also be found here: https://gist.github.com/asus4/f5aef0f3f46fde198436da12f0332013

The error you got in your next post is because there is no rate control called cqp for hevc_nvenc. There is cbr, vbr and constqp. cbr is constant bitrate and what you need for streaming, vbr is variable bitrate and if used together with qp=<number> it works like the crf=<number> in x264. This is what you want to use for recordings. constqp is constant quantizer, and not recommended, since vbr with cq is getting smaller file size with same constant quality.

To clarify: there is a "constant quality" mode (activated with rc=vbr cq=<value>) and a "constant quantizer" mode (activated with rc=constqp qp=<value>". cq (constant quality) and qp (quantizer parameter) are not the same. The quantizer tells how much detail to remove. If it is constant, the same amount of detail is removed for every frame, regardless of the character of the scene. The quality parameter tells somethin like how much quality is kept. It controls an internal algorithm within the encoder to vary the quantizer up and down in a way that more detail is removed where it is appropriate (and not perceivable) and less detail if required for high complexity scenes. This results in smaller files with all frames the same quality.

 

[DayGeckoArt]

This is from the standalone ffmpeg.exe you need to download extra. It's the command line for getting all parameters for hevc_nvenc. It can also be found here: https://gist.github.com/asus4/f5aef0f3f46fde198436da12f0332013

The error you got in your next post is because there is no rate control called cqp for hevc_nvenc. There is cbr, vbr and constqp. cbr is constant bitrate and what you need for streaming, vbr is variable bitrate and if used together with qp=<number> it works like the crf=<number> in x264. This is what you want to use for recordings. constqp is constant quantizer, and not recommended, since vbr with cq is getting smaller file size with same constant quality.

To clarify: there is a "constant quality" mode (activated with rc=vbr cq=<value>) and a "constant quantizer" mode (activated with rc=constqp qp=<value>". cq (constant quality) and qp (quantizer parameter) are not the same. The quantizer tells how much detail to remove. If it is constant, the same amount of detail is removed for every frame, regardless of the character of the scene. The quality parameter tells somethin like how much quality is kept. It controls an internal algorithm within the encoder to vary the quantizer up and down in a way that more detail is removed where it is appropriate (and not perceivable) and less detail if required for high complexity scenes. This results in smaller files with all frames the same quality.

Thanks, have the parameters list saved

FerretBomb said " Also not sure if ffmpeg reverses this, but when using CQP, lower is higher quality (and larger filesizes). The CQ value is how far the encoder is allowed to deviate from perfect, uncompressed frames. 24 is pretty far; you can see artifacting at 22. 16 is about the line where visually-lossless hits. Would not use VBR or CBR for recording. "

Is he mixing up CQ and QP? The CQ can be between 0 and 51 so I guess 24 is in the middle? Is a higher number better quality?


Hmmm, I think I'm going to make my own thread, because I'm trying to optimize settings for TV recording, and there doesn't seem to be a thread on this topic

 

[koala]

It might be FerretBomb didn't realize we (or at least I) talked about hevc_nvenc in ffmpeg output, not about h264_nvenc in simple or advanced output mode. With hevc_nvenc, the meaning of the rate controls and parameters slightly changed in comparison to h264_nvenc. The latter missed a constant quality mode and had only cqp for constant quantizer, so this is what we all used to not fallback to the inferior cbr or vbr modes. But with hevc, there is a constant quality mode available with vbr, so this is what should be used instead for best quality and best usage of bandwidth/disk space.

 

[TryHD]

And I tried I444 which does get the result I want, no chroma subsampling! Thanks, now I know not to use RGB and instead use I444

The RGB color space doesn't know chroma subsampling, that is a YUV thing. Since NVENC doesn't support RGB it will convert to bgr0 which is still less lossy than YUV 4:4:4. Also you should use sRGB and not Rec.709.

 

[DayGeckoArt]

Well I posted a thread and it isn't anywhere to be found. I'm guessing the moderators deleted it and didn't tell me.

I experimented a bit yesterday and settled on rc=vbr cq=1

I tried cq=24 and cq=45... I can't tell a visual difference between 24 and 1 but there's also no bitrate difference. I have it set to 40mbps and most of the time the video is 20mbps, but will drop down to 5mbps if it's a totally still frame being recorded. cq=45 is much lower image quality and records at 5mbps max.

But it makes me wonder if cq=1 is the same as cq=0 IE automatic. I'll try some other values