True constant quality mode for h264_nvenc and hevc_nvenc?

[whitepuzzle]

Currently the only way to use true constant quality mode with these encoders is with the FFmpeg custom output mode set to options "rc=vbr b=0 cq=[0-51]", which doesn't work with replay buffer. Is there any plan to add a proper CQ mode to the standard output in the future? VBR and CQP are both inferior for quality-to-filesize ratio.

I did a test game recording with CQ mode and it seems to work just fine without overloading the encoder or anything, so I doubt it's a technical limitation that prevents it from being officially implemented.

 

[rockbottom]

I think you're confused or looking at bad information.
-cq <float> E..V....... Set target quality level (0 to 51, 0 means automatic) for constant quality mode in VBR rate control (from 0 to 51) (default 0)

 

[whitepuzzle]

I think you're confused or looking at bad information.
-cq <float> E..V....... Set target quality level (0 to 51, 0 means automatic) for constant quality mode in VBR rate control (from 0 to 51) (default 0)

Definitely not confused about anything. I was right that CQ (Constant Quality) and CQP (Constant Quantizer Parameter) are not the same thing, however I just did some test encodes with ffmpeg comparing CQ (-rc vbr -b 0 -cq X) and CQP (-rc constqp -qp X) modes for hevc_nvenc and found that, at the same file size, CQ mode actually has worse quality than CQP, at least at lower bitrates. I guess OBS devs don't implement the true CQ mode for a reason? According to information I've read on this forum and other forums, CQ mode should in theory outperform CQP mode. Perhaps that's only true for h264_nvenc which I will test later.

 

[rockbottom]

The attached docs are correct for NVENC. CQ is the target quality setting. Your encoder settings are more than likely being rejected. Look at your log.

Try this to start & adjust as needed. H265

preset=hq profile=main tier=high rc=constqp rc-lookahead=0 init_qpI=15 init_qpP=18 init_qpB=20 bf=0 b_ref_mode=disabled refs=4

 

[DayGeckoArt]

I have a thread on this https://obsproject.com/forum/threads/nvenc-hevc-vbr-settings.151747/

After much experimenting I settled on rc=constqp and cq=20 for high quality or cq=25 for medium quality. For some things like games I do as much as cq=35 to reduce bitrate.

These settings give big changes in bitrate depending on the details in the scene, which is exactly what you want for "constant quality".