Bug Report QuickSync ignores Custom Encoder Settings and Terrible quality

[Matthew M. Dean]

extbrc=1:tu=1:trellis=3:num-ref-frame=4:gop-ref-dist=3:gop-pic-size=30:b-pyramid=1 doesn't do anything at all according to Mediainfo.
Infact QuickSync OBS looks terrible when compared to handbrake output with only average bitrate set.
I dont think Encoder preset does anything either. It looks like it's stuck on balanced even when set to quality. There's a huge difference in VMAF scores between those 2 settings.
No matter what you change Mediainfo remains the same. Main@L4 M=8 N=60 2 RefFrames.
https://i.imgur.com/Oa48zkI.png produces output far greater at 75 FPS then the QuickSync in OBS.
https://i.imgur.com/N3zFiyZ.png - OBS qsv output
https://i.imgur.com/P4ifZqz.png - Handbrake qsv output

 

[koala]

Quicksync is not x264. If you want to use x264-specific encoding options, use x264. Quicksync does not have these options. The options are ignored, by Obs as well as by Handbrake.
The quality difference with Handbrake is from something else, for example bitrate, or you use a different target usage or profile.

Apropos bitrate: make sure you set and compare the bitrate for the video stream only. Some apps only count the video part of the stream, some apps add video and audio part and count the added bitrate. As far as I remember, OBS makes it so that the overall bitrate is respected. That means if you set bitrate to 5300, the actual video bitrate is a bit smaller, because it has to also include audio within the 5300. In Handbrake, on the other hand, you set the bitrate for video only, and bandwidth for audio comes on top of that.

 

[R1CH]

There is no support for custom QSV parameters in OBS. Where are you pasting those options?

Also the target usage is not a preset, it only affects the quality / speed balance passed to the encoder - it won't change any settings.

 

[Matthew M. Dean]

@koala https://handbrake.fr/docs/en/latest/technical/video-qsv-options.html
These custom options in handbrake will reflect in Mediainfo and in VMAF scores.

~/vmaf/ffmpeg2vmaf 1920 1080 Apex_1080p30_OBS.avi Apex_1080p30_qsv_quality_5300kbps.m4v --pool perc5
VMAF_score: 67.384195

~/vmaf/ffmpeg2vmaf 1920 1080 Apex_1080p30_OBS.avi Apex_1080p30_qsv_quality_noextra_5300kbps.m4v
VMAF_score:65.900981

~/vmaf/ffmpeg2vmaf 1920 1080 Apex_1080p30_OBS.avi Apex_1080p30_x264_high_5300kbps.m4v
VMAF_score:74.574434

These scores are terrible, x264 is decent. I guess content does matter. Unreal 3 was different. QuickSync balanced scored with VCE, and QuickSync quality with extra options scored with x264 Fast. Not expecting miracles but OBS QSV for Apex Legends certainly looks worse than 65. The preset and encoder options on the simpleview don't do anything. Thats why it looks closer to VCE than it does to x264.

I understand what your saying, thats a great point about audo bitrate, I wish obs did video bitrate only instead of overall. Less confusing I think. I should have posted in 6mbit but I don't believe 96kbs should make that much perceptual difference.

I record raw in OBS, stream in QSV, download the source from twitch which I've been able to prove is unaltered. Then handbrake the raw footage and compare. I dont expect an exact match though I don't think thats actually unreasonable it being a hardware encoder?

@R1CH According to the documentation tu is an option with a value from 1 to 7.

Settings -> simple-> encoder preset & custom encoder settings. I'm sure they don't actually do anything. Under advanced view atleast the encoder preset does and its reflected in mediainfo so we're getting somewhere.

M is hardcoded 8 instead of 3, but now I can set N to be 30 frames. refframes are hardcoded to 3 instead 2 with profile high@L4. I have to turn off enforce streaming service encoder settings. idc about maximum compatibility, only quality.

I'm convinced https://i.imgur.com/EGYOqAm.jpg is because of trellis=0:extbrc=0:gop-ref-dist=8 Doom 4 looks nowhere near this bad and actually scores high so it's obviously content related. I dont expect to reach Doom 4 quality but I'm certain the GOP is too far apart.

I could be wrong of course, I'd love to find out. How different is it though?

I don't know because to compare OBS Raw to OBS QSV ffmpeg PSNR and VMAF filters require the exact same offsets which I haven't figured out how to do yet. Tried to ffmpeg -ss 2.13 to get them back in sync but the scores were still invalid, have to do more testing.

If I can reproduce the same quality between OBS and Handbrake with the same settings then I would consider them to be "calibrated".

To recap -
• In Simple view Encoder Preset appears to be ignored
• In Simple view Custom Encoder presets are definitely ignored
• You can't change the Reference Frames, the distance between anchor frames, trellis and extended bitrate control algorithms.

 

[koala]

To repeat: QSV does not have all these parameters. QSV is a hardware encoder with a few simple tuning settings that are exposed in the GUI. It is not as flexible as x264. It's all hardcoded within the QSV encoder.

 

[Matthew M. Dean]

To repeat: QSV does not have all these parameters. QSV is a hardware encoder with a few simple tuning settings that are exposed in the GUI. It is not as flexible as x264. It's all hardcoded within the QSV encoder.

After testing each parameter in the encoder options in Handbrake and verifying with mediainfo.
num-ref-frame works. gop-ref-dist works, gop-pic-size works, b-pyramid got added to the qsv plugin on github less then a month ago.
extbrc? https://software.intel.com/en-us/ar...fficiency-and-video-quality-with-adaptive-ltr
trellis? VMAF goes way way down when I remove this more then 1 JND which is why I think it really needs to be added. https://software.intel.com/en-us/node/696131

I dont know what it is your saying because all these parameters clearly work.

 

[koala]

After reviewing the ffmpeg doc there seems indeed some mapping of ffmpeg parameters to Quicksync parameters within ffmpeg. Sorry I ignored that.
But since OBS uses its own Quicksync implementation and does not use the ffmpeg encoder for Quicksync, thus probably no ffmpeg option parsing, I cannot say anything about how OBS will handle additional parameters not exposed in the OBS GUI.

 

[Matthew M. Dean]

After looking through the code on github OBS looks like its using the Intel SDK. It just appears those functions are simply not implemented. Unfortunately I'm not well versed enough to modify the code myself.