Bug Report Weird distortion using VAAPI with Intel graphics

[fragmentalstew]

I have a pc with Intel HD2000 as well as a geforce 650 ti. I tried out the new VAAPI plugin in v23 and I'm getting heavily distorted video
https://youtu.be/5dmnXRpnu9s as you can see. It looks good until something moves and then it looks like streaks, almost like a painting. It's interesting, but not very useful. I've seen a lot of video distortion, but I've never seen any like this.

Video is 720p 60fps
VAAPI Codec is H.264
Level is 4.0
Bitrate is 6000
Keyframe Interval is 0
I also tried keyframe interval of 2 and compatibility level of 4.1 as well as 720p30fps with level 4.0 and level 3.1 and the results were more or less the same.

NVENC looks fine, but I want to try to get quick sync working.

When I try "vainfo --display drm --device /dev/dri/renderD128", I get:
ibva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.1 (libva 2.1.0)
vainfo: Driver version: Intel i965 driver for Intel(R) Sandybridge Desktop - 2.1.0
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264StereoHigh : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc

"vainfo" returns successful for nvidia, but "vainfo --display drm --device /dev/dri/renderD129", which I believe is the nvidia card renderer, will fail. I believe that makes sense, because I don't think the nvidia card can actually do encoding using vaapi. I haven't been able to get any other vainfo commands for the intel to succeed.

I'm on Linux Mint 19.1

It's probably unrelated, but there were some changes that broke the quicksync driver in windows. There is already a bug report for that.

 

[fragmentalstew]

I forgot to upload a log file. Here's a new one after recording at the same settings at 10k bitrate https://obsproject.com/logs/_ldljJxUXY6Wi7nI

And here's a new video at 10k bitrate https://youtu.be/qyg6c_3s12I

It looks better at 10k, but it still looks pretty bad. I used to record at 1080p 60fps 10k bitrate with quicksync in windows and it looked pretty good. Here is one example https://youtu.be/54nzET1iV9I.

 

[fragmentalstew]

I tried using

LD_PRELOAD=/usr/lib/dri/i965_drv_video.so LIBVA_DRIVER_NAME=i965 ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i '2019-02-10 20-09-42.mp4' -vf 'format=nv12|vaapi,hwupload' -threads 8 -c:v h264_vaapi -preset slower -c:a libopus "f-new.mkv"

to transcode a video and it looks fine afterward.

 

[fragmentalstew]

Does anyone have any idea what could be going wrong? Has anyone else used vaapi with qsv at similar bitrates and had a clean encoding?

 

[kurufu]

I tried using

LD_PRELOAD=/usr/lib/dri/i965_drv_video.so LIBVA_DRIVER_NAME=i965 ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i '2019-02-10 20-09-42.mp4' -vf 'format=nv12|vaapi,hwupload' -threads 8 -c:v h264_vaapi -preset slower -c:a libopus "f-new.mkv"

to transcode a video and it looks fine afterward.

Those encoding settings are absolutely nothing like what is being used in OBS so its a mostly useless comparison.
Try using the ones actually output in your log for your convenience. https://obsproject.com/logs/_ldljJxUXY6Wi7nI