Question / Help QuickSync or AMD support on MESA

[TyMiles2012]

EDIT: I fixed my problem! All I did was compile FFmpeg with --enable-vaapi and --enable-shared, then I compiled OBS-Studio forked by GloriousEggroll. I used the debian build instructions from the official github as a guide. :) Encoding works with both my iGPU and AMD GPU! MESA drivers are the Oibaf bleeding edge drivers for Ubuntu.

Old post below
------
Hi! I've been looking all over and I can't seem to figure out how to enable support for these. I'm a bit new to Linux and I'm not super familiar with compiling stuff. From what I've read on here, VAAPI encoding isn't supported on the official drivers, but I'm using the Oibaf MESA drivers. This is my VAINFO. I'm more of a fan of QuickSync and my BIOS were already set up for it on Windows, but if I'm stuck with AMD's encoder, I'll be just as happy. It's just that x264 is way too intensive for me. My FPS on Wine plummets when I screen capture with it using the CPU.

I found this, but it's only for arch. I use Ubuntu.
https://forum.level1techs.com/t/obs-studio-ffmpeg-vaapi-plugin/121705

libva info: VA-API version 0.40.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_0_40
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.40 (libva )
vainfo: Driver version: Mesa Gallium driver 18.1.0-devel for AMD Radeon (TM) R9 390 Series (HAWAII / DRM 3.18.0 / 4.13.0-36-generic, LLVM 6.0.0)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :   VAEntrypointVLD
      VAProfileMPEG2Main              :   VAEntrypointVLD
      VAProfileVC1Simple              :   VAEntrypointVLD
      VAProfileVC1Main                :   VAEntrypointVLD
      VAProfileVC1Advanced            :   VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:   VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:   VAEntrypointEncSlice
      VAProfileH264Main               :   VAEntrypointVLD
      VAProfileH264Main               :   VAEntrypointEncSlice
      VAProfileH264High               :   VAEntrypointVLD
      VAProfileH264High               :   VAEntrypointEncSlice
      VAProfileNone                   :   VAEntrypointVideoProc

 

[TyMiles2012]

I solved it! Look at first post.

So, I decided to try again. I managed to compile and install both FFmpeg and OBS forked by GloriousEggroll. Both AMD and QSV showed up (Both card0 and card1) but whenever I try to record with either, it immediately crashes, and trying to use CPU says it fails to output and to make sure the latest drivers are installed. Again, I'm using Oibaf.

The log file shows no errors, and the last line is just a time stamp with a blank line.

At least I know how to compile source code with cmake now! So it's something. lol

EDIT: Opened OBS through the terminal. Trying to record using OBS using CPU gives me an error on AAC encoder.

[aac @ 0x559e93769500] Unsupported channel layout

 

[GloriousEggroll]

glad to hear you got it resolved. for anyone else in the future, I've removed the obs-studio pre-patched fork as i have too many repositories to maintain and cleaned some of them up, however, you can find the patch here:

https://github.com/GloriousEggroll/...t-ffmpeg-vaapi/blob/master/ffmpeg-vaapi.patch

just download it and put it into your cloned obs folder, then:
patch -Np1 -i ffmpeg-vaapi.patch

then compile obs like you normally would (with --enable-vaapi and --enable-shared)