Experimental ffmpeg-vaapi plugin

bywant · Aug 9, 2017

Has anyone tried this methode with libva from Intel Media Server Studio?

i am getting this error in logs:
libva info: VA-API version 0.99.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
warning: [VAAPI encoder]: found no exact match for profile, defaulting to best match 'constrained baseline'
error: [VAAPI encoder]: "vaCreateContext(enc->display, enc->config, enc->width, enc->height, 0x1, ((void *)0), 0, &enc->context)": resolution not supported
error: [VAAPI encoder]: failed to initialize encoder for profile constrained baseline

Logfile:
https://gist.github.com/anonymous/66c48de31926dbee14678886f9333990

GloriousEggroll · Nov 19, 2017

Arch users:

So I made a patch out of this so it could easily be used on arch with obs-studio-git on the AUR rather than cloning + merging the repo every time there's an update. I've also edited my pkgbuild to build the latest release instead of the latest git commit version.

I tested this on mesa 17.3 on arch on a vega 64, working well!

https://github.com/GloriousEggroll/arch-obs-studio-latest-ffmpeg-vaapi

easy steps:

git clone https://github.com/GloriousEggroll/arch-obs-studio-latest-ffmpeg-vaapi
cd arch-obs-studio-latest-ffmpeg-vaapi
makepkg -i

GloriousEggroll · Nov 19, 2017

David Carver said:
I managed to get the reboot/vaapi-h264 version working with the master branch of obs-studio. No changes were necessary to get it to compiled besides what was in his base branch. It would be fantastic if somebody could get this incorporated into the main obs-studio code base. I can create a branch and pull request but we really should contact Reboot to get permission to include his work into the main distribution.

My CPU usage went from hitting 19 to 20 percent, to between 5% and 8% with using the VAAPI-H264 encoder settings.

Update: I created a simple blog entry with a link to Reboot's code working with the current master branch (19.x) In case anybody is interested. No problems merging the code in and getting it to work. Here is the link.

https://wordpress.com/post/intellectualcramps.wordpress.com/1151

is there any benefit to using this branch as opposed to w23's version?

I've just made a patch out of w23's version and have it working on mesa 17.3 rc5 with amd vega 64

edit: I tested both. w23's worked successfully with mesa, while reboot's did not work on mesa or with libva-vdpau-driver installed. unsure if it works using intel's non-mesa vaapi driver, was not able to test as I also have an amd cpu

edit 2: according to https://wiki.libav.org/Hardware/vaapi the device should be
/dev/dri/renderD128
rather than
/dev/dri/card#

even with this change, it doesn't work for me, and gets stuck at the "stopping recording" button

GloriousEggroll · Nov 25, 2017

Hi, so I went in and cleaned up the patch, added the ability to choose which card to use, added a few missing options, and updated options which should override eachother (ie bitrate/qp and b-frames/high profile). I've also added some basic support for other codecs (hevc, mpeg2, vp8, vp9). I've made a video about it here:

https://youtu.be/s2EZ_H-POSM

I've made a patch here:
https://github.com/GloriousEggroll/...t-ffmpeg-vaapi/blob/master/ffmpeg-vaapi.patch

I've forked obs and added the patch here:
https://github.com/GloriousEggroll/obs-studio

screenshot:

I plan to submit a pull request as soon as I get more input on the intel side of things.

Have tested this using vega 64, I was also able to stream and record simultaneously using vaapi.

I did have to make a quick hack so that it uses 1072 instead of 1080 so that no weird green bars show up while streaming. This is because the dimensions have to be divisible by 16 for h.264, and setting it at 1080 auto-bumps it to 1088, and 1088 causes an ugly green line at the bottom on twitch.tv. other resolutions worked fine with no bar issues.

Bleuzen · Nov 28, 2017

@GloriousEggroll
I can't compile on Manjaro (Arch based):

I have a NVIDIA card, but wanted to try out to use my Intel iGPU with OBS.
(Maybe the Nvidia driver blocks the open source driver? But would this end up in such a compile error?)

My
GPU: GTX 1060
CPU: i7-7700

GloriousEggroll · Nov 29, 2017

ah its a small error with the profile name, it should be FF_PROFILE_HEVC_MAIN_10, fixed in both repos now. You may need to try libva_driver_name envar befor launching obs as nvidia tends to try to take over as the default vaapi driver. I wasnt able to get it working on my laptop, which is intel+nvidia

Bleuzen · Nov 29, 2017

I also don't get it working. I started obs just with the
obs
command and got this error:
[AVHWDeviceContext @ 0x55a3b4756340] libva: va_getDriverName() failed with unknown libva error,driver_name=(null)
[AVHWDeviceContext @ 0x55a3b4756340] Failed to initialise VAAPI connection: -1 (unknown libva error).
warning: [FFMPEG VAAPI encoder: 'recording_h264'] Failed to create VAAPI device context: Eingabe-/Ausgabefehler

Also i tryed to start it so:
env LIBVA_DRIVER_NAME=i965 obs

and got this error:
[AVHWDeviceContext @ 0x5637b79b1140] libva: /usr/lib/dri/i965_drv_video.so init failed
[AVHWDeviceContext @ 0x5637b79b1140] Failed to initialise VAAPI connection: -1 (unknown libva error).
warning: [FFMPEG VAAPI encoder: 'recording_h264'] Failed to create VAAPI device context: Eingabe-/Ausgabefehler

And yes, I have the libva-intel-driver package installed.

GloriousEggroll · Nov 30, 2017

couple things:

1st, turns out my mesa version was 17.3 rc5, was doing some testing between the two and found this to be the version that worked.

2nd - test vaapi with vainfo command
if you don't get something like this, your vaapi driver isnt set up correctly:

Code:

[gloriouseggroll@shittywok lib32-mesa-git]$ vainfo 
libva info: VA-API version 1.0.0 
libva info: va_getDriverName() returns 0 
libva info: Trying to open /usr/lib/dri/radeonsi_drv_video.so 
libva info: Found init function __vaDriverInit_1_0 
libva info: va_openDriver() returns 0 
vainfo: VA-API version: 1.0 (libva 2.0.0) 
vainfo: Driver version: mesa gallium vaapi 
vainfo: Supported profile and entrypoints 
     VAProfileMPEG2Simple            : VAEntrypointVLD 
     VAProfileMPEG2Main              : VAEntrypointVLD 
     VAProfileVC1Simple              : VAEntrypointVLD 
     VAProfileVC1Main                : VAEntrypointVLD 
     VAProfileVC1Advanced            : VAEntrypointVLD 
     VAProfileH264ConstrainedBaseline: VAEntrypointVLD 
     VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice 
     VAProfileH264Main               : VAEntrypointVLD 
     VAProfileH264Main               : VAEntrypointEncSlice 
     VAProfileH264High               : VAEntrypointVLD 
     VAProfileH264High               : VAEntrypointEncSlice 
     VAProfileHEVCMain               : VAEntrypointVLD 
     VAProfileHEVCMain10             : VAEntrypointVLD 
     VAProfileNone                   : VAEntrypointVideoProc

to see which vaapi drivers are available on your system:
cd /usr/lib/dri
ls

my system example:

Code:

nouveau_drv_video.so  nvidia_drv_video.so  r600_drv_video.so  radeonsi_drv_video.so  s3g_drv_video.so  vdpau_drv_video.so

sheisrisen · Dec 12, 2017

I used your pkgbuild GloriousEggroll and it compiled/installed fine.
However, when I record my screen is covered with green and pink and is duplicated like so:

I've tried messing around with settings but nothing seems to change it.
My mesa version is only 17.2.6-1. Could this be the problem?
Any other ideas?

GloriousEggroll · Dec 13, 2017

the problem is mesa. This needs mesa 17.3 or higher to work. mesa-git/mesa-dev also works

Bleuzen · Dec 15, 2017

GloriousEggroll said:
the problem is mesa. This needs mesa 17.3 or higher to work. mesa-git/mesa-dev also works

I tryed it now with mesa 17.3 on my Nvidia + Intel system, but it still doesn't work and prints this error:

Code:

[AVHWDeviceContext @ 0x55ba2521dbe0] libva: /usr/lib/dri/i965_drv_video.so init failed
[AVHWDeviceContext @ 0x55ba2521dbe0] Failed to initialise VAAPI connection: -1 (unknown libva error).

vainfo:

Code:

libva info: VA-API version 1.0.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/nvidia_drv_video.so
libva info: Found init function __vaDriverInit_1_0
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.0 (libva 2.0.0)
vainfo: Driver version: Splitted-Desktop Systems VDPAU backend for VA-API - 0.7.4
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileMPEG4Simple            : VAEntrypointVLD
      VAProfileMPEG4AdvancedSimple    : VAEntrypointVLD
      <unknown profile>               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD

env LIBVA_DRIVER_NAME=i965 vainfo:

Code:

libva info: VA-API version 1.0.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'i965'
libva info: Trying to open /usr/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_0
libva error: /usr/lib/dri/i965_drv_video.so init failed
libva info: va_openDriver() returns -1
vaInitialize failed with error code -1 (unknown libva error),exit

So it seems like libva doesn't work at all on my Intel iGPU currently :/
It worked back when I had an AMD GPU. (Maybe the nvidia driver blocks the open source one somehow?)
I'm using Manjaro (based on Arch).

GloriousEggroll · Dec 21, 2017

When I tried on my intel/nvidia laptop a few weeks ago I was having problems getting it to work as well. Over the holiday break I'll be putting together an intel system from some parts I have around. I'll see if I can test it then, although for nvidia you should just use nvenc

Kai Hendry · Mar 23, 2018

Great work GloriousEggroll. Just trying it on https://www.twitch.tv/kaihendry/ and it works.

I think CPU went down from 70% to 7%?

https://www.twitch.tv/videos/241800171##

Marcedo · Mar 23, 2018

heyo - Wooaah. thats awesome! Didnt knew about till that thread came to my eyes now :)
Thank you for divin into that deep sea here! Also had a look at ffmpegs sources a while ago and wasnt very happy with their "style" .. libav seems to be _much cleaner, besides..
Ok -but as you suggested gstreamer which seems to work out of the box it might be a better choice.
cheers, Marcedo

GloriousEggroll · Apr 8, 2018

Hi, just an update on this. The hardcoded gpu issue has been resolved. I cleaned up the patch today and removed the non-supported options. I've submitted a pull request for this.

I was able to successfully record 1080p 60 fps on my desktop using the mesa radeonsi vaapi driver.
I was also able to simultaneously stream on my intel/nvidia laptop using nvenc to stream while recording with vaapi.

This works using the latest stable mesa 17.3 branch and higher.

pete910 · Apr 11, 2018

Just built your obs folk Glorios, Only thing that caught me out was I didn't have libva-mesa-driver installed.

Seems to work great (rx64). Will try later on a gaming session tonight.

Much appreciated !

pete910 · Apr 15, 2018

Well after some testing over the weekend it stutters/drops a lot of frames. If i set it higher that 3500 bitrate that I stream with. Have tried it recording to on a 10000 bitrate with the same issue.
Also get "encoder overload" Any suggestions ?

Antergos
RX64
Mesa 18

Kithop · May 7, 2018

After making sure I had the various Intel kernel config bits enabled, and Multi-GPU support enabled in my UEFI settings, so far it looks like I'm able to successfully use your patch to build a copy of OBS Studio that I can switch between NVENC and VA-API for Intel QuickSync. Will do some more testing for stability, options, and such, but this is on a Gentoo system (latest nvidia-drivers and media-libs/mesa-18.1.0_rc2) pulling https://github.com/GloriousEggroll/...t-ffmpeg-vaapi/blob/master/ffmpeg-vaapi.patch in as a userpatch (i.e., put a copy in /etc/portage/patches/media-video/obs-studio-9999 and re-emerge).

In my case, /dev/dri/renderD128 is the Intel driver, while I think D129 is nVidia (and doesn't work, of course). I can see in the terminal when starting to stream locally using D128 that it loads:

Code:

info: [FFMPEG VAAPI encoder: 'streaming_h264'] settings:
    device:       /dev/dri/renderD128
    qp:           20
    quality:      0
    profile:      578
    level:        42
    bitrate:      6000
    keyint:       120
    width:        1920
    height:       1080
    b-frames:     0

libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/va/drivers/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
info: ---------------------------------
info: [FFmpeg aac encoder: 'Track1'] bitrate: 160, channels: 2, channel_layout: 3

Gentoo-specific, I have 'nvenc' in my USE flags so OBS builds with NVENC support, and I have OBS' ebuild set to always compile from the latest git master (i.e., add '~media-video/obs-studio-9999 **' to /etc/portage/package.accept_keywords). I've got the required Intel bits in the kernel config from https://wiki.gentoo.org/wiki/Intel#Kernel as modules where possible, specifically leaving Nouveau out completely as not to conflict with the binary nVidia driver.

GloriousEggroll · May 31, 2018

pete910 said:
Well after some testing over the weekend it stutters/drops a lot of frames. If i set it higher that 3500 bitrate that I stream with. Have tried it recording to on a 10000 bitrate with the same issue.
Also get "encoder overload" Any suggestions ?

Antergos
RX64
Mesa 18

Your dropped frames are due to encoder overload, which correlates to the strength of the card's encoding hardware. Regarding encoder overload:
https://github.com/obsproject/obs-studio/pull/1256#issuecomment-393018169
https://github.com/GloriousEggroll/arch-obs-studio-latest-ffmpeg-vaapi/issues/5

nunks · Jul 12, 2018

This is awesome, @w23 and @GloriousEggroll ! CPU usage drops by a factor of 10 and now my old laptop is able to work at 60fps =)

One question, though, I apologize in advance if I'm being stupid: does the interface only support CBR? How can I setup VBR or CQP encoding as per FFmpeg documentation? I see by the patch code that it sets a default qp of 20, but how can I set it to a different value?

Experimental ffmpeg-vaapi plugin

New Member

Member

Member

Member

New Member

Member

New Member

Member

New Member

Member

New Member

Member

New Member

Member

Member

New Member

New Member

New Member

Member

New Member