# Experimental ffmpeg-vaapi plugin



## w23 (Jan 16, 2017)

So I really wanted to stream Clustertruck in 1440p60, so I spent a whole weekend reading ffmpeg sources instead.
As a result, there's a very naive ffmpeg-vaapi plugin (basically is a copy of ffmpeg-nvenc with vaapi-specific hw frame upload added) in the obs-ffmpeg module in this branch: https://github.com/w23/obs-studio/tree/ffmpeg-vaapi
The exact commit adding it: https://github.com/w23/obs-studio/commit/9c70ee2347285c4d7e087106c565ba5b5bbe16a6

It is in a rather early stage:

no GUI controls
display/device name is hardcoded to ":0"
memory management/leaks were not considered at all
actual performance wasn't measured

I'd appreciate any kind of feedback if anyone is interested.

My experience playing with it for a few hours so far:

FFmpeg@master, Mesa 13.0.3, VA-API 0.39 (libva 1.7.3), kernel 4.8.12 on AMD Radeon R9 Fury X: there are weird issues with the video it produces. Basically it looks like ffmpeg+h264_vaapi emits packets very rarely, only a couple per second. Which could be fine otherwise, but the thing is that this packet rate equals apparent framerate of the produced video (1440p2 is not what I wanted!). Also, I don't visually see any P-frames at all. Tuning gop_size or other parameters doesn't make this packet rate better. This HW also requires b-frames to be set to zero, and VAAPI_DISABLE_INTERLACE=1 envvar set.
On another hw (some intel on dell xps'13 from 2015) that I've tested very-very briefly, the stream is also low-fps and jittery, but not that much (likely the vaapi is fine, but the machine itself is rather weak). And p-frames are clearly visible.

Thing is that this jitter is not obs-specific. For example, running a screen capture with ffmpeg itself produces the same (if not actually worse) result:

```
ffmpeg \
    -loglevel debug \
    -f x11grab -video_size 2560x1440 -framerate 60 -x 1920 -i :0.0 \
    -vaapi_device ":0" \
    -vf 'format=nv12,hwupload' -map 0:0 -threads 8 -aspect 16:9 -y -f mp4 \
    -bf 0 -qp 42 -quality 8 \
    -vcodec h264_vaapi -profile 100 \
    test-vaapi.mp4
```

Testing instructions.
0. libva is obviously needed.
1. Rather fresh ffmpeg with h264_vaapi encoding support is required. I took latest master (and I probably shouldn't have done that! This would be ironic if the issues above are due to an dev-unstable ffmpeg).
If you need to compile ffmpeg yourself, then you'd need it to have at least the following options for its ./configure:

```
--enable-shared --enable-pic --disable-static \
--enable-hwaccel=h264_vaapi \
--enable-filter=hwupload,scale \
--enable-encoder=h264_vaapi,aac \
--enable-muxer=h264,mp4,flv,md5 \
--enable-protocol=file,rtmp \
--enable-decoder=rawvideo
```

Also for cmdline ffmpeg testing:

```
--enable-indev=v4l2,x11grab_xcb,xcbgrab \
--enable-parser=mjpeg \
--enable-decoder=mjpeg
```

Also also, don't forget to set envvar PKG_CONFIG_PATH=<where-you-installed-ffmpeg>/lib/pkgconfig before you run cmake on OBS.
2. Build OBS and run it as usual. Go to advanced and pick VAAPI encoder.


----------



## Jim (Jan 23, 2017)

Just popping in to say that this is awesome that you wrote this.  Unfortunately at the moment my dedicated linux machine is down so I can't test, but I'm going to have some other people try this out in the mean time and see how it runs for them with different hardware.


----------



## Xaymar (Jan 23, 2017)

As far as I know, VAAPI is only supported officially for decoding on AMD gpus, encoding is afaik only available with mesa drivers. So you might need to find mesa drivers where it works or straight up blacklist AMD cards for now.

There's a ticket open for on the AMF issue tracker: https://github.com/GPUOpen-LibrariesAndSDKs/AMF/issues/4


----------



## w23 (Jan 24, 2017)

Thanks! I have spent a lot more time on this HW encoding on Linux problem. Here's what I found.

TL;DR: I could get AMD GPU to encode h264 only using gstreamer-vaapi.

Mesa (as of 13.0.3) does support hardware h264 encoding on AMD GPUs. However, there are limitations: vaDeriveImage() function always fails, there is no support for B-frames and packed headers. These are pretty much hardcoded in Mesa VAAPI state tracker, so hardware is not even asked of its capabilities. Have no idea what no packed headers mean (haven't read the MPEG4 AVC spec {yet?:|}), and also not sure about implications of not having B-frames for streaming games. No vaDeriveImage is also not fatal, but it means that we can't direcly map to GPU mem, so there's a performance loss by yet another memcpy (and a more complicated codepath).
And another thing, there's a bug where AMD driver interprets everything as interlaced (despite what's been told via libva API), so one should always have VAAPI_DISABLE_INTERLACE=1 in the environment all the time.

I couldn't make any version of FFmpeg to correctly make use of VAAPI. The *release* (3.2.2) version just crashes. Current master complains that packed headers aren't there and produces this low-framerate output that I talked about above.

The official libva tests/samples also don't work, because they expect vaDeriveImage to work.

There already was another VAAPI plugin for OBS made a few years ago: https://github.com/reboot/obs-studio/tree/vaapi-h264/plugins/linux-vaapi. Making it compile and load with contemporary OBS is trivial. But I couldn't make it work on my driver. Basically, it expects h264 packed headers, which aren't there.

The only thing that does work, and seems to work sufficiently well is using gstreamer-vaapi. This command does produce a valid 1440p60 video while using under 15% CPU: 

```
gst-launch-1.0 -e ximagesrc display-name=:0 use-damage=0 startx=1920 starty=0 endx=$((1920+2560-1)) endy=1439 !\
    multiqueue ! video/x-raw,format=BGRx,framerate=60/1 ! videoconvert ! video/x-raw,format=I420,framerate=60/1 !\
    multiqueue ! vaapih264enc dct8x8=true ! h264parse ! multiqueue ! matroskamux name=muxer muxer. ! progressreport name=Rec_time !\
    filesink location=/tmp/gstreamer-video.mkv
```
However:
- I haven't tried to use it for longer than a few minutes.
- Capturing frames still does interfere with games. E.g. Clustertruck (which triggered all this endeavor!) still experiences frame drops when capturing. This needs to be profiled.

VAAPI seems to have no way of accessing hardware framebuffer. It can use GLX context and texture for output, but not for input. Maybe its possible to use lower level dri2 apis to do something like that, but this is way beyond my immediate capabilities.

Or maybe it would be possible to write a special Xorg compositor that could capture frames at a lower level and more efficiently. I have no idea.

So, a conclusion:
1. The only way forward for me is to make _yet another_ VAAPI plugin for OBS, this time based on gstreamer. From the looks of it, gstreamer seems to be documented and sane (yes, I am looking at you, FFmpeg), so maybe this or next weekend I will come up with something.
2. I need to profile the hell out of all this if I ever want to share with my friends how bad I am at Clustertruck.


----------



## ZombieMeat (Feb 1, 2017)

You are the best! Just created an account to tell you that.

I've been able to test it out on Dell XPS 13(Kaby Lake), with a few more lines to let it be able to tweak the per-encoder options.

I would say my experience have been good, although I haven't used it for a long period. The only thing of note is that higher value for "quality" means faster or lower quality, per "ffmpeg -h encoder=h264_vaapi." And, they are not even supported on intel GPU, with only 0 and 1 supported while other values give crashes.


----------



## beniwtv (Feb 2, 2017)

Nice work! Going to try that out this weekend - have hoped someone would implement this!
I'm going to try with a AMD RX 480 8GB - MESA 13.0.4.


----------



## ZombieMeat (Feb 3, 2017)

Been playing around a bit. It seems like the most significant parameter is QP. Also, Specifying bitrate made FFmpeg behave a bit funky. It forces the bitrate to be consistent with the specified one even when it doesn't need to. Moreover, when the image changes drastically it needs more bits to encode: the bitrate rises; the buffer overflows, which seems to degrade the encoding quality quite a bit.

I'm all new to this so just conjecturing, but instead of setting bitrate, what if we only set the buffer size to reduce the possibility of overflow. Right now, QP and buffer size combination needs to be tested empirically. 

In case anyone is interested, I made a PKGBUILD(archlinux) for my test setup.


----------



## Xaymar (Feb 3, 2017)

The easiest way to explain that is to know the VCE part responds to certain parameters. There's several ones which affect what values are picked on the hardware but lets go with the absolute default one (Usage: Transcoding). Since I don't know what Mesa's VAAPI integration all actually sets, this is mostly from the actual usage on Windows, which should be identical since it maps almost directly to the hardware from my experience (+ some gpu transfer/conversion stuff).

There are three main Rate Control Methods that VCE has: Constant QP, Constant Bitrate and Variable Bitrate. Variable Bitrate has a Peak Constrained and a Latency Constrained version (latter is great for recording with no impact, former is great for actual quality). Constant Bitrate is the only one of these where VCE uses Filler Data, though normally only if enabled. If Constant Bitrate is used without Filler Data, it behaves like Peak Constrained Variable Bitrate except that the Target Bitrate is the Peak Bitrate and Peak Bitrate is ignored.

So in order to actually get Variable Bitrate behaviour, FFMPEG would need to be configured to use VBR mode. I'm not sure if Mesa's VAAPI exposes this.

As for Buffer Size (VBV Buffer Size), if you want your Bitrate to be perfectly matched you'd want a value between 1/FPS*Bitrate and 8/FPS*Bitrate. The lower you go the less space an individual packet can take up (directly affects I&P&B Frame quality.


----------



## beniwtv (Feb 5, 2017)

So I got around to test this now, and after spending the whole day I have to report success!

I used the PKGBUILD files that @ZombieMeat provided above - allthough adapted for Docker and Ubuntu.

With FFmpeg 3.2.2 it crashes - just like @w23 reported. Also it complains about no B-frames and crashes if you set these to 0.
With FFmpeg master from today (5. Feb, 2017), it no longer complains about B-frames and does not crash.

Actually, it seems to encode just fine - and CPU does stay low - the same as if you're not encoding :)
I did notice some text artifacts - but I have not yet been playing around with the options provided (just leaving them standard as they came with it).

So my final configuration was:
MESA 17.0.1-devel from Padoka ppa
Docker using Ubuntu 16.04 as base
AMD RX 480

I have attached my Docker files in case anyone wants to have a quick way of testing :)
NOTE: The start script is currently hard-coded for display :0 and UID 1000

EDIT: Made a video
https://www.youtube.com/watch?v=m8OBFLaNl5Q


----------



## petunder (Feb 5, 2017)

I'm very excited, but how to install pkgbuild in ubuntu? Can you build a .deb? 
Thank you!


----------



## David Cooper (Mar 15, 2017)

I had no trouble merging the reboot/vaaapi-h264 tree into latest master, building, and installing. I can upload a .DEB if you folks would like. I made sure that the libva* deps were installed.


----------



## Angry Penguin (Mar 16, 2017)

David Cooper said:


> I had no trouble merging the reboot/vaaapi-h264 tree into latest master, building, and installing. I can upload a .DEB if you folks would like. I made sure that the libva* deps were installed.



If you can do it. I would be very grateful (probably not only me) :)


----------



## w23 (Mar 19, 2017)

My apologies for long absence and no progress on my plugin.
The thing is I figured out that VAAPI doesn't help me with the capture performance problems I have on my system, and the actual bottleneck is somewhere else (likely XSHM). Therefore, I don't think I will be making any progress here soon. If anyone wants to take the plugin and make it production ready, be my guest. I believe the only major thing left to do is to add proper GUI controls. License is whatever license OBS is under.

I want to do a thorough Linux screen capture performance research (including things like custom compositors, Wayland and friends) in the coming months. But I cannot promise anything as there is just too much stuff on my plate already.


----------



## Steeled_Pick (Apr 25, 2017)

Tried to compile but getting the following error and it fails: /home/user/Downloads/obs-studio-vaapi-h264/plugins/linux-vaapi/surface-queue.h:3:19: fatal error: va/va.h: No such file or directory
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2
ubuntu 16.04
ffmpeg version 3.2.4

Fixed: I forgot to install libva-dev


----------



## Bleuzen (Apr 28, 2017)

Works very well on Arch with an i7-7700 iGPU (Intel® HD Graphics 630). I just had to change the display/device name from ":0" to "/dev/dri/renderD128".
Thanks! Now I can record / stream without much CPU usage.
Please implement this in OBS (with GUI) ... this would be great ;D


----------



## David Carver (Jun 7, 2017)

Going to try this out with 16.10 of Ubuntu tonight.  It has to be better than the horrible experience I had trying to get QSV support working.  At least this doesn't require a new kernel compilation.


----------



## David Carver (Jun 8, 2017)

I managed to get the reboot/vaapi-h264 version working with the master branch of obs-studio.   No changes were necessary to get it to compiled besides what was in his base branch.  It would be fantastic if somebody could get this incorporated into the main obs-studio code base.   I can create a branch and pull request but we really should contact Reboot to get permission to include his work into the main distribution.

My CPU usage went from hitting 19 to 20 percent, to between 5% and 8% with using the VAAPI-H264 encoder settings.

Update:  I created a simple blog entry with a link to Reboot's code working with the current master branch (19.x) In case anybody is interested.   No problems merging the code in and getting it to work.  Here is the link.

https://wordpress.com/post/intellectualcramps.wordpress.com/1151


----------



## Arjen (Jun 16, 2017)

I tried the merge git repo, it all seems to go quite well until I press 'stop recording'. The terminal then shows the following error message:


```
info: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/arjen/Videos/2017-06-16_16-09-32.mkv'...
error: [VAAPI encoder]: "vaEndPicture(q->display, q->context)": invalid parameter
error: [VAAPI encoder]: unable to encode frame
error: Error encoding with encoder 'streaming_h264'
```

My vainfo output is as follows:


```
vainfo: VA-API version: 0.40 (libva )
vainfo: Driver version: Intel i965 driver for Intel(R) Haswell Mobile - 1.8.2
vainfo: Supported profile and entrypoints
  VAProfileMPEG2Simple  :   VAEntrypointVLD
  VAProfileMPEG2Simple  :   VAEntrypointEncSlice
  VAProfileMPEG2Main  :   VAEntrypointVLD
  VAProfileMPEG2Main  :   VAEntrypointEncSlice
  VAProfileH264ConstrainedBaseline:   VAEntrypointVLD
  VAProfileH264ConstrainedBaseline:   VAEntrypointEncSlice
  VAProfileH264Main  :   VAEntrypointVLD
  VAProfileH264Main  :   VAEntrypointEncSlice
  VAProfileH264High  :   VAEntrypointVLD
  VAProfileH264High  :   VAEntrypointEncSlice
  VAProfileH264MultiviewHigh  :   VAEntrypointVLD
  VAProfileH264MultiviewHigh  :   VAEntrypointEncSlice
  VAProfileH264StereoHigh  :   VAEntrypointVLD
  VAProfileH264StereoHigh  :   VAEntrypointEncSlice
  VAProfileVC1Simple  :   VAEntrypointVLD
  VAProfileVC1Main  :   VAEntrypointVLD
  VAProfileVC1Advanced  :   VAEntrypointVLD
  VAProfileNone  :   VAEntrypointVideoProc
  VAProfileJPEGBaseline  :   VAEntrypointVLD
```

Any ideas?


----------



## cRaZy-bisCuiT (Aug 1, 2017)

David Carver said:


> Update:  I created a simple blog entry with a link to Reboot's code working with the current master branch (19.x) In case anybody is interested.   No problems merging the code in and getting it to work.  Here is the link.
> 
> https://wordpress.com/post/intellectualcramps.wordpress.com/1151


Unfortunately I can't read that blog entry: Wordpress asks me for my login credentials. Could you check that address please?

Also, has someone else managed to merge the patch with the current master of obs? Is there a tutorial somewhere? Thanks to all girls & guys participating here! :)


----------



## RytoEX (Aug 1, 2017)

cRaZy-bisCuiT said:


> Unfortunately I can't read that blog entry: Wordpress asks me for my login credentials. Could you check that address please?
> 
> Also, has someone else managed to merge the patch with the current master of obs? Is there a tutorial somewhere? Thanks to all girls & guys participating here! :)



I assume @David Carver meant this URL:  https://intellectualcramps.wordpress.com/2017/06/08/obs-studio-and-hardware-encoding-for-linux/

There is a pull request on GitHub, which I was able to successfully compile in a VM, but I didn't extensively test it.  As far as I know, it's currently waiting on some pretty substantial rewrites.


----------



## bywant (Aug 9, 2017)

Has anyone tried this methode with libva from Intel Media Server Studio?

i am getting this error in logs:
libva info: VA-API version 0.99.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
warning: [VAAPI encoder]: found no exact match for profile, defaulting to best match 'constrained baseline'
error: [VAAPI encoder]: "vaCreateContext(enc->display, enc->config, enc->width, enc->height, 0x1, ((void *)0), 0, &enc->context)": resolution not supported
error: [VAAPI encoder]: failed to initialize encoder for profile constrained baseline

Logfile:
https://gist.github.com/anonymous/66c48de31926dbee14678886f9333990


----------



## GloriousEggroll (Nov 19, 2017)

Arch users:

So I made a patch out of this so it could easily be used on arch with obs-studio-git on the AUR rather than cloning + merging the repo every time there's an update. I've also edited my pkgbuild to build the latest release instead of the latest git commit version.

I tested this on mesa 17.3 on arch on a vega 64, working well!

https://github.com/GloriousEggroll/arch-obs-studio-latest-ffmpeg-vaapi

easy steps:

git clone https://github.com/GloriousEggroll/arch-obs-studio-latest-ffmpeg-vaapi
cd arch-obs-studio-latest-ffmpeg-vaapi
makepkg -i


----------



## GloriousEggroll (Nov 19, 2017)

David Carver said:


> I managed to get the reboot/vaapi-h264 version working with the master branch of obs-studio.   No changes were necessary to get it to compiled besides what was in his base branch.  It would be fantastic if somebody could get this incorporated into the main obs-studio code base.   I can create a branch and pull request but we really should contact Reboot to get permission to include his work into the main distribution.
> 
> My CPU usage went from hitting 19 to 20 percent, to between 5% and 8% with using the VAAPI-H264 encoder settings.
> 
> ...




is there any benefit to using this branch as opposed to w23's version?

I've just made a patch out of w23's version and have it working on mesa 17.3 rc5 with amd vega 64

edit: I tested both. w23's worked successfully with mesa, while reboot's did not work on mesa or with libva-vdpau-driver installed. unsure if it works using intel's non-mesa vaapi driver, was not able to test as I also have an amd cpu

edit 2: according to https://wiki.libav.org/Hardware/vaapi the device should be
/dev/dri/renderD128
rather than
/dev/dri/card#

even with this change, it doesn't work for me, and gets stuck at the "stopping recording" button


----------



## GloriousEggroll (Nov 25, 2017)

Hi, so I went in and cleaned up the patch, added the ability to choose which card to use, added a few missing options, and updated options which should override eachother (ie bitrate/qp and b-frames/high profile). I've also added some basic support for other codecs (hevc, mpeg2, vp8, vp9). I've made a video about it here:

https://youtu.be/s2EZ_H-POSM

I've made a patch here:
https://github.com/GloriousEggroll/...t-ffmpeg-vaapi/blob/master/ffmpeg-vaapi.patch

I've forked obs and added the patch here:
https://github.com/GloriousEggroll/obs-studio

screenshot:






I plan to submit a pull request as soon as I get more input on the intel side of things.

Have tested this using vega 64, I was also able to stream and record simultaneously using vaapi.

I did have to make a quick hack so that it uses 1072 instead of 1080 so that no weird green bars show up while streaming. This is because the dimensions have to be divisible by 16 for h.264, and setting it at 1080 auto-bumps it to 1088, and 1088 causes an ugly green line at the bottom on twitch.tv. other resolutions worked fine with no bar issues.


----------



## Bleuzen (Nov 28, 2017)

@GloriousEggroll
I can't compile on Manjaro (Arch based):




I have a NVIDIA card, but wanted to try out to use my Intel iGPU with OBS.
(Maybe the Nvidia driver blocks the open source driver? But would this end up in such a compile error?)

My
GPU: GTX 1060
CPU: i7-7700


----------



## GloriousEggroll (Nov 29, 2017)

ah its a small error with the profile name, it should be FF_PROFILE_HEVC_MAIN_10, fixed in both repos now. You may need to try libva_driver_name envar befor launching obs as nvidia tends to try to take over as the default vaapi driver. I wasnt able to get it working on my laptop, which is intel+nvidia


----------



## Bleuzen (Nov 29, 2017)

I also don't get it working. I started obs just with the
_obs_
command and got this error:
_[AVHWDeviceContext @ 0x55a3b4756340] libva: va_getDriverName() failed with unknown libva error,driver_name=(null) 
[AVHWDeviceContext @ 0x55a3b4756340] Failed to initialise VAAPI connection: -1 (unknown libva error).                                                                                                                                          
warning: [FFMPEG VAAPI encoder: 'recording_h264'] Failed to create VAAPI device context: Eingabe-/Ausgabefehler   _


Also i tryed to start it so:
_env LIBVA_DRIVER_NAME=i965 obs
_
and got this error:
_[AVHWDeviceContext @ 0x5637b79b1140] libva: /usr/lib/dri/i965_drv_video.so init failed 
[AVHWDeviceContext @ 0x5637b79b1140] Failed to initialise VAAPI connection: -1 (unknown libva error).                                                                                                                                          
warning: [FFMPEG VAAPI encoder: 'recording_h264'] Failed to create VAAPI device context: Eingabe-/Ausgabefehler    _

And yes, I have the_ libva-intel-driver _package installed.


----------



## GloriousEggroll (Nov 30, 2017)

couple things:

1st, turns out my mesa version was 17.3 rc5, was doing some testing between the two and found this to be the version that worked.

2nd - test vaapi with vainfo command
if you don't get something like this, your vaapi driver isnt set up correctly:

```
[gloriouseggroll@shittywok lib32-mesa-git]$ vainfo 
libva info: VA-API version 1.0.0 
libva info: va_getDriverName() returns 0 
libva info: Trying to open /usr/lib/dri/radeonsi_drv_video.so 
libva info: Found init function __vaDriverInit_1_0 
libva info: va_openDriver() returns 0 
vainfo: VA-API version: 1.0 (libva 2.0.0) 
vainfo: Driver version: mesa gallium vaapi 
vainfo: Supported profile and entrypoints 
     VAProfileMPEG2Simple            : VAEntrypointVLD 
     VAProfileMPEG2Main              : VAEntrypointVLD 
     VAProfileVC1Simple              : VAEntrypointVLD 
     VAProfileVC1Main                : VAEntrypointVLD 
     VAProfileVC1Advanced            : VAEntrypointVLD 
     VAProfileH264ConstrainedBaseline: VAEntrypointVLD 
     VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice 
     VAProfileH264Main               : VAEntrypointVLD 
     VAProfileH264Main               : VAEntrypointEncSlice 
     VAProfileH264High               : VAEntrypointVLD 
     VAProfileH264High               : VAEntrypointEncSlice 
     VAProfileHEVCMain               : VAEntrypointVLD 
     VAProfileHEVCMain10             : VAEntrypointVLD 
     VAProfileNone                   : VAEntrypointVideoProc
```

to see which vaapi drivers are available on your system:
cd /usr/lib/dri
ls

my system example:

```
nouveau_drv_video.so  nvidia_drv_video.so  r600_drv_video.so  radeonsi_drv_video.so  s3g_drv_video.so  vdpau_drv_video.so
```


----------



## sheisrisen (Dec 12, 2017)

I used your pkgbuild GloriousEggroll and it compiled/installed fine.
However, when I record my screen is covered with green and pink and is duplicated like so:





I've tried messing around with settings but nothing seems to change it.
My mesa version is only 17.2.6-1. Could this be the problem?
Any other ideas?


----------



## GloriousEggroll (Dec 13, 2017)

the problem is mesa. This needs mesa 17.3 or higher to work. mesa-git/mesa-dev also works


----------



## Bleuzen (Dec 15, 2017)

GloriousEggroll said:


> the problem is mesa. This needs mesa 17.3 or higher to work. mesa-git/mesa-dev also works


I tryed it now with mesa 17.3 on my Nvidia + Intel system, but it still doesn't work and prints this error:

```
[AVHWDeviceContext @ 0x55ba2521dbe0] libva: /usr/lib/dri/i965_drv_video.so init failed
[AVHWDeviceContext @ 0x55ba2521dbe0] Failed to initialise VAAPI connection: -1 (unknown libva error).
```
vainfo:

```
libva info: VA-API version 1.0.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/nvidia_drv_video.so
libva info: Found init function __vaDriverInit_1_0
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.0 (libva 2.0.0)
vainfo: Driver version: Splitted-Desktop Systems VDPAU backend for VA-API - 0.7.4
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileMPEG4Simple            : VAEntrypointVLD
      VAProfileMPEG4AdvancedSimple    : VAEntrypointVLD
      <unknown profile>               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
```
env LIBVA_DRIVER_NAME=i965 vainfo:

```
libva info: VA-API version 1.0.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'i965'
libva info: Trying to open /usr/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_0
libva error: /usr/lib/dri/i965_drv_video.so init failed
libva info: va_openDriver() returns -1
vaInitialize failed with error code -1 (unknown libva error),exit
```

So it seems like libva doesn't work at all on my Intel iGPU currently :/
It worked back when I had an AMD GPU. (Maybe the nvidia driver blocks the open source one somehow?)
I'm using Manjaro (based on Arch).


----------



## GloriousEggroll (Dec 21, 2017)

When I tried on my intel/nvidia laptop a few weeks ago I was having problems getting it to work as well. Over the holiday break I'll be putting together an intel system from some parts I have around. I'll see if I can test it then, although for nvidia you should just use nvenc


----------



## Kai Hendry (Mar 23, 2018)

Great work GloriousEggroll. Just trying it on https://www.twitch.tv/kaihendry/ and it works. 

I think CPU went down from 70% to 7%?

https://www.twitch.tv/videos/241800171##


----------



## Marcedo (Mar 23, 2018)

heyo - Wooaah. thats awesome! Didnt knew about till that thread came to my eyes now :)
Thank you for divin into that deep sea here! Also had a look at ffmpegs sources a while ago and wasnt very happy with their "style"  .. libav seems to be _much cleaner, besides..
Ok -but as you suggested gstreamer which seems to work out of the box it might be a better choice.
cheers, Marcedo


----------



## GloriousEggroll (Apr 8, 2018)

Hi, just an update on this. The hardcoded gpu issue has been resolved. I cleaned up the patch today and removed the non-supported options. I've submitted a pull request for this.

I was able to successfully record 1080p 60 fps on my desktop using the mesa radeonsi vaapi driver.
I was also able to simultaneously stream on my intel/nvidia laptop using nvenc to stream while recording with vaapi.

This works using the latest stable mesa 17.3 branch and higher.


----------



## pete910 (Apr 11, 2018)

Just built your obs folk Glorios, Only thing that caught me out was I didn't have libva-mesa-driver installed. 

Seems to work great (rx64). Will try later on a gaming session tonight. 

Much appreciated !


----------



## pete910 (Apr 15, 2018)

Well after some testing over the weekend it stutters/drops a lot of frames. If i set it higher that 3500 bitrate that I stream with. Have tried it recording to on a 10000 bitrate with the same issue.
Also get "encoder overload" Any suggestions ? 

Antergos
RX64
Mesa 18


----------



## Kithop (May 7, 2018)

After making sure I had the various Intel kernel config bits enabled, and Multi-GPU support enabled in my UEFI settings, so far it looks like I'm able to successfully use your patch to build a copy of OBS Studio that I can switch between NVENC and VA-API for Intel QuickSync.  Will do some more testing for stability, options, and such, but this is on a Gentoo system (latest nvidia-drivers and media-libs/mesa-18.1.0_rc2) pulling https://github.com/GloriousEggroll/...t-ffmpeg-vaapi/blob/master/ffmpeg-vaapi.patch in as a userpatch (i.e., put a copy in /etc/portage/patches/media-video/obs-studio-9999 and re-emerge).




In my case, /dev/dri/renderD128 is the Intel driver, while I think D129 is nVidia (and doesn't work, of course).  I can see in the terminal when starting to stream locally using D128 that it loads:


```
info: [FFMPEG VAAPI encoder: 'streaming_h264'] settings:
    device:       /dev/dri/renderD128
    qp:           20
    quality:      0
    profile:      578
    level:        42
    bitrate:      6000
    keyint:       120
    width:        1920
    height:       1080
    b-frames:     0

libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/va/drivers/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
info: ---------------------------------
info: [FFmpeg aac encoder: 'Track1'] bitrate: 160, channels: 2, channel_layout: 3
```

Gentoo-specific, I have 'nvenc' in my USE flags so OBS builds with NVENC support, and I have OBS' ebuild set to always compile from the latest git master (i.e., add '~media-video/obs-studio-9999 **' to /etc/portage/package.accept_keywords).  I've got the required Intel bits in the kernel config from https://wiki.gentoo.org/wiki/Intel#Kernel as modules where possible, specifically leaving Nouveau out completely as not to conflict with the binary nVidia driver.


----------



## GloriousEggroll (May 31, 2018)

pete910 said:


> Well after some testing over the weekend it stutters/drops a lot of frames. If i set it higher that 3500 bitrate that I stream with. Have tried it recording to on a 10000 bitrate with the same issue.
> Also get "encoder overload" Any suggestions ?
> 
> Antergos
> ...




Your dropped frames are due to encoder overload, which correlates to the strength of the card's encoding hardware. Regarding encoder overload:
https://github.com/obsproject/obs-studio/pull/1256#issuecomment-393018169
https://github.com/GloriousEggroll/arch-obs-studio-latest-ffmpeg-vaapi/issues/5


----------



## nunks (Jul 12, 2018)

This is awesome, @w23 and @GloriousEggroll ! CPU usage drops by a factor of 10 and now my old laptop is able to work at 60fps =)

One question, though, I apologize in advance if I'm being stupid: does the interface only support CBR? How can I setup VBR or CQP encoding as per FFmpeg documentation? I see by the patch code that it sets a default qp of 20, but how can I set it to a different value?


----------



## scurrvy2020 (Jul 16, 2018)

GloriousEggroll, thanks so much for this.  I compiled the latest release with your patch and it works great on my intel machine.  However, I'm having problems with my HD7770 (AMD southern islands) GPU.  I have many dropped frames and it takes about 2 minutes to stop recording a 10 second screen capture.

I'd be grateful if you have any tips.

edit: I forgot to mention that I am on ubuntu 18.04 with 4.15 kernel, oibaf xorg updates, standard repo ffmpeg (ffmpeg -hwaccels lists vaapi).


```
info: ---------------------------------
info: video settings reset:
    base resolution:   1920x1080
    output resolution: 1920x1080
    downscale filter:  Bicubic
    fps:               24000/1001
    format:            NV12
    YUV mode:          601/Partial
info: Settings changed (video)
info: ------------------------------------------------
info: ---------------------------------
info: [FFMPEG VAAPI encoder: 'recording_h264'] settings:
    device:       /dev/dri/renderD128
    qp:           20
    quality:      0
    profile:      578
    level:        40
    bitrate:      2500
    keyint:       120
    width:        1920
    height:       1080
    b-frames:     0

mesa: for the -simplifycfg-sink-common option: may only occur zero or one times!
[h264_vaapi @ 0x56246e79f820] Warning: some packed headers are not supported (want 0xd, got 0).
info: libfdk_aac encoder created
info: libfdk_aac bitrate: 160, channels: 2
info: ==== Recording Start ===============================================
info: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/pc/2018-07-16 20-58-32.flv'...
[flv @ 0x563b3e23c400] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[flv @ 0x563b3e23c400] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
info: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/pc/2018-07-16 20-58-32.flv' stopped
info: Output 'adv_file_output': stopping
info: Output 'adv_file_output': Total frames output: 181
info: Output 'adv_file_output': Total drawn frames: 1278 (1281 attempted)
info: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 3 (0.2%)
info: ==== Recording Stop ================================================
info: Video stopped, number of skipped frames due to encoding lag: 197/217 (90.8%)
info: libfdk_aac encoder destroyed
```


----------



## cRaZy-bisCuiT (Jul 25, 2018)

Dear GloriousEggroll,
thank you very much for your work! I'll try out tonight if Intel GPU and my RX 480 will succeed.

In addition: Can we expect this to be merged to mainline OBS any time soon? Is there already a pull request you could link here?


----------

