Output -> Recording -> Custom FFMPEG Output to URL with VAAPI

[Gavin H]

I'm running OBS 25.0.8 on a physical (hetzner.de) Ubuntu 20.04 remote server inside 'TigerVNC Server' - the 'desktop' session does not have acceleration via the usual X11 GLX mechanism, however access via VAAPI on the commandline works fine.

dj@Ubuntu-2004-focal-64-minimal:~$ vainfo --display drm | grep H264.*Enc
libva info: VA-API version 1.7.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
      VAProfileH264Main               :    VAEntrypointEncSlice
      VAProfileH264Main               :    VAEntrypointEncSliceLP
      VAProfileH264High               :    VAEntrypointEncSlice
      VAProfileH264High               :    VAEntrypointEncSliceLP
      VAProfileH264ConstrainedBaseline:    VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:    VAEntrypointEncSliceLP

e.g. this command works perfectly from an XFCE Terminal and eats very few CPU %

ffmpeg -init_hw_device vaapi=vaapi:/dev/dri/renderD128 -filter_hw_device vaapi -i rtmp://myserver/path/to/stream -c:v h264_vaapi -vf "format=nv12,hwupload" -y  output.mp4

If I select 'Video Encoder' as libx264 in OBS, it does work and send the video to the correct place, but of course it's using software libx264 encoding with high CPU %. If I change the Video Encoder to h264_vaapi I receive the generic error:

11:37:42 PM.338: Failed to open video codec: Invalid argument
11:37:42 PM.338: ffmpeg_data_init failed
11:37:42 PM.358: ==== Recording Stop ================================================

How can I shoehorn the extra 'magic incantations' regarding init_hw_device and 'format=nv12,hwupload' to the Output -> Recording -> Custom FFMPEG panel?

Cheers,
Gavin.

 

[Tuna]

FFMPEG may be doing more here. Pretty sure VAAPI wants an existing surface to work with, which as you have recognized you don't have. FFMPEG may instantiate an off screen surface to do so which OBS doesn't.

OBS was not designed with you use-case in mind. Note that if you don't have an OpenGL session the composition step is also being done in software.

 

[Gavin H]

Yup, 100% agree this is a fudge - and that's a great observation about the composition - it certainly explains why the CPU usage sits at 30-40% even when not streaming or recording. I'll take a look at another approach :) Thanks for the reply.