w23
New Member
I was having performance issues when streaming livecoding heavy GLSL raymarching and pathtracing shaders, so instead of optimizing my shaders I read about libdrm, KMS, DMA-BUF and EGL. As a result, I made a very experimental zero-copy screen capture OBS plugin for Linux based on DMA-BUF fds and EGLImages.
It does solve most of my performance woes, and you can see this screen capture method in action here: https://youtu.be/L2Y7_vBfWm8 or here https://www.twitch.tv/videos/389112056
Notice how even for heavy shaders it still maintains real-time fps. Vanilla OBS with XSHM would struggle capturing it and would barely keep ~10 fps, even though the shader itself can be as high as 40-60fps.
Background
There's this DRM infrastructure that is used to talk to GPUs in a vendor-agnostic way on Linux. Among other things it incorporates:
Among these extensions we can find:
I bet you can see that all of these fit together perfectly. Moreover, this can be used to capture not only X11, but everything else that is backed by KMS, including Wayland and even bare terminals!
One caveat though is that getting framebuffer DMA-BUF fd requires
However, you can remember such things as UNIX sockets, sendmsg() functions and SCM_RIGHTS flags. These can be used to transfer open file descriptors between processes. This also works on DMA-BUF fds, and so we can construct a small binary with `
Now we can make it work.
Overview
Necessary changes to OBS are:
You can find these changes in this branch: https://github.com/w23/obs-studio/tree/linux-libdrm-grab
A setuid/cap_sys_admin+ep utility is also needed to send the dma-buf fd.
It can be found in a separate repository: https://github.com/w23/drmtoy/tree/wip-drm-send-recv
How to
Now, as I got it to work only yesterday, this is highly experimental.
It is painful to set up at the time of writing.
Please test and enjoy!
Notes and questions
There are several things that I want to point out and discuss with experienced OBS developers.
It does solve most of my performance woes, and you can see this screen capture method in action here: https://youtu.be/L2Y7_vBfWm8 or here https://www.twitch.tv/videos/389112056
Notice how even for heavy shaders it still maintains real-time fps. Vanilla OBS with XSHM would struggle capturing it and would barely keep ~10 fps, even though the shader itself can be as high as 40-60fps.
Background
There's this DRM infrastructure that is used to talk to GPUs in a vendor-agnostic way on Linux. Among other things it incorporates:
- Kernel modesetting (KMS), a technology to enumerate and control video outputs, their video modes and framebuffers
- DMA-BUF objects, basically handles to GPU-side memory, e.g. framebuffers and textures. Handles to these are file descriptors.
Among these extensions we can find:
EGL_EXT_image_dma_buf_import, it allows creating EGLImage objects bound to existing DMA-BUF object (using its fd)GL_OES_EGL_image, it allows binding GL textures to EGLImage objects. Note that while it is technically an OpenGL ES extenstion, it is exposed in Mesa implementation of desktop OpenGL, and works there just fine on opensource amdgpu and intel drivers.
I bet you can see that all of these fit together perfectly. Moreover, this can be used to capture not only X11, but everything else that is backed by KMS, including Wayland and even bare terminals!
One caveat though is that getting framebuffer DMA-BUF fd requires
CAP_SYS_ADMIN, and running OBS with what basically amounts to root privileges is a YOLO practice.However, you can remember such things as UNIX sockets, sendmsg() functions and SCM_RIGHTS flags. These can be used to transfer open file descriptors between processes. This also works on DMA-BUF fds, and so we can construct a small binary with `
setcap cap_sys_admin+ep` with only purpose to get the framebuffer fd and transfer it to whoever.Now we can make it work.
Overview
Necessary changes to OBS are:
- Use EGL instead of GLX to initialize OpenGL context. This is a change to
libobs-opengllibrary. - Create a new source plugin that creates a GL texture from EGLImage created from DMA-BUF fd read from UNIX socket.
You can find these changes in this branch: https://github.com/w23/obs-studio/tree/linux-libdrm-grab
A setuid/cap_sys_admin+ep utility is also needed to send the dma-buf fd.
It can be found in a separate repository: https://github.com/w23/drmtoy/tree/wip-drm-send-recv
How to
Now, as I got it to work only yesterday, this is highly experimental.
It is painful to set up at the time of writing.
- It is assumed that you have all the necessary development libraries and tools installed on your system
- Get drmtoy enum and drmsend
Code:
git clone https://github.com/w23/drmtoy.git cd drmtoy git checkout wip-drm-send-recv make enum drmsend - Find the right framebuffer id for your screen
It will output lots of lines. You're interested in the last few lines that will looks something like these:Code:./build/rel/enum
Code:count_fbs = 2 0: 0x56 width=3200 height=1800 pitch=12800 bpp=32 depth=24 handle=0 1: 0x55 width=256 height=256 pitch=1024 bpp=32 depth=32 handle=0
Here you can see that there's a framebuffer 0x56 sized 3200x1800. This looks like the main screen.
0x55 is 256x256 and doesn't look like anything to me
- (a) Run drmsend with elevated privileges. Replace 0x56 with your framebuffer id
Code:sudo ./build/rel/drmsend 0x56 drmsend.sock & # chown socket so that your user can access it USER=$(whoami) sudo chown $USER drmsend.sock
(b) Alternatively you can set the right caps on drmsend and run it under a regular user. Note that you'd need your fs to be mounted without nosuid flag.
Code:sudo setcap cap_sys_admin+ep ./build/rel/drmsend sudo chown root ./build/rel/drmsend ./build/rel/drmsend 0x56 drmsend.sock & - Get patched OBS
Code:git clone https://github.com/w23/obs-studio.git cd obs-studio git checkout linux-libdrm-grab - Build patched OBS
Code:mkdir build cd build cmake .. -DUNIX_STRUCTURE=0 -DUSE_EGL=1 -GNinja ninja - Run it.
Note that-DUSE_EGL=1forcefully replaces GLX with EGL, and none of GLX-dependent things are patched to support EGL. E.g. XSHM and Xcomp screen capture modules won't work, and the entire linux-capture plugin won't even load due to missing symbols.
Code:cd rundir/RelWithDebInfo/bin/64bit/
-p is for portable mode, for it to not mess with your existing obs configuration
If you're feeling adventurous (like I am), back up your obs config and run it w/o -p flag
Code:./obs -p - Now you can add "DMA-BUF source" to your scene as you would with any other regular source.
In configuration dialog you need to specify drmsend socket:- Click "Browse" and navigate to the directory with
drmsend.sock, e.g. where you checked out drmtoy and ran build/rel/drmsend - QFileDialog will be unhelpful enough to not show UNIX sockets, so you'd need to type your filename manually, e.g.
drmsend.sockand press enter. - Now you should have a preview of your framebuffer screen. Note that there's no cursor capture yet.
- VYGODA
- Click "Browse" and navigate to the directory with
Please test and enjoy!
Notes and questions
There are several things that I want to point out and discuss with experienced OBS developers.
- Current experimental implementation makes EGL vs GLX a compile-time choice using -DUSE_EGL=1 cmake argument. It is not that hard to make a separate libobs-opengl-egl.so plugin and make choosing between it and legacy GLX-based libobs-opengl.so a runtime choice based on user preference, like DX vs GL choice on Windows.
XSHM, Xcomp and DMABUF sources will need to be able to detect GLX vs EGL at runtime.
Is there a better way than `obs_get_video_info()` and then checking `graphics_module` name?
There's also an issue of one of GLX or EGL not being available (e.g. on Wayland or older/weird X respectively) on some system.
I haven't looked at how libglad works, but it will likely require splitting libglad into libglad-gl, libglag-glx and libglad-egl. Also, missing symbols will need to be handled gracefully by linux-capture plugin.
- linux-dmabuf plugin is a temporary thing. I feel it needs to be integrated into linux-capture w/ added xcursor support. But see note above about dynamic decision of EGL/GLX and symbols.
- dmabuf_source module requires EGLDisplay handle, which lives deep inside libobs-opengl/EGL plugin. Is there a recommended way to get handles internal to some graphics impl? I couldn't find one.
Currently dmabuf_source includes graphics-internal.h, declares struct gl_platform itself (copied from gl-x11-egl.c), calls gs_get_context() and chases some pointers (see https://github.com/w23/obs-studio/b...c3cd0f0ba7/plugins/linux-dmabuf/dmabuf.c#L145). This is obviously not sustainable.
- Mode changes aren't handled. A the moment I have no idea how to do that, and how current implementation would behave in such case.
- I believe that sampling this DMA-BUF-backed texture will read framebuffer memory directly. No synchronization is implemented. This may or may not be a problem.
- Obviously this scheme with manually running enum and drmsend (from another repo!) is not very user-friendly. However, making it user-friendly in general case is hard.
One feasibly-looking approach is:- integrate drmsend into obs repo
- make drmsend perform framebuffer discovery
- dmabuf_source would have picker for framebuffers in its config dialog
- dmabuf_source would spawn drmsend with right arguments itself
Later we could make drmsend even smarter:- make it not framebuffer-centric, but CRTC (monitor)-centric.
- make it listen on mode change events
- make dmabuf_source listen on drmsend events
This would make it almost work for mode changes, but not quite. Xorg creates one huge framebuffer for multiple monitors, so mode changes would affect monitor positions and cropping coords. I don't think these rules are generalizable with OBS transformations.
- Privileged drmsend itself is too public morozov:
- CAP_SYS_ADMIN is too much, we need to patch kernel with something like CAP_DRM_CAPTURE as a more fine-grained capability
- leave control over which users in the system can capture screens into distro packagers hands (they might e.g. add video-capture group or whatever).
Last edited:
