Best solution for headless OBS on Linux ?

[whitewater]

Hey there, I've been trying to find a solution to run OBS headless on a NVIDIA Jetson. The app runs really well when connected to the display but my application requires that I control OBS remotely and I have tried a combination of VNC and VirtualGL but the performance drop significantly. Has anyone found a solution that works well that they could recommend?

Thanks

 

[whitewater]

Not sure what the bottleneck is, my CPU and GPU utilisation seems pretty low but the app shows my FPS to be around 30 when I use the remote desktop session (TurboVNC + VirtualGL). I can easily get over 60 when connected directly to the device with a monitor.

Does the FPS counter shown in the app measure how fast the UI can be rendered on the display and will that same limit impact the FPS of recording / streaming ? Does seem that way

 

[khaver]

There are HDMI dongles that trick the computer into thinking a display is attached.

 

[whitewater]

Hey there, thanks for the reply.

Are you suggesting that I use this method in conjunction with a remote client that shares the main desktop / display ? I don't have a dongle but I tried running VINO with an attached monitor and it did increase the framerate to about 50 FPS but VINO seems to be a bit of a pig and the CPU utilisation is pretty high.

Would there be a better method ?

 

[AaronD]

Why are you running headless? There might be a valid reason, but this may also be an XY problem:

Home - The XY Problem

xyproblem.info

 

[whitewater]

Why are you running headless? There might be a valid reason, but this may also be an XY problem:

Home - The XY Problem

xyproblem.info

Fair enough, I am currently on a dev kit that has a display but the final system that I am targeting is headless and I would like to be able to control OBS from a remote system to configure it, create scenes and switch between them. I've done a few more experiments and I've been able to get OBS to run better by using x11vnc to mirror the main screen instead of usingTurbo VNC + VirtualGL. I believe that x11vnc asynchronously samples the frame buffer while VirtualGL intercept openGL calls, maybe blocking and slowing down OBS, I'll do a bit more research to see if there is a better way to confgure it.

Of course, if there are better ways to control OBS remotely than trying to use a remote desktop approach, I'd be happy to hear about it.

Thanks for the help

 

[AaronD]

Does OBS have to be on the remote platform itself? Could the camera(s) and any other sources be streamed individually to a base station? Could they all be recorded separately, and the recordings recovered later and edited then?

Or does it really need to be a finished live stream from the remote platform itself? If so, why?

---

If you can get OBS to run on the remote system, then you might look at this plugin:

Advanced Scene Switcher

This plugin will allow you to automate various tasks using "Macros". Macros consist of a list of conditions under which a list of actions will be performed. Examples and guides can be found in the wiki. Feel free to contribute! If you run...

obsproject.com

It can automate almost everything that OBS can do, and then some, in response to almost anything. For example:

The last two screenshots are all that a slightly outdated version can respond to and control. I need to update this week myself. :-/

 

[whitewater]

Hi @AaronD I appreciate the brainstorm. So this plugin is using the websocketapi to control OBS is that right ? Do you know if there is a way to start OBS without the GUI and have it listen only to websocket ?

That's definitely something to consider. I would still need to find a solution to initially configuring the scenes, sources etc but maybe that could also be achieved through the plugin or websocket API.

For this application, this appliance would be the only device that would be used to take sources, composite and stream. A PC could be use to control it but would not have access to the media sources

Again thanks for the interest and helpful brainstorming on how to solve my problem

 

[AaronD]

So this plugin is using the websocketapi to control OBS is that right ?

Sorta. Websocket messages need to be formatted to get through OBS's filters and reach the plugin. That's all that OBS cares about for that transaction. Then the plugin does what it does, and calls OBS's functions from itself. OBS has no idea that those two things are connected.

I've forgotten what the format is to get through OBS's filters, but if you have two instances of OBS at the same time, both of them running that plugin, communication from one instance of the plugin to the other "just works". So you might capture that and copy/paste into your controller.

Or you could ask here:

Advanced Scene Switcher

Warmuptill submitted a new resource: Automatic Scene Switching - Switches to specified scenes depending on which window is in focus A remake of the "Simple Scene Switcher"(https://obsproject.com/forum/resources/simple-scene-switcher.29/) for obs-studio. Installation 1. Extract the folder. 2...

obsproject.com

The author is very active on there and very accommodating. Most of what it does is because of user requests, some of them mine.

Do you know if there is a way to start OBS without the GUI and have it listen only to websocket ?

Don't think so. It's designed to be a TV production studio, and what's that without a user interface? What you're doing might find a way to work, but it's definitely not what it's designed for!

I would still need to find a solution to initially configuring the scenes, sources etc but maybe that could also be achieved through the plugin or websocket API.

obs --help in a terminal, to see all of the startup options. Adv. SS settings are stored in the Scene Collection.

If you're going to have some automation in Adv. SS, then I'd want to have as much as I can in there. Here's a macro that runs once on startup and never again:

"Running" is always true, and the "only on change" checkbox starts with a default of false. So it runs exactly once, immediately. Use the command-line option to make sure that the right settings are loaded in the first place, which includes the macros, then do the rest of your setup in here.

 

[AaronD]

NVIDIA Jetson

By the way, how well does that work in general? Resolution, encoder settings, resulting quality, system load, thermal performance, etc.

I have a project in the back of my mind, not started yet, that might use something like that. Record 3D and stream 2D from a pair of cameras on a robot. Barely any processing before the encoder - just side-by-side raw cameras for the 3D recording, and maybe a transparency gradient for the 2D stream - but it does need to keep up with the data rate and those two different encodings. Probably 1920x1080p60 from each camera, so that's the streaming 2D size and framerate. Recording 3D is the same but twice the width: 3840x1080p60.

Probably need two simultaneous instances of OBS, just for the different frame sizes. If access to the cameras is exclusive, as was true for almost all systems until recently for performance reasons, then the recording instance might do *all* of the compositing with the larger frame, then send the too-big 2D version through the virtual camera to the streaming instance, which only crops it. Anyway......

The only other device that might be even semi-reliably present would be a phone hotspot, or maybe a laptop that does the same function, but neither one is reliable enough to consider as anything more than a streaming passthrough and occasional maintenance. (it doesn't always stream, but it does always record) So it does need to do everything on the platform itself.

It'll have a GUI, not headless, but the GUI is already taken by another app. It's perfectly okay though, for me to set up OBS graphically, and then switch to the other app while OBS just runs. No further changes required. It's more a record of what happened, than a live production.

 

[whitewater]

Thank you ! I am still testing but on a Xavier AGX so far so good. I was able to get hardware encoding / decoding through a mix of using the gstreamer plug in and code mofication to the ffmpeg / media decode. It's pretty hacky right now but I was able to bring up v4l2 sources and other network sources in 1080p60 with about 10% CPU usage and lots of GPU headroom. I am hoping to give it a go on Xavier and Orin NX next. I hope to take a deeper look the plugin you suggest and also possibly other method over the next few weeks. I am running an older version because my system has ubuntu 18 and QT5 but realized that it seems like you can build newer version with QT5 so I might give that a go as well

If other have more ideas keep them coming !

Cool product, I had not much experience with it but so far have been impressed

 

[AaronD]

I am still testing but on a Xavier AGX so far so good. I was able to get hardware encoding / decoding through a mix of using the gstreamer plug in and code mofication to the ffmpeg / media decode. It's pretty hacky right now but I was able to bring up v4l2 sources and other network sources in 1080p60 with about 10% CPU usage and lots of GPU headroom. I am hoping to give it a go on Xavier and Orin NX next.

Hmm... So the hardware capability is easily there, but the software support isn't quite yet?

I am running an older version because my system has ubuntu 18 and QT5 but realized that it seems like you can build newer version with QT5 so I might give that a go as well

I recently had to go digging for the latest old version of Adv. SS that would run on OBS 25 or something like that, because my attempt to upgrade that rig went sideways AND it rejected my backup! So I had to rebuild it all from scratch, with old versions of things that I knew worked, so I could run the next hard-scheduled stream. Then I tried a different upgrade path, and that worked. Back to current versions again!

That was upgrading from Lubuntu 20.04 LTS to Ubuntu Studio 22.04 LTS on the same x86/64 hardware.

Lubuntu – The official Lubuntu home

lubuntu.me

Ubuntu Studio – A free and open operating system for creative people.

ubuntustudio.org

• Lubuntu is mostly just a lightweight display manager and preinstalled apps.
• Ubuntu Studio has a low-latency kernel and a TON of media-related apps preinstalled, including an old version of OBS. Add the official PPA, and then the normal update process gets you the current version:
  sudo apt-add-repository ppa:obsproject/obs-studio
  THEN install plugins and build your rig. Updating from the preinstalled version to current will break everything anyway, so you really do need to update first.

When I was rebuilding the rig on Lubuntu 20.04, in early 2023, it was a PAIN to find the exact old version of Adv. SS! Pogo-sticking between the table of contents and each specific description, and there are a lot of old versions...

If you can stay current, DO! Or if you absolutely have to use old stuff, it's probably easier to ask here and just take the answer:

Advanced Scene Switcher

Warmuptill submitted a new resource: Automatic Scene Switching - Switches to specified scenes depending on which window is in focus A remake of the "Simple Scene Switcher"(https://obsproject.com/forum/resources/simple-scene-switcher.29/) for obs-studio. Installation 1. Extract the folder. 2...

obsproject.com

Cool product, I had not much experience with it but so far have been impressed

Yeah. It does a LOT if you know how to set it up. :-)

 

[aaronjward22]

Hey, I registered an account JUST to share this with you.

I put together a race car livestreaming platform based on a thin client running debian, OBS and 3 wyze cams, absolute rock bottom budget, all running inside the car connected to the 12V system.

I use KasmVNC to interface with the thin client through a browser and KasmVNC allows for hardware acceleration use in OBS vs traditional VNC servers/clients.

Good luck.

I can share more if you would like. LMK.

 

[AaronD]

I can share more if you would like.

I'd be interested. I still haven't started my project yet, and it looks like I still won't get to for a while, but it's still good to gather ideas, even if the available hardware at the time I finally do makes some of them moot.

I'm especially interested in how you got OBS to encode video on a thin client. Those are typically bare bones machines from what I understand. They are technically complete computers, but what little resources they have are intended to be more of a local cache and signal conditioner for the real system that runs on a server somewhere. Something like a low-end netbook in terms of processing power on their own.

Or are you using something other than what I'm envisioning?

 

[Lawrence_SoCal]

Aaron, I see you haven't received a reply... but FYI, years ago, with Skype, Cisco soft-phone, etc, thin clients started getting a lot less 'thin' as some local processing was required, so local video offload on some (upper end enterprise) thin clients was a thing over 10 years ago....