Hi, this project is in early testing; so it's not as user-friendly as it will be - but I'm keen for any feedback!
Introduction
If you've looked into machine learning-based green screens, you probably already know of the mighty OBS background removal plugin, the OG Fake Background Webcam, or backscrub.
So, why make another one? Well, all the current ones soak up a lot of CPU/GPU and they add a lot of latency to the feed.
To solve these issues, linuxgreenscreen:
Demonstration
Here's an early version on stream using OBS's Chroma Key filter with a 1280x720 @ 30FPS (MJPG) webcam, and a AMD Ryzen 4900HS CPU (my lil' laptop).
Chroma Key settings
How it works
Wraps OpenCV (opencv-python) webcam capture, a compositor using Google's MediaPipe, and writes to a virtual video device created by v4l2loopback via the v4l2py module. In addition to the advantages above, linuxgreen screen:
Future work
Acknowledgements
The module and demo are inspired by and based on the work of Fufu Fang (and others) who wrote the Linux-Fake-Background-Webcam script.
Introduction
If you've looked into machine learning-based green screens, you probably already know of the mighty OBS background removal plugin, the OG Fake Background Webcam, or backscrub.
So, why make another one? Well, all the current ones soak up a lot of CPU/GPU and they add a lot of latency to the feed.
To solve these issues, linuxgreenscreen:
- Uses asynchronous segmentation
: reduces latency between capture and render of webcam. - Can scale segmentation rate: reduce CPU (or GPU) load depending on preferences.
Demonstration
Here's an early version on stream using OBS's Chroma Key filter with a 1280x720 @ 30FPS (MJPG) webcam, and a AMD Ryzen 4900HS CPU (my lil' laptop).
- Segmentation @ 10 FPS uses ≤70% of one core; so about 5% CPU overall.
- ~11ms latency between capture and display in OBS.
Chroma Key settings
How it works
Wraps OpenCV (opencv-python) webcam capture, a compositor using Google's MediaPipe, and writes to a virtual video device created by v4l2loopback via the v4l2py module. In addition to the advantages above, linuxgreen screen:
- Uses the latest image segmentation API and selfie segmentation model.
- 'Tis all python : easy to modify (hopefully).
Future work
- Add a user-interface
- Turn into a true OBS plugin using memory mapped files/buffer
Acknowledgements
The module and demo are inspired by and based on the work of Fufu Fang (and others) who wrote the Linux-Fake-Background-Webcam script.