indolering
New Member
There have been a few posts requesting a Zoom-esque background blur/virtual green screen, which uses some form of computer vision to segment the speaker from the background. Until recently, existing solutions based on Deeplab and BodyPix were very slow, ~10 FPS. However, Google released a model specifically designed for background segmentation that works well, even on mobile clients. Florian's DeepBackSub is now using the Google Meet model, so example code exists.
I thought about making a suggestion on the ideas.obsproject.com site, but I wanted to flesh-out an idea for a more generic filter that can be used as a garbage matte.
IMHO, "smart" background removal sucks. It's okay for blurring the background during a video conference, but I couldn't even get commercial solutions like XSplit's VCam or ChromaCam.me to work well.
However, the greenscreen I work with is not perfect and the existing filter struggles to compensate for things like wrinkles and shadows. Lowering the sensitivity threshold (or doing much of anything beyond the default settings) results in the filter becoming overly aggressive. However, the wrinkles (etc) are generally at the edge of the screen and away from the speaker.
The ideal solution would be to use the Google Meet model to segment the image and apply an aggressive chromakey on the background, the default chromakey filter in a halo around the foreground, and a conservative chromakey filter over the foreground.
Thoughts?
I thought about making a suggestion on the ideas.obsproject.com site, but I wanted to flesh-out an idea for a more generic filter that can be used as a garbage matte.
IMHO, "smart" background removal sucks. It's okay for blurring the background during a video conference, but I couldn't even get commercial solutions like XSplit's VCam or ChromaCam.me to work well.
However, the greenscreen I work with is not perfect and the existing filter struggles to compensate for things like wrinkles and shadows. Lowering the sensitivity threshold (or doing much of anything beyond the default settings) results in the filter becoming overly aggressive. However, the wrinkles (etc) are generally at the edge of the screen and away from the speaker.
The ideal solution would be to use the Google Meet model to segment the image and apply an aggressive chromakey on the background, the default chromakey filter in a halo around the foreground, and a conservative chromakey filter over the foreground.
Thoughts?
