Background Removal / Virtual Green-screen & Low-Light Enhance v1.1.13

[dquille]

Agreed, it does appear to be working for all of the video format/color space selections now, but also agreed that resolution is being effected compared to beta 1. Maybe the effect is being handled at a lower resolution for performance (which is OK) but the output image was accidentally kept at that lower resolution as well instead of the resulting effect being applied back to the original?

 

[royshilkrot]

Hello again! Thanks for the latest update and easy to install Windows files.

The color conversion problem was solved but the performance issue remais a big issue here:

View attachment 70682

Also, the media file used in this test was a 1080p30 local recording, the effect changed its quality to maybe a fourth of its original resolution.

Is there something I'm missing here?

Thanks.

Ah yes. You're correct... I'm applying the mask on the downscaled version instead of on the full-scale version... I'll fix it soon!

 

[ASchneider]

Ah yes. You're correct... I'm applying the mask on the downscaled version instead of on the full-scale version... I'll fix it soon!

No worries, makes sense...

I was wondering if the amount of resources used in my machine to make it happen are correct (screenshot above)?

Thanks.

 

[royshilkrot]

Agreed, it does appear to be working for all of the video format/color space selections now, but also agreed that resolution is being effected compared to beta 1. Maybe the effect is being handled at a lower resolution for performance (which is OK) but the output image was accidentally kept at that lower resolution as well instead of the resulting effect being applied back to the original?

Yes that's exactly the case. I'll fix it, it's a small change... look out for v0.2.1

 

[royshilkrot]

royshilkrot updated Background Removal - Portrait Segmentation with a new update entry:

v0.2.1 - fixing scaling bug in v0.2

In this release

• Fixing the problem of down&up scaling the original image. Instead - apply the mask to the original image size, without reducing its resolution.

Read the rest of this update entry...

 

[ASchneider]

Hello @royshilkrot ! Thanks for the update

I'm still facing problems with high CPU usage:

Image is now at full resolution but showing some jagged edges.

Thank you!

 

[royshilkrot]

Hello @royshilkrot ! Thanks for the update

I'm still facing problems with high CPU usage:

View attachment 70713

Image is now at full resolution but showing some jagged edges.

Thank you!

i'm working on bringing in other, potentially lighter, segmentation models as well as using the GPU on Windows. this will take the load off the CPU

the jagged edges i can take care of in the release - please open an issue for it https://github.com/royshil/obs-backgroundremoval/issues

 

[ASchneider]

i'm working on bringing in other, potentially lighter, segmentation models as well as using the GPU on Windows. this will take the load off the CPU

the jagged edges i can take care of in the release - please open an issue for it https://github.com/royshil/obs-backgroundremoval/issues

Nice!

I was wondering if this could help: I was in talks with Steve, from OBS Ninja, suggesting a background remover and he actually implemented it :) This was the issue opened, I think there might be some good references to use from good AIs. I've even mentioned you there - https://github.com/steveseguin/obsninja/issues/750#issuecomment-825564010

He managed to achieve a very nice result with that AI model, really nice.

The main interest for me, to use your solution, is to remove a background for someone using OBS Ninja (web source) from an iPhone device, that cannot do it locally.

Thanks!

 

[KenhCuaDaoLeMinh]

Hello, could you make a tutorial video about this? I can't find the plugin after paste it in the OBS format.

 

[Olivier J.]

Hello,

On PC with Windows, no problem ...
but on my mac, i don't understand .. I select the plugin in filter and OBS is crashing ..
there is the report: https://pastebin.com/qzSQzkQZ

Thanks !

 

[ASchneider]

Hello, could you make a tutorial video about this? I can't find the plugin after paste it in the OBS format.

Hey there! Look for the above square, Video and Audio Effects, when open the filter options from a source.

 

[Olivier J.]

Hello, could you make a tutorial video about this? I can't find the plugin after paste it in the OBS format.

You must select a video source ...
and make a filter Background Removal !

 

[royshilkrot]

royshilkrot updated Background Removal - Portrait Segmentation with a new update entry:

v0.2.2 Beta - GPU support for Windows

In this release
- Better versioning of the plugin
- GPU support on Windows through DirectML (any available GPU)

Install:
- Windows (64bit): Unzip the `obs-backgroundremoval-win64.zip` file in the OBS directory (e.g. `C:\Program Files\obs-studio`)
- Mac OSX: Unzip `obs-backgroundremoval-macosx.zip` to `/Applications/OBS.app/Contents/`. e.g. `$ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/`

Read the rest of this update entry...

 

[ASchneider]

royshilkrot updated Background Removal - Portrait Segmentation with a new update entry:

v0.2.2 Beta - GPU support for Windows
Read the rest of this update entry...

Thanks for the update! Not sure if I've done something wrong here (checked the new GPU box), but it still uses 100% CPU time...

 

[royshilkrot]

Thanks for the update! Not sure if I've done something wrong here (checked the new GPU box), but it still uses 100% CPU time...

Does it use the GPU though?
you can see GPU utilization in the Task Manager

the GPU support is from DirectML, and maybe it's not compatible with all GPUs. on my Intel HD Graphics GPU - it's actually much slower than the CPU (6-7 FPS with %80 GPU utilization), but it does take load off the CPU...

 

[ASchneider]

Does it use the GPU though?
you can see GPU utilization in the Task Manager

the GPU support is from DirectML, and maybe it's not compatible with all GPUs. on my Intel HD Graphics GPU - it's actually much slower than the CPU (6-7 FPS with %80 GPU utilization), but it does take load off the CPU...

Not at all, I had MSI AB monitor opened and it not used my GPU at all, just CPU. I have a 2080ti.

 

[royshilkrot]

Not at all, I had MSI AB monitor opened and it not used my GPU at all, just CPU. I have a 2080ti.

I suspected NVidia GPUs don't work. OK - i'll keep investigating this... look out for another release.

 

[dquille]

My experience with the GPU switch. I have a desktop with a nvidia 1060, and it seems to be using the GPU in some way with it enabled because the cpu usage drops 10-15%, but the gpu usage doesn't appear to go up significantly. I have a laptop with an intel 620, and the cpu drops way down, but like you roy the 620 gpu maxes out in gpu usage and is only able to render like 2fps.

 

[royshilkrot]

royshilkrot updated Background Removal - Portrait Segmentation with a new update entry:

v0.2.3 Beta - Optimized model

In this release

• Optimized ONNX model with slightly better runtime/resources performance (5-10%)
• Fix to DirectML build (including the DLL)

Install:

• Windows (64bit): Unzip the obs-backgroundremoval-win64.zip file in the OBS directory (e.g. C:\Program Files\obs-studio)
• Mac OSX: Unzip obs-backgroundremoval-macosx.zip to /Applications/OBS.app/Contents/. e.g. $ unzip -o obs-backgroundremoval-macosx.zip -d /Applications/OBS.app/Contents/

Read the rest of this update entry...

 

[dquille]

With 0.2.3 on my less powerful system, the CPU usage itself hasn't dropped but my percantage of frames missed due to rendering lag seems to have gone down a bit, so that's a move in the right direction.