Detect - Object Detection, Tracking built-in OBS

Detect - Object Detection, Tracking built-in OBS v0.0.3

Go to download
Overview Updates (2) Reviews (4) History Discussion
W

Want To Learn

Member
Want To Learn said:
Thanks.
Click to expand...
Hi Roy, Thank You for the update. Your prior versions wouldn't allow me to do much, but it seems that all the effects appear to be working with a quick look on my Mac!!! Thank You so much! Actually the zoom feature is working so well that it is making me sea sick. I meant that as a compliment. LOL.
 
Last edited:
T

TheRealFr0g

New Member
In the roadmap you mention adding other real time models. I'd like to vote for rabbit to be added in the next wave of additions please. Was hoping "Cat" would be close enough but apparently it's better at identifying animals than I thought it would be haha. I figure rabbit (not in the list) is likely going to be more common than for example elephant (in the list already).

Plugin looks amazing though, here's hoping to get some bunny streams <3 thanks!
 
I

IHideMyName

New Member
Hi,

Can I use this plugin to detect and blur people face in a IRL outdoor stream? If yes, I was looking this for years.

Would this make usage of my RT cores for it?
Would be possible to only blur people face if they are too close or too far (setting minimal size or max size for bluring)?
 
royshilkrot

royshilkrot

Member
TheRealFr0g said:
In the roadmap you mention adding other real time models. I'd like to vote for rabbit to be added in the next wave of additions please. Was hoping "Cat" would be close enough but apparently it's better at identifying animals than I thought it would be haha. I figure rabbit (not in the list) is likely going to be more common than for example elephant (in the list already).

Plugin looks amazing though, here's hoping to get some bunny streams <3 thanks!
Click to expand...
@TheRealFr0g if you have a dataset with annotated bunnies you can build a model for bunny detection
you may also want to explore https://public.roboflow.com/object-detection to see if such a dataset exists where bunnies are annotated - that is simple to use for model training for which I have a guide https://github.com/occ-ai/obs-detect/blob/master/docs/train_model.md
 
royshilkrot

royshilkrot

Member
IHideMyName said:
Hi,

Can I use this plugin to detect and blur people face in a IRL outdoor stream? If yes, I was looking this for years.

Would this make usage of my RT cores for it?
Would be possible to only blur people face if they are too close or too far (setting minimal size or max size for bluring)?
Click to expand...
@IHideMyName there's a "person" category which would include a face if it sees it in the image, you can use that. I'm pretty sure it'll work well on IRL vid
This would use your GPU on Windows. If you have a GPU it should automatically be used
I don't have any size-based filtering yet.. althugh this could be implemented fairly easily
 
khaver

khaver

Member
I've been testing your plugin to see if it would be usable to track and follow a person on a stage (church sermon). After I get it set to work as I want, and then exit OBS and restart it, the tracking wont start until I uncheck and recheck the Tracking (Zoom, Follow) Options box. This also causes the Detect Tracking filter in the list to disappear and then reappear at the end of the filter list messing up the framing my PAN ZOOM ROTATE filter has been set to.
 
royshilkrot

royshilkrot

Member
khaver said:
I've been testing your plugin to see if it would be usable to track and follow a person on a stage (church sermon). After I get it set to work as I want, and then exit OBS and restart it, the tracking wont start until I uncheck and recheck the Tracking (Zoom, Follow) Options box. This also causes the Detect Tracking filter in the list to disappear and then reappear at the end of the filter list messing up the framing my PAN ZOOM ROTATE filter has been set to.
Click to expand...
i opened an issue for this problem https://github.com/occ-ai/obs-detect/issues/12
will get it fixed
 
I

IHideMyName

New Member
royshilkrot said:
@IHideMyName there's a "person" category which would include a face if it sees it in the image, you can use that. I'm pretty sure it'll work well on IRL vid
This would use your GPU on Windows. If you have a GPU it should automatically be used
I don't have any size-based filtering yet.. althugh this could be implemented fairly easily
Click to expand...
Thanks for the answer.

Would be nice if you can add a "face" category, so I dont need to mask the whole person.

I will try to test it this weekend. I would love if this can detect and mask multiple faces at the scene without too many misses.

I asked about the sizing because when I was searching it using the gstreamer pipelines from nvidia, I saw that you could add some filters like, only mark as "detected" if the size (retangular shape around the face) is bigger or smaller than %, or total size for example. Maybe this is too specific for this plugin, but would be nice also to bypass missdetections on common scenarios.
 
T

TheRealFr0g

New Member
royshilkrot said:
@TheRealFr0g if you have a dataset with annotated bunnies you can build a model for bunny detection
you may also want to explore https://public.roboflow.com/object-detection to see if such a dataset exists where bunnies are annotated - that is simple to use for model training for which I have a guide https://github.com/occ-ai/obs-detect/blob/master/docs/train_model.md
Click to expand...
Not sure I’m smart enough for all this but I do appreciate the links. I’ll poke at it and see how it goes. Thanks
 
I

IHideMyName

New Member
I tried to create a dataset and train it to detect only faces but unsuccessfully. I really think if the plug-in get able to do that it would be a game change. Please someone with expertise help us achieve that
 
khaver

khaver

Member
@royshilkrot, I've been looking at different premade onnx format models on the internet to try with your plugin, but none of them come with a json configuration file. Do you know if there's some way to generate the required json file from a model?

Thanks.
 
royshilkrot

royshilkrot

Member
khaver said:
@royshilkrot, I've been looking at different premade onnx format models on the internet to try with your plugin, but none of them come with a json configuration file. Do you know if there's some way to generate the required json file from a model?

Thanks.
Click to expand...
unfortunately any random model won't work because so far the plugin supports only EdgeYOLO models https://github.com/LSH9832/edgeyolo
you could train edgeYOLO to detect anything you have data for, but thats not an easy process, although i've documented it https://github.com/occ-ai/obs-detect/blob/master/docs/train_model.md

i'm working to support more standard models types and that will open up some possibilities
 
I

IHideMyName

New Member
khaver said:
@royshilkrot, I've been looking at different premade onnx format models on the internet to try with your plugin, but none of them come with a json configuration file. Do you know if there's some way to generate the required json file from a model?

Thanks.
Click to expand...

royshilkrot said:
unfortunately any random model won't work because so far the plugin supports only EdgeYOLO models https://github.com/LSH9832/edgeyolo
you could train edgeYOLO to detect anything you have data for, but thats not an easy process, although i've documented it https://github.com/occ-ai/obs-detect/blob/master/docs/train_model.md

i'm working to support more standard models types and that will open up some possibilities
Click to expand...
Maybe we are trying the same thing,

I am trying to use these ONNX models:
- https://github.com/FaceONNX/FaceONNX.Models
- https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB/tree/master/models/onnx

I tried to follow the guide and create the JSON with the following structure, but it did not work:
{ "names": ["face"] }
Click to expand...

The last one specially, can detect up to A LOT of faces on the same image, which is very good for IRL streams. They provide the images used for training.
 
royshilkrot

royshilkrot

Member
IHideMyName said:
Maybe we are trying the same thing,

I am trying to use these ONNX models:
- https://github.com/FaceONNX/FaceONNX.Models
- https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB/tree/master/models/onnx

I tried to follow the guide and create the JSON with the following structure, but it did not work:


The last one specially, can detect up to A LOT of faces on the same image, which is very good for IRL streams. They provide the images used for training.
Click to expand...
Let me see how simple it would be to support these models. I see a lot of folks asking for face detection. I will make it work for the next version of the plugin
 
royshilkrot

royshilkrot

Member
royshilkrot updated Detect - Object Detection, Tracking built-in OBS with a new update entry:

v0.0.3 - Many new features! Face detection, Pixelate, File save, SORT algo, more...

Big Release! With so many new features:
  • Face detection algorithm! Based on the very fast YuNet model
  • SORT algorithm for smooth tracking and consistent ID
  • Pixelate "blur" effect
  • Filter by minimal size
  • Track/Zoom the biggest or "oldest" object
  • Save detections to file real-time (combine with Streamer.bot!)
Enjoy!

Download:
  • ...
Click to expand...

Read the rest of this update entry...
 
E

EB123

Member
i havent installed this plugin yet but it looks good... im wondering though, what happens if its setup to track people and it sees 3 or 4 people in the video, how does it prioritize which one to follow ? and is there a way to ensure it doesn't switch back and forth between different subjects too quickly if it changes priority ? also have you considered adding the ability to control the movement of PTZ cameras ? that would be very helpful..
 
royshilkrot

royshilkrot

Member
EB123 said:
i havent installed this plugin yet but it looks good... im wondering though, what happens if its setup to track people and it sees 3 or 4 people in the video, how does it prioritize which one to follow ? and is there a way to ensure it doesn't switch back and forth between different subjects too quickly if it changes priority ? also have you considered adding the ability to control the movement of PTZ cameras ? that would be very helpful..
Click to expand...
You have a few options for tracking: First, Oldest, Biggest or "All" which keeps everything in view

Do you have an example of PTZ camera you want to connect?
I haven't explored it but I think it should be simple given they provide an API.
Right now you can save the detection to a file and pick it up in external programs
 
You must log in or register to reply here.
Top