PTZ control by Object Tracking

TheNukem

New Member
Hi all.

I've been a lurker on these forums for a while, but recently I've not been able to sit by and watch.


So this is a call out to someone who has skills far beyond mine....and I can tell you that having read a LOT of posts on these kind of forums, that this is a plugin that people are crying out for.

It'll save $1,000s of dollars on these crappy PTZs that claim to have AI tracking. OBS can do it all.

There are 2 awesome plugins that basically need linking, but I'm not the one to do it. I'm techie AF, but not a coder.

https://github.com/glikely/obs-ptz/
AND
https://github.com/locaal-ai/obs-detect
This is how I see it flowing:
  1. OBS captures video → Object Detection Plugin processes frames.
  2. Detection script calculates object position → sends PTZ adjustments.
  3. PTZ Plugin translates this into VISCA commands → Camera moves.

a few challenges:
  • Latency: PTZ movement may lag behind fast-moving objects.
  • Smoothing: I'll need to implement dead zones (only move if object is >X pixels off-center).
  • Calibration: Fine-tune pan/tilt speeds for smooth tracking.

Assuming that 'detect' already calcs the x, y positioning, then that data needs to feed it into PTZ control with an instruction set along the lines of 'if object is 'z' number of pixels off a specific point, then move camera until object is within tolerance.

Like I say, I'm no coder, but I am an award winning film maker as well as doing lots of multi-cam (PTZs) streaming of live events, performances and conferences. If I can be of any help to you in the development process, please let me know.
 
I’m all for it! obs-ptz already exposes a basic api for control by another plugin, and it is used by @norihiro ‘s Face Tracker plugin . Other plugins could use it too.

The API is a little basic though. I’m thinking about creating a common API that allows other plugins (e.g., @DistroAV ) to expose PTZ controls which can be driven by obs-ptz or other plugins. Object tracking would fit right with that.
 
Second this!
i need this for my local raceclub, i could setup cameras to auto follow cars and then return to preset 1 if none found for 10 seconds or something like that. That would give me more time to focus on commentating!
Please someone make this happen. i will raise money for you if needed!
Detect plugin worked amazing on cars but no PTZ control.
 
SO,

As much as I've loved this idea as it is so far, I've run into another issue.

I have 'proper' PTZ cameras. Panasonic AW HE130s....4 of them and guess what (and I've known this forever)....they run on their own protocol. Not Onvif / Visca / Pelco.

This hasn't been much of a problem for me as I use Companion (BitFocus) to control and preset / recall etc and a touch screen windows machine to do all the quick 'touch (click) to centre on the cursor' kinda things. And that's ALL good...

BUT this isn't going to work so easily with OBS PTZ Control, because OBS PTZ Control....doesn't control /my/ PTZs. :-(
 
How far with this? Has anyone been able to achieve it yet ? This plugin is exactly what I need
I literately made progress on this today. I published a new draft api that is implemented in PTZ Controls and an object tracker can use to control a camera. It can be found here:


That’s only the camera API side though. It still needs a controller to use it.
 
SO,

As much as I've loved this idea as it is so far, I've run into another issue.

I have 'proper' PTZ cameras. Panasonic AW HE130s....4 of them and guess what (and I've known this forever)....they run on their own protocol. Not Onvif / Visca / Pelco.

This hasn't been much of a problem for me as I use Companion (BitFocus) to control and preset / recall etc and a touch screen windows machine to do all the quick 'touch (click) to centre on the cursor' kinda things. And that's ALL good...

BUT this isn't going to work so easily with OBS PTZ Control, because OBS PTZ Control....doesn't control /my/ PTZs. :-(
What does “Proper” mean in terms of control protocol? Surely they talk some form of standard protocol if they integrate with other control systems. If you know the protocol it is possible to add it to the PTZ Controls plugin.
 
Back
Top