Using Multiple PTZ Cameras and OBS - Video Delay

[CowetaAssembly]

Our church has 3 PTZ cameras set up for live-streaming through OBS. We initially had a center camera set up (Using PTZ20x) and then purchased and implemented 2 more cameras (PTZ30x). The problem is the 2 side cameras are both delayed. The left-side camera has about a 1-2 second delay and the right camera has anywhere from a 4-15 second delay. Rebooting the cameras is a temporary fix, but it still will get worse over time. The reboot does not even completely get rid of the issue, it just makes the delay about 1-2 seconds instead of 15. We have no issue with the center camera after the installation of the other two. Another temporary solution was setting the reconnect delay to the lowest setting for each camera, but that only worked for about a few minutes. We could set the delay to match the slowest camera, but when it is 15 seconds off, that is tough to work with as we need a quick response time. There is also a setup we have on a flat screen for the camera controller to use during the service to control the cameras with the iPad with PTZOptics where the cameras are responsive on screen. The delay for the flat screen and the Apple computer that controls OBS is not the same. The flat screen has a .5-1 second delay to the controls. We also tried swapping up the location of the two camera side cameras to see if it was a location issue, but not much changed and we still got more delay on the right camera when it was on the left.

https://obsproject.com/logs/iJnOFU6FPFy8zjrT

 

[pjokerst]

I was curious if there was any resolution to this delay issue? I was happy with OBS until I tried to add a PTZOptics camera and this delay started. I would rather find a solution versus learning a new software. Thanks in advance.

 

[nottooloud]

How are you receiving the signals? NDI, HDMI, USB, RTSP?

 

[pjokerst]

When adding a video capture device, I input the following: rasp://192.168.1.88/1 which then pulls up the PTZOptics SDI camera.

 

[nottooloud]

You'll have much less delay and more stability if you use either the SDI or HDMI outputs on the camera.

 

[thirdvalve]

Can multiple HDMI inputs be used at once? If so, how? What sort of interface is best to get them in via USB? Or... is it ONLY possible to get one (1) HDMI source visible at once in OBS? This seems to be a difficult question to get a straight forward answer on. Thanks for advice on this...

 

[nottooloud]

I run 3 HDMI sources via 2 CamLink 4k adapters and a VidBox.

 

[Lawrence_SoCal]

I take it you are using NDI. right?
If yes, are you on a gigabit (100T) switched network? ideally an isolated network (protect from rest of church traffic/network)
you aren't using WiFi, right?

 

[CowetaAssembly]

Thanks for all the advice! Our personal fix for the delay did not completely fix the issue but made it much more manageable. The major issue was we were double connected to the network, both over wireless and wired. I am not sure why but the connection favored the WiFi option between the two, even though that cause a lot of latency. There is still a bit of delay, but it is easily fixed by clicking on the source in OBS and "resetting" by clicking on the properties and closing the properties dialog box. The rest of this delay is clearly the fault of the communication between the cameras and OBS and no longer because of the connection.

 

[CowetaAssembly]

Current Update: One major fix that worked for our system was removing the static IPs of the cameras to dynamic IPs. Now this would seem counterintuitive in a lot of cases, but it increases the response times of the cameras. Baseline the cameras do not line up audio wise, but it is an easy fix with strategic audio and video delay. Now for the real test of a full on stream!

 

[Lawrence_SoCal]

Current Update: One major fix that worked for our system was removing the static IPs of the cameras to dynamic IPs. Now this would seem counterintuitive in a lot of cases, but it increases the response times of the cameras.

Uh, not only counter-intuitive, but weird
How a device gets its IP address should NOT impact the speed at it future communication over that interface.

if real, I'd think, maybe firmware bug... or
is this a case of which interface (wired vs WiFi) in use? ie separate (or VLANed off) segments?
Static is a better idea though I am using DHCP for camera on isolated network with a DHCP reservation.
Without knowing more, I'd recommend disabling WiFi on the cameras for a number of reasons (security, WiFi performance (due to beaconing), consistency, etc).

 

[wilderf353]

We had a similar problems at a church I volunteer at. They have 3 PTZOptics camera all using NDI.

To fix the problem, we created a simple video of someone clapping 1x, then 2x, then 3x and played it on laptop on the altar in a loop (or use this video). We then created a scene in OBS with all 3 cameras and aimed them at the laptop. Using the new scene, we identified the 2 fastest cameras and added a video delay filter on each camera to match the slowest camera.

We then added a sync offset on the audio input to match up to the cameras (in the Advanced Audio Properties dialog).

 

[Lawrence_SoCal]

Across a typical LAN, properly setup and running, should have a latency in the 1ms range. And length of cable will make a small, but negligible difference in latency for this use case (human brain not really gonna notice the difference in that nanosecond range). Now, can different generations of devices have different processing speeds? sure. So with that 3 PTZOptics environment, I'm hoping the those were different model/generations. If not, I'd call and open a support case, as all 3 cameras should all process at the same speed on a LAN (or LAN needs fixing)

Your sync method works though I'd recommend getting a good pre-made downloadable audio/video test sync video and using it (vs hand made clapping) as you can get even a better granularity on time sync. I believe the one I'm using is the one from Ballast Media https://www.ballastmedia.com/audio-video-sync-test.html [link to both downloadable video and YouTube video instructions https://www.youtube.com/watch?v=UCmr0KwbXxA].
Note that laptops or tablets are both viable as platforms to play test video on

 

[CowetaAssembly]

Uh, not only counter-intuitive, but weird
How a device gets its IP address should NOT impact the speed at it future communication over that interface.

if real, I'd think, maybe firmware bug... or
is this a case of which interface (wired vs WiFi) in use? ie separate (or VLANed off) segments?
Static is a better idea though I am using DHCP for camera on isolated network with a DHCP reservation.
Without knowing more, I'd recommend disabling WiFi on the cameras for a number of reasons (security, WiFi performance (due to beaconing), consistency, etc).

The problem with disconnecting it from the wifi would be that we would not be able to do our iPad controls, I have considered switching to the board for controls but it’s not necessarily cheap.
I am aware that the logic for a static IP would make sense but there has been obvious improvement since switching to a Dynamic IP.
Also the computer in question that runs OBS is physically connected to the cameras, albeit through the network. Definitely puts us in an interesting situation.

 

[CowetaAssembly]

We had a similar problems at a church I volunteer at. They have 3 PTZOptics camera all using NDI.

To fix the problem, we created a simple video of someone clapping 1x, then 2x, then 3x and played it on laptop on the altar in a loop (or use this video). We then created a scene in OBS with all 3 cameras and aimed them at the laptop. Using the new scene, we identified the 2 fastest cameras and added a video delay filter on each camera to match the slowest camera.

We then added a sync offset on the audio input to match up to the cameras (in the Advanced Audio Properties dialog).

Sadly we have tried that, but it is not such a simple fix, the fluctuation of delay is a constant on-going battle.

 

[Lawrence_SoCal]

The problem with disconnecting it from the wifi would be that we would not be able to do our iPad controls

No, definitely shouldn't be the case unless network is mis-configured
You want the cameras talking to OBS over wired connections. And streaming PC uploading to Internet via a wired connection. Adding an iPad to the network via WiFi to talk to the wired devices is simple. However, in typing this, it occurs me that doing so requires basic network understanding of your Access Point/router default settings and possibly (probably?) understanding the distinction in (especially, ) consumer router/APs in regards to Guest vs Internal networks (and Guest WiFi client isolation) ... if that all sounds like a foreign language, and you have no idea what I'm talking about, then I'd recommend hiring someone to help out on the network side if there isn't a church member with this knowledge).

For me it was easier to leave the cameras on DHCP, however, I did set up DHCP Reservations. Otherwise, no end of complications

And latency fluctuations of streaming multiple video cameras over WiFI is basically to be expected.. one of many reasons to NOT use WiFi (without thorough understanding of the local environment and how WiFi works, with appropriate management and monitoring tools in place to keep it running smoothly)

 

[CowetaAssembly]

No, definitely shouldn't be the case unless network is mis-configured
You want the cameras talking to OBS over wired connections. And streaming PC uploading to Internet via a wired connection. Adding an iPad to the network via WiFi to talk to the wired devices is simple. However, in typing this, it occurs me that doing so requires basic network understanding of your Access Point/router default settings and possibly (probably?) understanding the distinction in (especially, ) consumer router/APs in regards to Guest vs Internal networks (and Guest WiFi client isolation) ... if that all sounds like a foreign language, and you have no idea what I'm talking about, then I'd recommend hiring someone to help out on the network side if there isn't a church member with this knowledge).

For me it was easier to leave the cameras on DHCP, however, I did set up DHCP Reservations. Otherwise, no end of complications

And latency fluctuations of streaming multiple video cameras over WiFI is basically to be expected.. one of many reasons to NOT use WiFi (without thorough understanding of the local environment and how WiFi works, with appropriate management and monitoring tools in place to keep it running smoothly)

Sorry, I must not have been clear. The connection between OBS and the cameras is a wired connection. The only non wired connection is with the Ipad control. I have been in talk with the network administrator for our Church, he is also lost in how the connection is fluctuating over a wired fiber connection. I will say I agree with you on it having its own subnet, but I am still trying to convince my network to go through with that.

We have done multiple trace routes with immediate feedback, but the cameras do not follow this same logic.

My main assumption is that because the connection between the two, although not necessarily needing any kind of internet connection is still affected by network traffic that shows up especially on Sundays, because the network the camera and computer are on is on the same one that anyone with a phone can connect to the of the access points on the same network.

What is strange to me however is that we observe the cameras from two separate wired connections and the other connection does not have extreme delay fluctuations like the connection to OBS does. It runs through a blackmagic box and on to a TV with barely any delay. The quickest fix for this would be connecting the camera feed through the blackmagic box, but then you would lose the control of camera switching on the computer and physically the box is not easy to get to for controls.

Please let me know what you think!

 

[wilderf353]

Sadly we have tried that, but it is not such a simple fix, the fluctuation of delay is a constant on-going battle.

Some cameras can use a NTP server to synchronize video streams. Can your cameras?

See https://help.ptzoptics.com/support/...ing-multiple-ptzoptics-ip-video-streams-audio

 

[Lawrence_SoCal]

Sorry, I must not have been clear. The connection between OBS and the cameras is a wired connection. The only non wired connection is with the Ipad control. I have been in talk with the network administrator for our Church, he is also lost in how the connection is fluctuating over a wired fiber connection. I will say I agree with you on it having its own subnet, but I am still trying to convince my network to go through with that.

Fiber? that seems odd. A standard Ethernet (copper CAT5/6/7,etc) cable is good for 90 meters + 5m patch on either end (hence people often referring to cable lengths up to 100m/300ft). As fiber is more expensive than copper wiring (and fiber needs transceivers on either end), it is rare to see fiber in shorter runs, unless interference or other unique circumstances. If fiber truly in place, then you may have a transceiver issue, or routing, or ?? issue

I'm suspecting, for remote assistance, that short forums posts won't cut it. I'm happy to help out (within reason). But I need some more explicit details. like cable types, lengths, exact routing, etc. For my environment, I did a relatively through Visio drawing showing our layout, cabling, devices, etc. Which really helps when discussing...

In general
- having camera and streaming PC and guest WiFi on same network is a TERRIBLE practice.
- And if you are sharing a common Internet connection (as would normally be the case as few HoW choose to pay for multiple ISP connections), then you need Quality of Service (QoS) or similar to make sure streaming traffic is NEVER running into contention with other traffic. All it takes is one kid (or adult) not knowing better to hammer your network. And without real-time tools in place to monitor and counteract such activity, well ... lost cause in my opinion.... or at least frustration I'm not willing to subject myself to

 

[Lawrence_SoCal]

Wait... in re-reading this... @CowetaAssembly - what EXACTLY is the connection topology of the 3 cameras to computer? I'm not seeing your post whether your church using NDI, SDI, HDMI, USB, or ??? for cameras

Are you mixing connection types? if so, latency variance would be expected.
And if you are NOT using NDI, then camera video traffic most likely not being bridged with adapters across Ethernet data network. The only ethernet/data traffic to each camera would be PTZ control traffic. In which case, QoS would primarily apply to livestream upload traffic