YorVeX
Member
"Robust" is the important part here, it means that audio and video stays in sync even if there is a short lag in one of the OBS instances. And "staying in sync" mostly (though not exclusively) means "don't produce the infamous 'adding X milliseconds of audio buffering' message", because that's when everything goes to hell (more on that later). Bonus points for when audio monitoring also keeps on working.
I know that in this forum to such questions people often get the answer to simply try to prevent the things that cause the lag and make sure their system is not overloaded, but even with a beefy PC...in a real world scenario with a live stream that is several hours long this is simply not realistic. The worst thing is that the offenders (at least for me) most of the time even are sources that OBS comes with:
For my test setup everything starts with OBS 1, which has a VLC source that plays this video in a loop and an Image source that is set to unload the image when not shown and load a quite large image (40 MB, resolution a bit above 5000x5000) - this is used to simulate lag. All tests are done within the same PC so network related issues are out of the picture.
All OBS instances have both NDI and Teleport installed, which is the two options that seem suitable for the task. OBS 1 is outputting both a Teleport and NDI stream at the same time (I tried with only one of them each but that didn't change anything about the problem so I left them both enabled all the time).
Setup with 2 instances -> Monitoring audio sync breaks but recording is good
Let's take the most simple setup at first:
Audio on both OBS instances is disabled, the receiving NDI and Teleport sources are set to output their audio streams to the monitoring device so that I can hear it and compare it to the OBS 2 preview window for testing. On OBS 2 only the NDI receiver source is active.
Obviously when starting everything is fine and in sync. Then I press the eye icon to load and unload my "Large Image" source several times. While doing that the audio and video feed is lagging, and eventually after a while the beloved "adding X milliseconds of audio buffering" message appears, saying the total audio buffer is 512 ms. This happens in both OBS instances simultaneously! From this point on audio and video is not in sync anymore in OBS 2, at least not audio over the monitoring device. Now when disabling the NDI receiver source and enabling the Teleport receiver source it doesn't get better.
Some interesting observations here:
Here is a video of such a test session. It also shows how a recording is done at the end, this is the recording that it produced.
Setup with 3 instances -> Monitoring audio sync breaks and recording also has A/V desync
I do use the audio monitoring feature so the first scenario was already bad enough. But it gets worse. Let's extend the setup to this:
OBS 2 is configured like before but also outputs a Teleport/NDI stream itself, which OBS 3 has receiver sources for.
Now when I produce lag on OBS 1 the same way I get the A/V desync when using monitoring audio just as before. But now when I record on OBS 3 the recorded video file also has A/V desync in it, albeit not as bad when compared to the monitoring audio ("only" about 200 ms). This is the recording of such a session, you can see it's in sync at first, then I produce the lag and when I stop (after I have seen the audio buffering message) it's no longer in sync.
To fix the desync both OBS 1 and OBS 2 need a restart, OBS 3 can be left running.
What now?
The actual setup I want to use is even more complicated than that: over network, even more OBS instances involved, two separate feeds with combined audio and video that are only combined at the last OBS instance of the chain, lots of scenes, sources and effects...
And it's not like this is not working at all. I have actually used such a setup for quite a long time and sometimes I had sessions with no A/V desync at all and sometimes I would have to reset everything multiple times within a stream of a few hours. That makes sense after my tests above, since even with the "load/unload large image" trigger causing the actual problem was very random. Sometimes it would happen after 10 seconds, sometimes even after 2 minutes all the lagging didn't cause any desync and/or audio buffering increase message.
Anyway, eventually I got annoyed and started to investigate a bit more, which is how I got to create those simplified scenarios, making the problem easier to debug and understand - but with the risk that there are more problems in my more complex setup that I don't even get here. In my full setup I have also seen A/V desyncs when no OBS log has shown any audio buffering message, but one step at a time, let's focus on the obvious parts first.
Also I was able to improve the situation already by using the Mute Audio filter linked above on various sources and replacing Media sources and Image sources that dynamically load images with Browser sources, because they don't produce a lag when loading in the first place (if they need more time for loading the image then it is just shown a bit later, the behavior I would have expected from the Image source too).
While this obviously hasn't helped with my scenario maybe this will, but who knows when it will be released.
Is there anything else I can do? While NDI and Teleport do basically the same thing, they internally work in quite different ways, yet suffer from the same problem. That makes me think the problem is more likely to come from OBS than from one of these plugins, but of course I can't know for sure, maybe they just both integrate into OBS in a bad way.
In my real setup I need both a networked transfer and a transfer between OBS instances within the same PC, I see no way to do all that with HDMI cables and capture cards. I thought about "streaming" from OBS to OBS using RTMP and a relay like nginx-rtmp but in my 3 instance scenario also the recording went desync so I would assume that the same would happen with the stream. It all comes down to OBS doing something weird when it is lagged, but again, I see no realistic way to avoid all potential lags that could occur.
At this point any ideas are welcome, even the most crazy ones...
I know that in this forum to such questions people often get the answer to simply try to prevent the things that cause the lag and make sure their system is not overloaded, but even with a beefy PC...in a real world scenario with a live stream that is several hours long this is simply not realistic. The worst thing is that the offenders (at least for me) most of the time even are sources that OBS comes with:
- the Media source configured to play an audio file is activated 7 times over the course of a few hours - when it's activated the 8th time it suddenly causes audio buffering increase (although DPC latency checker tools say everything is perfectly fine)
- an Image source used for dynamically loaded images or when loaded the first time can lag OBS and has a chance to cause this, especially on bigger images
- a Game Capture source capturing a game that is being switched to full-screen our out of it ("tabbing into the game") makes OBS lag and can cause sync issues
- sources that don't even have audio cause audio buffering (which is why something like this exists)
For my test setup everything starts with OBS 1, which has a VLC source that plays this video in a loop and an Image source that is set to unload the image when not shown and load a quite large image (40 MB, resolution a bit above 5000x5000) - this is used to simulate lag. All tests are done within the same PC so network related issues are out of the picture.
All OBS instances have both NDI and Teleport installed, which is the two options that seem suitable for the task. OBS 1 is outputting both a Teleport and NDI stream at the same time (I tried with only one of them each but that didn't change anything about the problem so I left them both enabled all the time).
Setup with 2 instances -> Monitoring audio sync breaks but recording is good
Let's take the most simple setup at first:
OBS 1 -> OBS 2Audio on both OBS instances is disabled, the receiving NDI and Teleport sources are set to output their audio streams to the monitoring device so that I can hear it and compare it to the OBS 2 preview window for testing. On OBS 2 only the NDI receiver source is active.
Obviously when starting everything is fine and in sync. Then I press the eye icon to load and unload my "Large Image" source several times. While doing that the audio and video feed is lagging, and eventually after a while the beloved "adding X milliseconds of audio buffering" message appears, saying the total audio buffer is 512 ms. This happens in both OBS instances simultaneously! From this point on audio and video is not in sync anymore in OBS 2, at least not audio over the monitoring device. Now when disabling the NDI receiver source and enabling the Teleport receiver source it doesn't get better.
Some interesting observations here:
- while both NDI and Teleport are no longer in sync just the same, they have different delays between audio and video when compared to each other
- I am using a VLC source for the video loop since these used to be more reliable/less buggy for me, but I also reproduced this with a Media source just the same
- doing a recording from OBS 2 does not show the issue, in the recorded video file everything is in sync
- the desync affects the whole A/V feed, e.g. if new sources with other videos are now added to OBS 1 they are also desync on OBS 2
- the audio meter in OBS 2 is in sync with the audio but not with the video anymore, so it actually visualizes the desync so that even deaf people could perceive the A/V desync
- using exeldro's monitor audio filter has the same problem
- restarting OBS 2 does not fix the problem, the stream is reconnected and once more desync
- restarting OBS 1 (and not restarting OBS 2) does fix the problem, A/V sync is recovered
Here is a video of such a test session. It also shows how a recording is done at the end, this is the recording that it produced.
Setup with 3 instances -> Monitoring audio sync breaks and recording also has A/V desync
I do use the audio monitoring feature so the first scenario was already bad enough. But it gets worse. Let's extend the setup to this:
OBS 1 -> OBS 2 -> OBS 3OBS 2 is configured like before but also outputs a Teleport/NDI stream itself, which OBS 3 has receiver sources for.
Now when I produce lag on OBS 1 the same way I get the A/V desync when using monitoring audio just as before. But now when I record on OBS 3 the recorded video file also has A/V desync in it, albeit not as bad when compared to the monitoring audio ("only" about 200 ms). This is the recording of such a session, you can see it's in sync at first, then I produce the lag and when I stop (after I have seen the audio buffering message) it's no longer in sync.
To fix the desync both OBS 1 and OBS 2 need a restart, OBS 3 can be left running.
What now?
The actual setup I want to use is even more complicated than that: over network, even more OBS instances involved, two separate feeds with combined audio and video that are only combined at the last OBS instance of the chain, lots of scenes, sources and effects...
And it's not like this is not working at all. I have actually used such a setup for quite a long time and sometimes I had sessions with no A/V desync at all and sometimes I would have to reset everything multiple times within a stream of a few hours. That makes sense after my tests above, since even with the "load/unload large image" trigger causing the actual problem was very random. Sometimes it would happen after 10 seconds, sometimes even after 2 minutes all the lagging didn't cause any desync and/or audio buffering increase message.
Anyway, eventually I got annoyed and started to investigate a bit more, which is how I got to create those simplified scenarios, making the problem easier to debug and understand - but with the risk that there are more problems in my more complex setup that I don't even get here. In my full setup I have also seen A/V desyncs when no OBS log has shown any audio buffering message, but one step at a time, let's focus on the obvious parts first.
Also I was able to improve the situation already by using the Mute Audio filter linked above on various sources and replacing Media sources and Image sources that dynamically load images with Browser sources, because they don't produce a lag when loading in the first place (if they need more time for loading the image then it is just shown a bit later, the behavior I would have expected from the Image source too).
While this obviously hasn't helped with my scenario maybe this will, but who knows when it will be released.
Is there anything else I can do? While NDI and Teleport do basically the same thing, they internally work in quite different ways, yet suffer from the same problem. That makes me think the problem is more likely to come from OBS than from one of these plugins, but of course I can't know for sure, maybe they just both integrate into OBS in a bad way.
In my real setup I need both a networked transfer and a transfer between OBS instances within the same PC, I see no way to do all that with HDMI cables and capture cards. I thought about "streaming" from OBS to OBS using RTMP and a relay like nginx-rtmp but in my 3 instance scenario also the recording went desync so I would assume that the same would happen with the stream. It all comes down to OBS doing something weird when it is lagged, but again, I see no realistic way to avoid all potential lags that could occur.
At this point any ideas are welcome, even the most crazy ones...
Last edited: