I ran into a really nasty audio problem today with a multicamera NDI setup involving a newly added TBS2605 encoder:
Within the first two seconds after launching the app, I got a huge spew of
messages in the log. Now in theory, this should not have mattered, because HDMI D's audio was muted, and was completely silent anyway, but every time I switched to HDMI D, the audio from HDMI E would burst into flames, stuttering and sputtering, BUT only on the stream going out to the world. The audio monitoring through my headphones was perfect. Had I not started getting frantic texts, I wouldn't have known about the problem at all. Not a good way to start off a streaming session.
Fortunately, the camera that was causing the problem was also redundantly coming in via FireWire (HDV), so I was able to just switch to that for the remainder of the streaming session and debug it afterwards. And to think that I almost unplugged the FireWire cable out of fear that bringing in six sources might overtax the system. Yikes.
Afterwards, I repeatedly tried turning off the various streams, changing the video format, etc. (all the way down to 2 Mbps CBR Baseline), but nothing worked until I completely disabled the audio for HDMI D, and after that it worked perfectly, presumably because the audio decoder on the OBS side wasn't getting any chunks, and therefore the video encoder wasn't waiting for an audio chunk from it that was being delayed by a whole second of unnecessary buffering.
I suspect this may be related to other, similar audio problems that I've had on a recurring basis with the NDI output from the AG-CX350, where its audio would become similarly garbled when taking the video portion from some other NDI source that also provided (muted) audio. (However, that problem was typically audible through the headphones, unlike this one, so it may not be the same issue.)
I thought I had fixed that first problem through some combination of setting the synchronization back to Network instead of Source (because apparently the developers changed the default for that setting, and when I switched machines and imported the settings, they were then different from the machine that worked correctly) and adding an external GPU that could handle all the H.264 decoding more easily. I'm not sure which of those changes actually fixed the problem.
Ironically, I forgot to set these new sources to be network-synced, and didn't catch it until much later, and I get the feeling that changing that setting might require relaunching OBS before it takes effect, which I forgot to do after noticing that the setting was wrong, so that *might* have been the problem today. I'm not sure.
But the possible culprits are:
Next question. I'm trying to understand why OBS is doing this buffering in the first place. What I think is happening under the hood (based solely on observing the app's behavior) is:
Code pointers/documentation would be appreciated.
Within the first two seconds after launching the app, I got a huge spew of
adding XX[X] milliseconds of audio buffering, total audio buffering is now [YYY] milliseconds (source: HDMI D)
messages in the log. Now in theory, this should not have mattered, because HDMI D's audio was muted, and was completely silent anyway, but every time I switched to HDMI D, the audio from HDMI E would burst into flames, stuttering and sputtering, BUT only on the stream going out to the world. The audio monitoring through my headphones was perfect. Had I not started getting frantic texts, I wouldn't have known about the problem at all. Not a good way to start off a streaming session.
Fortunately, the camera that was causing the problem was also redundantly coming in via FireWire (HDV), so I was able to just switch to that for the remainder of the streaming session and debug it afterwards. And to think that I almost unplugged the FireWire cable out of fear that bringing in six sources might overtax the system. Yikes.
Afterwards, I repeatedly tried turning off the various streams, changing the video format, etc. (all the way down to 2 Mbps CBR Baseline), but nothing worked until I completely disabled the audio for HDMI D, and after that it worked perfectly, presumably because the audio decoder on the OBS side wasn't getting any chunks, and therefore the video encoder wasn't waiting for an audio chunk from it that was being delayed by a whole second of unnecessary buffering.
I suspect this may be related to other, similar audio problems that I've had on a recurring basis with the NDI output from the AG-CX350, where its audio would become similarly garbled when taking the video portion from some other NDI source that also provided (muted) audio. (However, that problem was typically audible through the headphones, unlike this one, so it may not be the same issue.)
I thought I had fixed that first problem through some combination of setting the synchronization back to Network instead of Source (because apparently the developers changed the default for that setting, and when I switched machines and imported the settings, they were then different from the machine that worked correctly) and adding an external GPU that could handle all the H.264 decoding more easily. I'm not sure which of those changes actually fixed the problem.
Ironically, I forgot to set these new sources to be network-synced, and didn't catch it until much later, and I get the feeling that changing that setting might require relaunching OBS before it takes effect, which I forgot to do after noticing that the setting was wrong, so that *might* have been the problem today. I'm not sure.
But the possible culprits are:
- Some weird audio timestamping bug related to multiple NDI streams with audio coming from the same IP address (but that seems unlikely, because presumably the IP address never leaves the NDI libraries, and if that were happening, we wouldn't get a viable decode through the headphones, or would we?)
- Audio timestamping bugs when using source timing with multiple sources (particularly if some are set to network timing).
- Some general timing-race-like condition where OBS is trying desperately to reconcile the audio timestamps while the app is still launching, and is incorrectly believing that it will need to add latency even though the system is only going to be slow for another second or so.
- Some bug where the H.264 decoder is getting behind because of poor performance right at launch time, and rather than dropping chunks to catch up, continues running behind forever.
Next question. I'm trying to understand why OBS is doing this buffering in the first place. What I think is happening under the hood (based solely on observing the app's behavior) is:
- When OBS detects that audio isn't arriving from some sources in time to render it, OBS adds latency into the audio processing pipeline (but, judging from folks complaining about audio drift, apparently *not* the video processing pipeline, which is probably a bug).
- The purpose of this buffer is to compensate for computers that exhibit periods of CPU overload, so that they don't stutter.
- This latency is being added only when the audio coming from the affected source is on the program bus.
- If the latency is longer than the maximum amount of time that the encoder is willing to wait for the audio corresponding with the current chunk of video data, you get stuttering anyway.
- The cap on the maximum latency is longer than the maximum amount of time that the encoder is willing to wait for data. (Yes? No?)
Code pointers/documentation would be appreciated.