Impossible to sync audio and video due to audio plugins causing targets to randomly move

D

Deleted member 48053

I've been trying and failing to sync all my audio (specifically outputs) and video for the past few days. I understand that it's impossible to sync everything 100% (as I'm not using something like Jack Audio that would actually sync the clocks of all these devices), but I'm not even able to keep everything within 50ms. Based on my testing, I assume that this is due to audio plugins and this doesn't seem to be exclusive to third-party plugins.
All of my inputs and outputs are set to 48khz.

Here's an example:
Adobe_Premiere_Pro_swbpCXeuUM.png

1. As shown in the above screenshot (ignore everything but the first audio track), I'll record a sync video (https://twitch-event-engineering-public.s3.amazonaws.com/sync-footage/Sync-Footage-V1-H264.mp4) to figure out how out of sync my desktop audio is from my desktop video.
Adobe_Audition_08S0xcKK86.png

2. I'll then figure out how bad the delay is in ms (using the amount of samples between the video and audio divided by my sample rate of 48khz) in Adobe Audition. I'll then change the audio device's sync offset in OBS accordingly.
Adobe_Premiere_Pro_WYqG824Mto.png

3. I'll try recording the sync video in obs again to find that I'm now off by a different number of ms.
Sometimes, my desktop audio moves over to being early and at other times, my desktop audio moves over to being more late. On the main output I'm having issues with, the change is always something crazy like 50ms and it's never predictable based on the sync offset change that I made.

I also find that audio track 1 (the track that has all my audio on one track) is inconsistently delayed in relation to all of the individual audio tracks. In two tests I just performed (without changing anything between tests), I found that audio track 1 was delayed by 10ms in the first test and 4ms in the second test in relation to the individual audio track. In these two tests, the individual audio track was also inconsistent in relation to the video (12ms early in the first test, 44ms early in the second test). Though, this specific issue isn't as big of a deal and I realise that it probably has something to do with the bigger inconsistency issue that I'm facing.


I genuinely do not understand what's going on and this has been driving me insane for the past few days. I have this issue with my other desktop audio output as well.


Here's my setup for troubleshooting purposes:

1. JDS Atom Labs DAC+Amp (with OBS compressor, OBS gain, OBS gain, and OBS compressor plugins on it)

-I just tested this output twice without changing its sync offset. In the first test, the device was 76ms ahead of the video. In the second test, the device was 11ms behind the video.
-I just tested this output twice without plugins and without changing its sync offset. In the first test, the device was 2ms ahead of the video. In the second test, the device was 7ms ahead. This would be acceptable if I could get this to happen with plugins.

2. Bose Companion 3 Speakers (with a Fabfilter Pro C-2 plugin on it)
-I just tested the speaker twice without changing sync offset between tests. The first test had this device at 11ms ahead of the video and the second test had the device 16ms behind the video.
-I just tested the speakers twice without the plugin and it wasn't any different. Without changing any settings between tests, the first test found the device 8ms late to the video and the second test found it 16ms early.

3. XLR microphone plugged into a Scarlett 2i4 interface (with OBS compressor, OBS gain, and OBS noise gate plugins on it)
-I use this input as a backup, so I don't care about whether it's inconsistent.

explorer_mYjX1Y9nT3.png

4. Same microphone and interface routed through Reaper using ASIO Link Pro, which also gets network input from another instance of ASIO Link Pro. This second instance of ASIO Link Pro is using ASIO4ALL to get input from a xenyx302 mixer. These two inputs are separate in Reaper, but combined to a virtual input before reaching OBS.
-In Reaper, the microphone is going through BlueCat Gain 3, RX 8 Voice De-noise, FabFilter Pro C-2, Waves CLA-2A, RX 8 Voice De-noise, Fabfilter Pro-Q 3, RX 8 De-ess, Izotope Ozone 7 Maximizer, Fabfilter Pro-Q 3, Cockos ReaGate, and Cockos ReaDelay.
-In Reaper, the xenyx302 mixer is going through Izotope Ozone 7 Maximizer.
(In OBS, OBS gain and Fabfilter Pro-Q 3 plugins are on the input)
-I understand that this particular part of the setup is a bit much, but I was having this same problem when I was just using voicemeeter and vsthost to host a few third-party plugins.
-I just tested this input twice using the sync video on my phone playing video through a webcam. The first test measured this input at 77ms behind the video. The second test measured it at 61ms behind. 10ms of deviation here is acceptable to me. I'll just adjust the sync offset after I figure out the issue with my outputs.


At first, I thought that the issue I was facing had to do with CPU usage or something, but my CPU never goes over 60%.
Taskmgr_OMIw6ZNiB3.png

The above screenshot was taken when I wasn't recording, but had a game and all of my usual programs opened.
Taskmgr_3SXdSaDfwP.png

The above screenshot was taken when I started recording with a game and all of my usual programs opened.


Does anyone know why my outputs are so inconsistent? I have no idea at this point.
 
D

Deleted member 48053

I don't know how helpful this will be, but I re-opened OBS, ran two tests, and created a log file. xmos audio is the JDS Labs Atom AMP+DAC and realtek audio is the Bose Companion 3. I do have one other audio input, but it's a virtual device that gets its input from wi-fi and it wasn't connected for any of these tests.

It's saying that my GPU is overloaded, but my gpu never gets above 13%.. I don't get that. Maybe that's just a result of the Hardware-accelerated GPU Scheduler? I'll turn that off and run some tests again in a second. I doubt that it's the problem, but I might as well try.
 
D

Deleted member 48053

Here is my updated log file after disabling the Hardware-Accelerated GPU Scheduler:

Though, I'm still getting latency variations. After restarting my computer and running two more recording tests, the latency variation between track one in these two recordings (using the JDS Atom Labs DAC+Amp output) was 40ms.
Without plugins on this output, the variation was 23ms.

I also forgot to mention that the delay between the audio and video changes within individual recordings as well.
I just created a test recording (using the JDS Labs Atom DAC+Amp output with no plugins) where I played one click at the beginning of the recording and I played one click 2 minutes later. The first click was 25ms ahead of the video and the second click was 15ms ahead of the video. Therefore, the variance between these two clicks in this one recording was 10ms.
When doing this same test with the plugins on the output, the variance was 13ms.

This would imply that the delay is jumping all over the place even throughout individual recordings. This could mean that my outputs could sway from -5ms at the beginning of a recording/stream to +50ms by the end.

I still have no idea as to what's going on and this is all the information I can think to provide. Thanks for reading.
 
Last edited by a moderator:
D

Deleted member 48053

sorry to keep adding to this, but my second last test also implies that it's not just audio plugins causing the issue (as the output didn't have any audio plugins on it and still had inconsistency within the same recording within 2 minutes).

In regards to the repeated error on the log file (about "gl_shtex_init_d3d11_tex" failing to "create texture"), I'm pretty sure that this is just the result of OBS trying to hook into osu! and not being able to because it's minimized.

I'm okay with my input having 16ms of delay variance because it's understandable, seeing as how there are so many plugins on it. The outputs varying this much just doesn't make sense to me as even when my outputs aren't going through any plugins, I have wild delay inconsistencies at random.
 
Last edited by a moderator:

FerretBomb

Active Member
Unfortunately there can be multiple sources of audio drift. Input side can vary at the OS level, if you have device timestamps enabled and the device isn't sending consistent stamps it can introduce drift, if you have sampling rate mismatch it can cause drift, added processing steps of course add variable delay when it's not a hardware stack. And even on the output side, if you're using NVENC there was a bug a couple of versions back that can cause audio tracks to be off by multiple seconds. Not sure if that ever got fixed.

I'd recommend recording using Audacity or similar to check delay outside of OBS, to help narrow down the root cause of the variance.

And @ShadyShroomz, seriously man. If someone's looking for advanced technical help, memeing isn't the way to go if you can't offer constructive feedback.
 
D

Deleted member 48053

Unfortunately there can be multiple sources of audio drift. Input side can vary at the OS level, if you have device timestamps enabled and the device isn't sending consistent stamps it can introduce drift, if you have sampling rate mismatch it can cause drift, added processing steps of course add variable delay when it's not a hardware stack. And even on the output side, if you're using NVENC there was a bug a couple of versions back that can cause audio tracks to be off by multiple seconds. Not sure if that ever got fixed.

I'd recommend recording using Audacity or similar to check delay outside of OBS, to help narrow down the root cause of the variance.

And @ShadyShroomz, seriously man. If someone's looking for advanced technical help, memeing isn't the way to go if you can't offer constructive feedback.

Shady's Recommendation:
Alright, I'm losing my mind, but I just finished the tests. In private, Shady told me to try 30fps with osu! closed. I ran two tests in which I played a click at around 0:00 and at around 2 minutes. I stretched the sync video out by 50% and rendered it at 30fps so that I could get more a accurate measurement here.
Here are the delays from these tests using the JDS Labs DAC+Amp along with the log file:

test 1:
0 - audio +3ms (behind video)
2mins - audio +102ms (behind video)

test 2:
0 - audio +22ms (behind)
2mins - +88 (behind)
https://obsproject.com/logs/dOH_Vsw-2QlFwrKv



A new issue emerged that began as a specific issue to OBS, but is now a broader issue. I'm just logging it here for later:
After these tests, I tried to record in audacity (using Windows WASAPI mode) while OBS was recording as well (back at 60fps). I did this to see if the variable delay in OBS would also be found in audacity.
I noticed that in the OBS audio files exclusively, I started getting popping/clicking on transients (when viewing the file in VLC or audition). I eventually narrowed it down to the first compressor that I have on the track. I tried replacing the OBS compressor with a Fabfilter ProC-2 (using the same settings) and I got the same popping/clicking on transients. Though, when applying the exact same compressor and compression settings after recording (in adobe audition) without any plugins on the track in OBS, I didn't get the popping/clicking .
I became salty and updated windows as a last resort. I know that this had something to do with Audacity using the device in WASAPI mode, but I wasn't thinking straight.
An hour or two after updating, the problem just went away. Then, it came back in a different form. Now, I get popping/clicking on transients even when I apply those same compression settings in Adobe Audition after recording in OBS with no plugins on the track. Why? I have no idea. These compression settings weren't causing problems before, but they are now..
I've tried multiple outputs and devices to see if my headphones just blew out or something, but the same pops/clicks on transients are present on other headphones, my speakers, and my phone. I don't know.
Regardless, this is the log file I went into the next test with: https://obsproject.com/logs/nvz3H1ozfBtuCm1h



Your Recommendation:
I conducted an altered version of my previous test with 60fps, almost everything closed (except for my browser, audacity, and OBS), and with audacity recording.
In the end, audacity stayed in time with OBS, so I won't display the delays separately here.
I also found that I could only get audacity to keep recording audio if the sync video was playing (even if it was muted between clicks).
In the first test, I had no plugins on the track. In the second test, I had plugins on the track.

test 1 (no plugins):
0 - audio -3ms (ahead)
1min - audio -7ms (ahead)

test 2 (w/ plugins):
0 - audio +12ms (behind)
1min - audio +13ms (behind)

I restarted windows, did the test with plugins and audacity again, and came up with a similar result (though, I forgot to log it).
I then did two test with audacity closed and the plugins on immediately after...

test 1:
audacity closed with plugins on the track:
0 - audio +72ms (behind)
1min - audio -.25ms (ahead)

test 2:
audacity closed with plugins on the track:
0 - audio -6ms (ahead)
1min - audio +3ms (behind)

I ran a third test of the same kind for good measure:
test 3:
audacity closed with plugins on the track:
0 - audio -2ms (ahead)
1min - audio +71ms (behind)

I think I ran the second test with the sync video playing muted in the background (instead of just playing one click and then playing another click a minute later), so I did a fourth test with the sync video muted in the background.

test 4:
audacity closed with plugins on the track:
0 - audio +3ms (ahead)
1min - audio +5ms (behind)

Okay, so my hypothesis is correct.. This behaviour is clearly different than the behaviour I started out with, but it's probably one of the major issues I'm facing. If there's no audio playing, my audio device goes to sleep or Windows just doesn't put much priority on it or something. When the device is asked to be active again, it's out of sync.
Though, this doesn't make sense as I have USB selective suspend turned off.
rundll32_KVNvZTOkt9.png
 
Last edited by a moderator:
D

Deleted member 48053

I think I figured it out. I just found sleepless drivers for the JDS Atom dac+amp. I'm pretty sure this is this fix.
brave_Ujmpehmbx5.png

I just installed the drivers and I'll test everything again in a bit.

My fear is that this was only one issue and that there are still other issues that were causing the initial inconsistencies with the other output.. Though, this probably explains why the inconsistencies with this output were much worse.
As for the new popping issue with the compressor, that's probably a personal thing I'll need to figure out another day.
Thank you so much for the help, @FerretBomb . I'll update in an hour or few.
 
Top