Asynchronous Audio Filter 0.3.0

[norihiro]

norihiro submitted a new resource:

Asynchronous Audio Filter - Adaptive sample rate conversion

Introduction

This plugin tries to fix an asynchronous audio by synchronizing to the master clock of OBS Studio.

OBS Studio is running with a clock provided by the OS. On the other hand, some audio interfaces are running with their own clock to sample the audio, which is asynchronous from the OBS's clock.
The difference of these clocks causes the audio buffer to reach overflow or underflow after several minutes or more and causes OBS to drop a bunch of audio frames or...

Read more about this resource...

 

[YorVeX]

Any ideas?

04:03:50.474: [obs-async-audio-filter] lock failed due to large error, 128989.863000 us
04:03:50.515: adding 128 milliseconds of audio buffering, total audio buffering is now 277 milliseconds (source: Mic/Aux)

I thought these audio buffering situations is exactly what the plug-in should prevent. What error then would not be considered "large"?

 

[YorVeX]

On a different OBS instance it seems to work, but it's never stopping this:

05:29:06.228: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.451023 vc2=1.443233
05:29:06.918: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.449924 vc2=1.443257
05:29:07.608: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.449404 vc2=1.443278
05:29:08.298: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.448625 vc2=1.443297
05:29:08.988: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.447774 vc2=1.443313
05:29:09.678: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.447217 vc2=1.443326
05:29:10.368: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.02 ms vc1=1.447711 vc2=1.443338
05:29:11.068: [obs-async-audio-filter] adding one frame, identified error is 29.8 ppm, e=0.01 ms vc1=1.446623 vc2=1.443350
05:29:11.758: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.447710 vc2=1.443363
05:29:12.448: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.00 ms vc1=1.447911 vc2=1.443377
05:29:13.138: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.449471 vc2=1.443395
05:29:13.828: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.00 ms vc1=1.448455 vc2=1.443414
05:29:14.518: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.447737 vc2=1.443429
05:29:15.208: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.00 ms vc1=1.447601 vc2=1.443442
05:29:15.898: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446417 vc2=1.443453
05:29:16.588: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446657 vc2=1.443462
05:29:17.278: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446516 vc2=1.443471
05:29:17.978: [obs-async-audio-filter] adding one frame, identified error is 29.8 ppm, e=0.01 ms vc1=1.446206 vc2=1.443479
05:29:18.668: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446163 vc2=1.443487
05:29:19.358: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446787 vc2=1.443496
05:29:20.048: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446553 vc2=1.443505
05:29:20.738: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446894 vc2=1.443514
05:29:21.428: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446934 vc2=1.443525
05:29:22.128: [obs-async-audio-filter] adding one frame, identified error is 29.8 ppm, e=0.01 ms vc1=1.446692 vc2=1.443535
05:29:22.818: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446571 vc2=1.443544
05:29:23.508: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.00 ms vc1=1.446637 vc2=1.443553
05:29:24.198: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446501 vc2=1.443562
05:29:24.888: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446811 vc2=1.443571
05:29:25.578: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.02 ms vc1=1.449201 vc2=1.443583
05:29:26.268: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.449308 vc2=1.443602
05:29:26.958: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.448329 vc2=1.443618

Is it supposed to be triggered that often? I expected to see something like this once in a while. Does it mean I have some bigger problem somewhere in my setup?
Also, ticking or unticking the "Use OBS time instead of source time" box doesn't seem to change anything about this.

 

[norihiro]

On a different OBS instance it seems to work, but it's never stopping this:

05:29:06.228: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.451023 vc2=1.443233
05:29:06.918: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.449924 vc2=1.443257
05:29:07.608: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.449404 vc2=1.443278
05:29:08.298: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.448625 vc2=1.443297
05:29:08.988: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.447774 vc2=1.443313
05:29:09.678: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.447217 vc2=1.443326
05:29:10.368: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.02 ms vc1=1.447711 vc2=1.443338
05:29:11.068: [obs-async-audio-filter] adding one frame, identified error is 29.8 ppm, e=0.01 ms vc1=1.446623 vc2=1.443350
05:29:11.758: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.447710 vc2=1.443363
05:29:12.448: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.00 ms vc1=1.447911 vc2=1.443377
05:29:13.138: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.449471 vc2=1.443395
05:29:13.828: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.00 ms vc1=1.448455 vc2=1.443414
05:29:14.518: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.447737 vc2=1.443429
05:29:15.208: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.00 ms vc1=1.447601 vc2=1.443442
05:29:15.898: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446417 vc2=1.443453
05:29:16.588: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446657 vc2=1.443462
05:29:17.278: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446516 vc2=1.443471
05:29:17.978: [obs-async-audio-filter] adding one frame, identified error is 29.8 ppm, e=0.01 ms vc1=1.446206 vc2=1.443479
05:29:18.668: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446163 vc2=1.443487
05:29:19.358: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446787 vc2=1.443496
05:29:20.048: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446553 vc2=1.443505
05:29:20.738: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446894 vc2=1.443514
05:29:21.428: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446934 vc2=1.443525
05:29:22.128: [obs-async-audio-filter] adding one frame, identified error is 29.8 ppm, e=0.01 ms vc1=1.446692 vc2=1.443535
05:29:22.818: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446571 vc2=1.443544
05:29:23.508: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.00 ms vc1=1.446637 vc2=1.443553
05:29:24.198: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446501 vc2=1.443562
05:29:24.888: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.446811 vc2=1.443571
05:29:25.578: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.02 ms vc1=1.449201 vc2=1.443583
05:29:26.268: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.449308 vc2=1.443602
05:29:26.958: [obs-async-audio-filter] adding one frame, identified error is 30.2 ppm, e=0.01 ms vc1=1.448329 vc2=1.443618

Is it supposed to be triggered that often? I expected to see something like this once in a while. Does it mean I have some bigger problem somewhere in my setup?
Also, ticking or unticking the "Use OBS time instead of source time" box doesn't seem to change anything about this.

This is expected behavior. As the log message says, your device has 30 ppm error, which is reasonable error. 30 ppm error results adding 1 sample for every 33k samples. Once confirmed it is working, I recommend to reduce the verbosity so that a lot of log lines are left.

 

[norihiro]

Any ideas?

04:03:50.474: [obs-async-audio-filter] lock failed due to large error, 128989.863000 us
04:03:50.515: adding 128 milliseconds of audio buffering, total audio buffering is now 277 milliseconds (source: Mic/Aux)

I thought these audio buffering situations is exactly what the plug-in should prevent. What error then would not be considered "large"?

The threshold is 70ms, hard coded in the plugin. Same threshold is also hard coded in the core of OBS Studio. When the audio source somehow stop sending audio data for more than 70ms, for example caused by jammed USB, this behavior happens. Unfortunately, this cannot be solved without changing OBS itself. The next OBS 28 has 'fixed buffering type' mode. This might improve but I have no confidence.

 

[YorVeX]

This is expected behavior. As the log message says, your device has 30 ppm error, which is reasonable error. 30 ppm error results adding 1 sample for every 33k samples. Once confirmed it is working, I recommend to reduce the verbosity so that a lot of log lines are left.

OK, I was just wondering because on another OBS instance it was trigger a lot more rarely, but I guess it really depends on the devices and sources involved.

The threshold is 70ms, hard coded in the plugin. Same threshold is also hard coded in the core of OBS Studio. When the audio source somehow stop sending audio data for more than 70ms, for example caused by jammed USB, this behavior happens. Unfortunately, this cannot be solved without changing OBS itself. The next OBS 28 has 'fixed buffering type' mode. This might improve but I have no confidence.

I didn't know about these OBS 28 plans, is there more information about this somewhere?

That audio buffering stuff has been bugging me for years and it's nearly impossible to debug. I even have coded a tool that monitors the OBS log and relays any message with "audio buffering" to the Twitch chat so that both the audience and me can immediately see what potentially caused it. In some cases it's something obvious, like a media or image source loading something the first time which can be fixed by replacing these poorly coded things with browser sources that perform a million times better in every aspect (and eliminate that issue at least for these sources). Another classic is I tab in and out of a game on the gaming PC causing a short freeze (that I can't do anything about) and the NDI source on the receiving OBS instance on the streaming PC causes an audio buffer increase of 800 ms or something.

And that's even the good cases, since I know where it's coming from. On other occasions it just happens on "Desktop Audio" when that is an internal audio chip that is not even used, since the streaming PC receives it's (audio) data via NDI/Teleport from another PC. Or even better, the log just says "Source: null". And surely there is also never anything visible during the moment of the buffer message that gives a clue what might have caused it, e.g. it happens after streaming for 4 hours and then 2 minutes into a boring AFK scene with 0.5 ms render time when absolutely nothing changed on any source and there was no sound and the OBS log doesn't show anything before shortly or after this line.
It's extremely frustrating to debug this. Or more precisely: there is no debugging, since I don't even have a starting point. Which is why I stumbled onto your "Mute Audio" plug-in and this one right here and hopefully can use it to further improve the situation a bit.

The worst thing is that if it's really caused by some video relevant source rendering I would rather just have that source show a short lag instead of increasing the audio buffer irreversibly for the rest of the session with no way of resetting it without an OBS restart (killing the live stream) - but OBS just doesn't even give me that option.
So is "fixed buffering mode" that option, or something else?

 

[norihiro]

> I didn't know about these OBS 28 plans, is there more information about this somewhere?
It's this commit. I happened to see Jim and tt2468 discussing about this on Discord.

libobs: Add ability to configure audio buffering latency · obsproject/obs-studio@f482111

Allows a frontend the ability to set the maximum audio buffering latency, and specify whether that audio buffering is either fixed (to the maximum audio buffering latency), or dynamically increasin...

github.com

> Which is why I stumbled onto your "Mute Audio" plug-in and this one right here and hopefully can use it to further improve the situation a bit.
The plugin "Mute Audio" is developed for that purpose. I mainly use an external audio mixer so that I don't need audio from other audio sources such as cameras.

> I would rather just have that source show a short lag instead of increasing the audio buffer irreversibly for the rest of the session with no way of resetting it without an OBS restart
IIRC, that's what the fixed-buffer mode, though I never tried it.

 

[norihiro]

norihiro updated Asynchronous Audio Filter with a new update entry:

Asynchronous Audio Filter 0.2.0

This is an experimental release of Asynchronous Audio Filter for OBS Studio.
Changes in 0.2.0.

• Revise loop filter parameters
• Rewrite build script to support macOS on Apple Silicon. Also use Github Actions instead of Microsoft Azure Pipeline.

If there is a problem, please consider using the previous version 0.1.0.

Limitation

• Not all binaries are tested.
  • Windows: never tested.
  • macOS: Version 0.2.0 was tested on Intel mac...

Read the rest of this update entry...

 

[norihiro]

norihiro updated Asynchronous Audio Filter with a new update entry:

Asynchronous Audio Filter 0.2.1

This is a hotfix release of Asynchronous Audio Filter for OBS Studio.

Changes in 0.2.1

• Fixed the plugin was not correctly loaded on OBS 27 on Windows.
• Fixed the plugin was not correctly loaded on OBS 28 on Ubuntu.

If you are using this plugin with

Limitation

• Not all binaries are tested.
  • Windows: 0.2.1 was tested with OBS 27.2.4 and 28.1.1.
  • macOS: Version 0.2.0 was tested on Intel mac.
  • Ubuntu...

Read the rest of this update entry...

 

[LastofAvari]

Thank you for this plugin. You're a living legend!

 

[therentabrain]

Thank you for this plugin! It seems to be helping. If I'm not mistaken, what had been happening (before I installed this) was, all kinds of random sources were apparently causing audio buffering delays, including things like long mp3s. Over time, the stream fell so far behind that people were responding to me something like a full minute late. Most streams, the log would show maximum buffer reached. And possibly unrelated, my camera would hiccup, freezing intermittently. I would try to stop the stream, and it would take over 30 seconds to stop. Then I'd start it again, and all these problems would be gone, at least for a little while.

I am still using OBS 27 but I'm finally about to upgrade. Is there any wisdom about what that'll mean, and the new fixed buffer settings?

Should I stop using regular media file sources and use VLC ones instead, or vice versa (I use some of each currently), or kludge something with a browser source?!

Thanks :)

 

[norihiro]

Thank you for this plugin! It seems to be helping. If I'm not mistaken, what had been happening (before I installed this) was, all kinds of random sources were apparently causing audio buffering delays, including things like long mp3s. Over time, the stream fell so far behind that people were responding to me something like a full minute late. Most streams, the log would show maximum buffer reached. And possibly unrelated, my camera would hiccup, freezing intermittently. I would try to stop the stream, and it would take over 30 seconds to stop. Then I'd start it again, and all these problems would be gone, at least for a little while.

I am still using OBS 27 but I'm finally about to upgrade. Is there any wisdom about what that'll mean, and the new fixed buffer settings?

Should I stop using regular media file sources and use VLC ones instead, or vice versa (I use some of each currently), or kludge something with a browser source?!

Thanks :)

I don't recommend using this plugin to Media Source or VLC Source.

Most streams, the log would show maximum buffer reached. And possibly unrelated, my camera would hiccup, freezing intermittently. I would try to stop the stream, and it would take over 30 seconds to stop.

At first, I recommend to update OBS to the latest (29.1.3 for now) and you should ask support for OBS Studio itself.

 

[therentabrain]

FTR, I did try this and the mute source plugin and my stream has been much better since. But I haven't upgraded OBS yet. We'll see how it goes this week I think, thanks!

 

[norihiro]

norihiro updated Asynchronous Audio Filter with a new update entry:

Asynchronous Audio Filter 0.3.0

This is an experimental release of Asynchronous Audio Filter for OBS Studio.

Changes in 0.3.0

• Use a resampler library so that the output audio is smooth.
  • Until the old version 0.2.1, the plugin uses a simple zero-order hold (ZOH) resampling algorithm.
  • In this release, the plugin uses sinc interpolation algorithm.

For packagers

From the version 0.3.0, this plugin requires a library...

Read the rest of this update entry...

 

[photoshopeando]

Thanks for creating this plugin. At this moment I'm not 100% sure that it fixes my problems for good, but for the first time I've been able to record a sync test video for 30 min without the audio drifting out of sync.
Regardless of whether it works for every situation or not, this is the only plugin I've found so far that tries to take care of such a widespread issue as drifting audio sync problems. It's weird that this plugin isn't much more popular (at least, it took me some time to find it) since, like I said, this kind of problem is very very common.
Again, thank you for creating it and (hopefully) keeping it updated.