I have the same issue as you. What happens for me is that when I select my device for output, I get a small delay. I figured this was normal since the OBS output monitor has a live delay. But after some time, the delay becomes longer and longer. If I just put the Audio Monitor on and wait....within about 10 minutes I can get up to a 3 second delay. I put this in the bug reports some time ago, because i would love to use this plugin, but it is unusable. I do a lot of routing to get into and out of OBS with VST's and other programs, but if there could be a solution that elminates all of that, it would be fantastic.