Hello, i was having this issue as well and i was able to fix it for my setup. I posted a short video that shows how to check if you're having the same problem i was and also how to fix it using the workaround i found.
https://youtu.be/LLggr5ZaSco
That's interesting, so it's a sample rate initialisation problem causing the slow drift. Great work for picking that up from the logs, and good thinking to use an OBS defined input, that's very handy that it forces the device sample rate to the working sample rate.
Is there any crossbar configuration or input configuration for the Elgato in the OBS source Properties menu, like there is on other video capture cards? I'm wondering if the audio sample rate setting may need additionally configuring. Apologies if you've already tried that.
Useful video, thanks for putting it together. I don't use an Elgato but I know quite a few who do, this is a very useful workaround to keep in mind.