A slowly changing AV delay is indicative of a clocking / reference issue. This can happen when you're using an internal source (i.e., the computer's own reference clock) to synchronise sources. This is also known as "free-running" sync.
What is your Behringer device? Are you able to feed it with the same source of reference clock as your cameras? More information on your external hardware (audio interface, video devices) would be useful.
Typically to avoid these kinds of issues in a professional environment, you have a primary source of reference ("black and burst" or "tri-level sync" are the predominant formats); this is fed over SDI to all audio/video devices including any capture devices, and they reference to that. Some prosumer or cheaper pro-grade equipment doesn't always include a feature to take external sync, so you have to design your capture setup carefully.
It may be that your video is drifting and your audio is OK... But without definitive reference to clock each device, it's hard to tell. Are you applying many effects to your scenes or incoming video? An OBS log is always helpful.
https://obsproject.com/forum/threads/please-post-a-log-with-your-issue-heres-how.23074/