Weird but predictable problem stuttering and missing frames with BlackMagic (4k) (decklink plugin) capture source

[dcmouser]

I've got a strange problem with a Black Magic capture source (4k Quad Hdmi capture board) causing missed frames.

Here's the problem:
If I switch away from the source (or simply toggle it invisible) for more than 15 seconds or so, and then come back to it (or mark it visible), I will get a very repeatable stuttering and dropping of 10-20 Missed Frames.
If I just switch away for a few seconds, the problem does not occur.

This happens even when not recording or streaming, is very repeatable, and is happening on a very fast machine with render times of 1.4ms and 1% cpu use.

The REAL confirmation about the oddity of this being unrelated to performance is that if i use Exeldro's source dock plugin to show an additional dock displaying the 4k blackmagic capture source at all times, then the problem goes away completely.

SO: The problem is being caused by the blackmagic source NOT being displayed for some amount of time (15 seconds or so), as if the long delay is causing OBS to free the capture object from some cache, or close the connection to the capture device, and then have to reconnect.

I've gone into the decklink plugin code and added some debug statements to see if I could catch the decklink plugin disconnecting and reconnecting but I didn't see anything triggered.

What I don't know is if the root of the problem is with OBS doing something like releasing and then reacquiring the capture device, or with the decklink/blackmagic api dll somehow deciding OBS hasn't asked for video data for a while so it's safe to release the device, etc., it's almost like the device is going to sleep. I'd be happy to add an option to prevent the device from sleeping if i knew what was happening.

Any ideas?

[ps. Before you ask -- the logs show nothing happening when this stuttering missed frames triggers.]

 

[AaronD]

Regardless of what's really happening, it looks like the workaround is to always render that source. Perhaps you'll find the Multiview to be useful enough to keep open?

 

[dcmouser]

Exeldro's source dock plugin is the only way I can think of to have a source always active even when not showing in the current scene.. But it's a much less than ideal situation since I'm basically paying the cpu/gpu cost of rendering a 4k source every frame that I don't need to be rendering 99.9% of the time.

I'd love to figure out the nature of the problem.

 

[dcmouser]

I've spent the day trying to go through the blackmagic plugin code and obs code to figure out the nature of this odd problem.
No matter what I do, the problem remains:

If I switch away from a blackmagic (4k) capture input scene for less than a minute or so, I can switch back to it, with no problems and sub 1ms render frame times.

But if i stay off that scene for more than a minute or two, and then switch back to it, I get a spike of a very high render time (>40ms) and a bunch of missed frames.

Can anyone make any guess about what is happening and where to look for this hiccup?

I can't tell if it's happening in core OBS code related to removing a source from some caching procedure, or if its the blackmagic decklink plugin code in obs/plugins, or if its somehow in the blackmagic sdk. I've added debuggin statements to the decklink plugin so I know it's not unloading/deactivating when not visible.. I can't figure out WHERE the hiccup is happening.

Desperate to fix this!

 

[dcmouser]

Note that there is nothing in the log at all when this happens.. Just the normal changing scenes.. The real clue to this mystery is that if I change scenes for less than minute, I can return to the 4k blackmagic capture scene with no hiccup and no stutter in super fast rendering times.. But if I switch away from this scene for more than a minute or so, and then switch back to it, it triggers the momentary high spike in frame render time and missed frames, then settles back down into fast rendering... So strange..

 

[dcmouser]

One more clue: This happens in OBS 28 and the very latest OBS 29.
It happens on a brand new clean install of OBS29 portable, with nothing but 2 test scenes showing the blackmagic capture in one, and nothing in the other.

(also note that the problem goes away if the blackmagic capture source is visible in both scenes -- it only happens if i switch to a scene where the capture source is NOT present and then back; note also that the source is NOT set to deactivate when source is hidden, and i have confirmed that it is not disconnected from the video capture source in the decklink/blackmagic plugin).

 

[khaver]

What happens if you add the decklink source to both scenes but in one you move the source out of the frame in one scene? Or if that still triggers the dropped frames, resize the decklink source in one scene so it's only a pixel or two visible.

 

[dcmouser]

I can confirm that if the blackmagic source is on both scenes and just off screen, no more missed frames.
This seems to confirm the problem is something going wrong when the blackmagic source is not displayed for a minute or two..

So this might be another workaround kludge, in addition to creating a source dock which always displays the blackmagic source, but again an uncomfortable one, because of the extra cpu usage rendering a full 4k source for every frame, and the need to add this source off screen to every other scene..

If anyone has theories about what is really going wrong with the OBS code, I would be happy to try changes (to core OBS or the decklink plugins) and try rebuilding and testing.

 

[khaver]

It may be the decklink drivers themselves. If you try capturing using the software that came with the decklink, does the capture start immediately, or are there some frames lost at the beginning before it starts capturing?

 

[Tomasz Góral]

source is set on autodetect resolution?
i newer see any problems on quad hdmi. Igot two cards one is quad hdmi and second mini recorder 4k (5 sources).

 

[dcmouser]

Problem remains whether it's set at autodetect resolution or not, and I've tried all the other source settings with no effect (buffering, audio channels, etc.)

I'll also note that this isn't one of those rare random problems, the hiccup happens consistently every time I switch away from the blackmagic source for more than a minute or two and then switch back (on a clean obs 29 install with no plugins and only 2 scenes).

The capture device is a DeckLink Quad HDMI Recorder, purchased new.

 

[Tomasz Góral]

Which driver version are you using?
I use one old. But not too old because from OBS 26.x minimal version driver is 11 (or maybe 10.x).

 

[AaronD]

Don't know if this means anything, but I might have had a bad Decklink Duo 2 (quad SDI) from Amazon. Never worked, always black screen, Support couldn't figure it out either. Ended up sending it back for a refund. A half-price Chinese thing works perfectly:

Acasis 4 SDI Input 1080P 60FPS 4 Channel PCIe Video Capture Card

Specifications Host interface: PCIe 2.0 X4 Transmission bandwidth: Up to 20Gbps Input interface: SDI*4 Compatible development interface: Directsound, Directshow (Windows) Output color space: YUY2 Capture resolution: Resolution within 1080p De-interlacing: Hardware de-interlacing + 3D noise...

www.acasis.com

They also have HDMI cards.

My rig is actually Linux, but I put a temporary unregistered copy of Windows 10 Pro on it for BM Support to be familiar with. Same problem both ways. The Acasis "just worked" on Linux, so I didn't try it on Windows, so you can take that for what it's worth.

Interestingly enough, their Linux driver comes as source code and a script that compiles it on the host machine. I guess that's one way to make sure it works on the wide variety of systems out there! Just run the script and watch it work, just like a Windows installer...except you do it in a terminal window.

In the meantime, have you tried this?:

Regardless of what's really happening, it looks like the workaround is to always render that source. Perhaps you'll find the Multiview to be useful enough to keep open?

 

[dcmouser]

I'm using the latest black magic drivers (12.4) but upgraded a while ago to see if it would fix this problem, so i had some earlier version that also had similar issue.

The workarounds of always rendering the 4k source on every frame do work, but ouch that's a painful workaround for reasons I've described above. I'd love to isolate where the problem is and whether it lies with the blackmagic drivers or the obs decklink source plugin, and then a "cheaper" workaround if not an outright fix.

Honestly I wouldn't be above coding a custom obs plugin kludge that fixed the problem by pinging the blackmagic driver once every few seconds, or kludging some fix to the obs decklink plugin if I could figure out the nature of the problem..

 

[dcmouser]

Does anyone know if OBS has any kind of non-trivial caching system for sources, where it might keep a source in memory for 10-20 seconds after its last display and then swap it out? Or is this more likely to be an issues with the BlackMagic plugin..

 

[dcmouser]

Spent another day trying to track down this problem.. timing functions but not having any luck figuring out where the hiccup is happening.. My gut tells me it's some kind of caching issue since it only kicks in (and always kicks in) after i have been switched away from the blackmagic source for a while.. I was looking into the obs-source.c functions for dealing with asynchronous sources and caching frames, but i'm having a hard time catching any function taking an unusual amount of time..

But I do think I may have a new clue.. If I force the blackmagic source to 1080p, same as project resolution, then the problem seems to go away.

This suggests to me that the problem may not be specific to the blackmagic source, but rather to it simply being a 4k source.

So perhaps the problem is simply caused by switching from ANY 4k source to a scene without a 4k source for a few minutes and then switching to it resulting in a bunch of dropped frames.. Does that make sense to anyone? Any ideas where in code to look for where the dropped frames are occuring?