In case it helps, here's some info from PixInsight's developers about how these top-level windows (what I called "floating windows" and the OP called "inner windows") that OBS fails to see when capturing the whole desktop are created.
These windows are independent controls, not child windows of PixInsight's main window. These top-level windows are always native controls. In Qt terms, this means that QWidget::winId() always returns a nonzero integer when invoked for these objects. For top-level windows QWidget is used merely as an abstract interface between the PixInsight core application and the underlying native windowing system. This allows us to implement graphical effects with hardware acceleration, such as translucency and animations, which would be too slow or even impossible with non-native controls.
Please note that disabling the graphical effects still generates the error. Any clues? It does appear to be as a bug in OBS. Thank you.
