Hm, I believe I am seeing the problem now. The problem wasn't with window capture, the problem appears to be.. Okay, first of all is that SLI? Also, hm, two monitors plugged in.
Okay, I have a theory of what's going wrong. I'm guessing that with your super-ultra-highend setup you most likely got SLI. And I'm going to somewhat guess that you may have plugged in one monitor in to one card and then another monitor in to the other (although that may not be significant, it may still happen if they're both plugged in the same one, and might be caused partly by SLI).
I don't know if SLI is in use here, SLI may be unrelated to the issue, but regardless, it appears it might be related to the two monitors. This happens under very rare circumstances and I've seen it happen before. I don't know exactly what conditions trigger it but I can only guess.
First, you may want to ensure that the two monitors are plugged in to the same adapter, not different adapters. May be part of the problem.
Second, try turning off the preview in OBS (right click the preview pane to get the menu where you can disable it). In this circumstance the preview may be causing a major stall due to the two monitors and causing data to be transferred in a super slow way by the driver somehow so it can draw on the other monitor (though if you followed the first step this usually shouldn't be an issue, but SLI may play a role in still causing problems with this, and this alone may fix it)
Note, that it's possible SLI might be involved as well. SLI and the capture methods OBS uses actually do not go together too well -- this recommendation is not something I like to have to tell people because of how expensive SLI is, but it's a common issue. I usually recommend temporarily disabling SLI while capturing if you're able. If you're running a super GPU intensive game that doesn't really play well without SLI, you may try to see what capture methods work best with SLI to try to ensure that you can capture it. Though you have enough CPU to spare that there are potential workarounds to allow capturing with SLI. Otherwise, I feel like you might want to disable it while streaming/recording. SLI is meant for playing games as fast as possible, not so much for capturing, capturing can have issues with it.
However if you get it resolved with the prior suggestions and things are working fine with SLI, then you can probably ignore that last bit about it.