I know this is a little old, but I'm facing similar issues. I'm trying to record gameplay as well as my face cam with the desire to be able to edit them independently once in my editing software (currently Hitfilm). As of now, I'm running two instances of OBS, but now that I'm playing more recently games (Callisto Protocol) OBS overloads very easily. Even if I dumb down the game graphics to medium and run at 1920x1080, and further lower recording capabilities in OBS, I still get cuts in my film and have to redo a lot of play.
I feel like I'm missing something important. Most of the recent knowledge I've been getting is just using one instance of OBS, but I don't know how I can get separate scenes to work with individually. One person mentioned (if trying to play at 2560x1440) is the record at 5120x1440 in OBS and just crop the scenes out in the editor. Based on a comment made right here on this thread, it sounds like it'd be the same computing power regardless if it was only one instance of OBS.
I just feel stuck - there's so many YouTubers out there recording really high-end videos with only one rig, so I must be missing something.