Thanks for the advice.
I've been launching OBS as admin exclusively now, I disabled both DVR from Nvidia and Xbox and the Psycho Visual Tuning is off, AND all of my footage is being recorded in MKV now. This did help a lot, there's very little frame loss, but for some reason there's still a bit, no matter what I do. I even started playing with the settings and priorities, including making OBS run on high priority and reducing both video quality and bitrate, but for some reason there's still a consistent frame drop here and there. Also, for some reason the logs still say I'm running a game DVR, but I made certain they're all disabled. Could it be an issue with Windows 11 or possibly with my Elgato only being HD60 S?