A more generic hint: NEVER mess with settings. Otherwise it WILL BE a mess. ;-) Even the method trial&error can be done in an ordered manner. For instance changing
one parameter/setting a time, then do short tests, if it makes it better or worsen the things. Note the result.
We could try to help out a bit, if you encounter specific issues and well expressed questions, and upload a log. It should contain the time you had a recording. The best thing is: Start obs, record a gameplay (with your issues persist), stop recording, close obs, re-open obs, upload "last log" from the menu:
OBS Studio creates a log file each time you run it which contains very useful diagnostic information. Without a log file, it's much harder for any support volunteers to figure out problems and your post may be ignored. To upload a log file, go to the Help menu > Log Files > Upload Last Log...
obsproject.com