Short answer: There is no optimal setting that anyone could ever tell you.
Long answer: This software gives you the ability to choose how you want to customize your recording. There are tons of videos on how to stream with this software, but few actually tell you how to record, and record well, and what options to test with. On YouTube look for
@EposVox . He's posted his videos in this forum somewhere. He has a really in-depth video on just the recording side of OBS.
For me, I like Output/Recording/Advanced. Rate Control: VBR
Make profiles for each game.
One game I have set at 3000 VBR, while another game I have at 12000k VBR.
I do record 1920x1080, 60fps.
I also record to a separate SSD. If you only have 1 hard drive consider installing a 2nd.
Encoder I do use the "NVENC H.264" encoder since I do have a GTX 1070. Not sure which models of NVIDIA can use NVENC H.264, but I know mine does. That may help out with the FPS drops.
Realize that your hardware might not be up to snuff for what you're trying to play & record. Have a dedicated drive just to record to will be a major help, and try to at least double that ram, your CPU can handle 32gb if your motherboard can support it. If you have GeForce Experience installed, disable the shadowplay feature, and also look up how to disable XBox DVR (unless you use it, lol).
Test out different games with various settings. Play them back in VLC media player.
[Not my videos, they are from
@EposVox on YouTube]
https://youtu.be/xjNMpiD4Pzc OBS Studio 118 - Recording Settings Walkthrough & Setup Tutorial
https://youtu.be/oQ7GasqKMck OBS Studio 121 - How to Record LAG-FREE with Hardware Encoding - OBS Nvenc AMD VCE Quick Sync Guide
My computer build:
i7-7700K Kaby Lake Quad Core 4.2 GHz
ASUS ROG GeForce GTX 1070 08G STRIX
ASUS ROG Maximus IX Hero Z270
G.Skill Ripjaws V 32GB
Deepcool Gamer Storm Captain 240EX CPU Liquid Cooler AIO
EVGA SuperNOVA 750W Platinum