The higher you set your monitor Hz, thus the higher you set the fps the monitor will be able to display, the more frames the GPU has to render, and this needs GPU computing power. If the GPU is maxed out, things start to slow down. OBS needs its share of the GPU, so it will suffer if you max out the GPU with your game.
So don't increase the fps to a value where things start to slow down.
You can try to start OBS as administrator, because in this mode OBS is able to forcibly reserve a bit of GPU power, so the capture is probably better.