Massive amount of skipped frames due to encoding lag on RTX 3080 (NVENC)

[saejii]

Recently I bought a new system with a Ryzen 7 5800x and RTX 3080. I planned to use this to make videos, however when I tried to record using the same settings I had used on my old PC (i7-6700 and GTX 1070) It lagged tremendously. Thought it was a playback issue at first, but it wasn't. I checked the stats and half of the frames were skipped due to encoding lag. I'm trying to record Minecraft at 240fps 1440p, something that should work just fine on this PC (I get 1400 FPS in game, game doesn't lag but OBS does.) Running OBS as admin didn't fix it. Any help would be appreciated.
Log: https://pastebin.com/2sXxGBMr

 

[qhobbes]

1. Change the refresh rate on your Acer monitor from 165 Hz to 120, cap your game at 120 FPS and set your OBS FPS to 60 after step 2.
2. In Windows 10 versions 1809 and newer, we recommend that "Game Mode" be enabled for maximum gaming performance. Game Mode can be enabled via the Windows 10 "Settings" app, under Gaming > Game Mode.
3. Display and Game Capture Sources interfere with each other. Never put them in the same scene (Minecraft).
4. Record to FLV or MKV. If you record to MP4 or MOV and the recording is interrupted, the file will be corrupted and unrecoverable.

If you require MP4 files for some other purpose like editing, remux them afterwards by selecting File > Remux Recordings in the main OBS Studio window or enable Auto remux in the advanced settings.
5. Settings, Output, Recording, uncheck box for Pyscho Visual Tuning.

 

[saejii]

1. Change the refresh rate on your Acer monitor from 165 Hz to 120, cap your game at 120 FPS and set your OBS FPS to 60 after step 2.
2. In Windows 10 versions 1809 and newer, we recommend that "Game Mode" be enabled for maximum gaming performance. Game Mode can be enabled via the Windows 10 "Settings" app, under Gaming > Game Mode.
3. Display and Game Capture Sources interfere with each other. Never put them in the same scene (Minecraft).
4. Record to FLV or MKV. If you record to MP4 or MOV and the recording is interrupted, the file will be corrupted and unrecoverable.

If you require MP4 files for some other purpose like editing, remux them afterwards by selecting File > Remux Recordings in the main OBS Studio window or enable Auto remux in the advanced settings.
5. Settings, Output, Recording, uncheck box for Pyscho Visual Tuning.

I'm trying to record at 240 fps, and I'd rather not change my refresh rate to that low. I also didn't use the display+game capture scene for this encoding session. The main thing is, on my old PC, I recorded with my FPS uncapped in game, game mode off, 144hz, etc using the same encoder. I will try turning on game mode but I'm not really that willing to turn down my refresh rate/cap my fps especially since I'm trying to record at 240fps. I have a friend with a 2 year old laptop who records at higher settings than these and it runs fine.

 

[qhobbes]

Start with those changes and work your way up. Try the following in order (Refresh rate/Game FPS/OBS FPS) 120/120/120, 120/240/120, 120/240/240