Here is my LOG file, I tried 480 and 600:
18:21:30.045: CPU Name: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
18:21:30.045: CPU Speed: 3600MHz
18:21:30.045: Physical Cores: 8, Logical Cores: 16
18:21:30.045: Physical Memory: 32706MB Total, 20598MB Free
18:21:30.045: Windows Version: 10.0 Build 18362 (revision: 535; 64-bit)
18:21:30.045: Running as administrator: false
18:21:30.045: Aero is Enabled (Aero is always on for windows 8 and above)
18:21:30.045: Windows 10 Gaming Features:
18:21:30.045: Game Bar: Off
18:21:30.045: Game DVR: Off
18:21:30.047: Sec. Software Status:
18:21:30.048: Windows Defender Antivirus: disabled (AV)
18:21:30.048: AVG Antivirus: enabled (AV)
18:21:30.048: Windows Firewall: enabled (FW)
18:21:30.048: Current Date/Time: 2019-12-16, 18:21:30
18:21:30.049: Browser Hardware Acceleration: true
18:21:30.049: Portable mode: false
18:21:30.378: OBS 24.0.3 (64-bit, windows)
18:21:30.378: ---------------------------------
18:21:30.378: ---------------------------------
18:21:30.379: audio settings reset:
18:21:30.379: samples per sec: 44100
18:21:30.379: speakers: 2
18:21:30.382: ---------------------------------
18:21:30.383: Initializing D3D11...
18:21:30.383: Available Video Adapters:
18:21:30.388: Adapter 0: NVIDIA GeForce GTX 1080
18:21:30.388: Dedicated VRAM: 4176478208
18:21:30.388: Shared VRAM: 4262578176
18:21:30.389: output 0: pos={0, 0}, size={2560, 1440}, attached=true
18:21:30.389: output 1: pos={-1920, 0}, size={1920, 1080}, attached=true
18:21:30.392: Loading up D3D11 on adapter NVIDIA GeForce GTX 1080 (0)
18:21:30.450: D3D11 loaded successfully, feature level used: b000
18:21:30.450: D3D11 GPU priority setup failed (not admin?)
18:21:30.798: ---------------------------------
18:21:30.798: video settings reset:
18:21:30.798: base resolution: 1920x1080
18:21:30.798: output resolution: 1920x1080
18:21:30.798: downscale filter: Lanczos
18:21:30.798: fps: 480/1
18:21:30.798: format: NV12
18:21:30.798: YUV mode: 601/Partial
18:21:30.800: NV12 texture support enabled
18:21:30.802: Audio monitoring device:
18:21:30.802: name: Default
18:21:30.802: id: default
18:21:30.803: ---------------------------------
18:21:30.814: [CoreAudio encoder]: Adding CoreAudio AAC encoder
18:21:30.817: Failed to load 'en-US' text for module: 'decklink-ouput-ui.dll'
18:21:30.899: [AMF] AMF Test failed due to one or more errors.
18:21:30.899: Failed to initialize module 'enc-amf.dll'
18:21:30.931: [obs-browser]: Version 2.7.15
18:21:30.941: NVENC supported
18:21:31.088: Couldn't find VLC installation, VLC video source disabled
18:21:31.096: No blackmagic support
18:21:31.105: ---------------------------------
18:21:31.105: Loaded Modules:
18:21:31.105: win-wasapi.dll
18:21:31.106: win-mf.dll
18:21:31.106: win-ivcam.dll
18:21:31.106: win-dshow.dll
18:21:31.106: win-decklink.dll
18:21:31.107: win-capture.dll
18:21:31.107: vlc-video.dll
18:21:31.108: text-freetype2.dll
18:21:31.108: rtmp-services.dll
18:21:31.108: obs-x264.dll
18:21:31.108: obs-vst.dll
18:21:31.109: obs-transitions.dll
18:21:31.109: obs-text.dll
18:21:31.110: obs-qsv11.dll
18:21:31.110: obs-outputs.dll
18:21:31.110: obs-filters.dll
18:21:31.110: obs-ffmpeg.dll
18:21:31.111: obs-browser.dll
18:21:31.111: image-source.dll
18:21:31.112: frontend-tools.dll
18:21:31.112: enc-amf.dll
18:21:31.112: decklink-ouput-ui.dll
18:21:31.113: coreaudio-encoder.dll
18:21:31.113: ---------------------------------
18:21:31.113: ==== Startup complete ===============================================
18:21:31.120: Service '' not found
18:21:31.122: All scene data cleared
18:21:31.123: ------------------------------------------------
18:21:31.141: WASAPI: Device 'VoiceMeeter Input (VB-Audio VoiceMeeter VAIO)' initialized
18:21:31.153: WASAPI: Device 'Microphone (Samson Q2U Microphone)' initialized
18:21:31.154: Switched to scene 'Scene'
18:21:31.154: ------------------------------------------------
18:21:31.155: Loaded scenes:
18:21:31.155: - scene 'Scene':
18:21:31.155: - source: 'Display Capture' (monitor_capture)
18:21:31.156: ------------------------------------------------
18:21:31.185: adding 46 milliseconds of audio buffering, total audio buffering is now 46 milliseconds (source: Mic/Aux)
18:21:31.185:
18:21:52.772: ---------------------------------
18:21:52.772: video settings reset:
18:21:52.772: base resolution: 1920x1080
18:21:52.772: output resolution: 1920x1080
18:21:52.772: downscale filter: Lanczos
18:21:52.772: fps: 600/1
18:21:52.772: format: NV12
18:21:52.772: YUV mode: 601/Partial
18:21:52.774: NV12 texture support enabled
18:21:52.778: Settings changed (video)
18:21:52.778: ------------------------------------------------
18:22:13.812: Starting recording due to hotkey
18:22:13.874: [jim-nvenc: 'recording_h264'] settings:
18:22:13.874: rate_control: CQP
18:22:13.874: bitrate: 0
18:22:13.874: cqp: 17
18:22:13.874: keyint: 250
18:22:13.874: preset: mq
18:22:13.874: profile: high
18:22:13.874: width: 1920
18:22:13.874: height: 1080
18:22:13.874: 2-pass: true
18:22:13.874: b-frames: 2
18:22:13.874: lookahead: false
18:22:13.874: psycho_aq: true
18:22:13.874:
18:22:13.888: [CoreAudio AAC: 'Track1']: settings:
18:22:13.888: mode: AAC
18:22:13.888: bitrate: 160
18:22:13.888: sample rate: 44100
18:22:13.888: cbr: on
18:22:13.888: output buffer: 1536
18:22:13.900: ==== Recording Start ===============================================
18:22:13.900: [ffmpeg muxer: 'adv_file_output'] Writing file 'D:/Obs Videos/2019-12-16 18-22-13.mp4'...
18:22:26.875: Stopping recording due to hotkey
18:22:27.433: Stopping recording due to hotkey
18:22:27.441: [ffmpeg muxer: 'adv_file_output'] Output of file 'D:/Obs Videos/2019-12-16 18-22-13.mp4' stopped
18:22:27.442: Output 'adv_file_output': stopping
18:22:27.442: Output 'adv_file_output': Total frames output: 2152
18:22:27.442: Output 'adv_file_output': Total drawn frames: 2728 (8124 attempted)
18:22:27.442: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 5396 (66.4%)
18:22:27.443: ==== Recording Stop ================================================
18:22:27.443: Video stopped, number of skipped frames due to encoding lag: 2143/2166 (98.9%)
18:22:36.985: ---------------------------------
18:22:36.986: video settings reset:
18:22:36.986: base resolution: 1920x1080
18:22:36.986: output resolution: 1920x1080
18:22:36.986: downscale filter: Lanczos
18:22:36.986: fps: 480/1
18:22:36.986: format: NV12
18:22:36.986: YUV mode: 601/Partial
18:22:36.990: NV12 texture support enabled
18:22:36.994: Settings changed (video)
18:22:36.994: ------------------------------------------------
18:22:43.101: Starting recording due to hotkey
18:22:43.162: [jim-nvenc: 'recording_h264'] settings:
18:22:43.162: rate_control: CQP
18:22:43.162: bitrate: 0
18:22:43.162: cqp: 17
18:22:43.162: keyint: 250
18:22:43.162: preset: mq
18:22:43.162: profile: high
18:22:43.162: width: 1920
18:22:43.162: height: 1080
18:22:43.162: 2-pass: true
18:22:43.162: b-frames: 2
18:22:43.162: lookahead: false
18:22:43.162: psycho_aq: true
18:22:43.162:
18:22:43.176: [CoreAudio AAC: 'Track1']: settings:
18:22:43.176: mode: AAC
18:22:43.176: bitrate: 160
18:22:43.176: sample rate: 44100
18:22:43.176: cbr: on
18:22:43.176: output buffer: 1536
18:22:43.187: ==== Recording Start ===============================================
18:22:43.188: [ffmpeg muxer: 'adv_file_output'] Writing file 'D:/Obs Videos/2019-12-16 18-22-43.mp4'...
18:23:01.774: [ffmpeg muxer: 'adv_file_output'] Output of file 'D:/Obs Videos/2019-12-16 18-22-43.mp4' stopped
18:23:01.774: Output 'adv_file_output': stopping
18:23:01.774: Output 'adv_file_output': Total frames output: 3556
18:23:01.774: Output 'adv_file_output': Total drawn frames: 4879 (8921 attempted)
18:23:01.775: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 4042 (45.3%)
18:23:01.776: ==== Recording Stop ================================================
18:23:01.776: Video stopped, number of skipped frames due to encoding lag: 3542/3563 (99.4%)