Some suggestions based on the log:
14:40:24.024: YUV mode: 709/Full
Change that to 709/Partial. You're incurring more load while decreasing quality for most users, whose displays do not support 709/Full.
https://obsproject.com/forum/resour...t-color-range-settings-guide-test-charts.442/
14:44:07.523: [jim-nvenc: 'recording_h264'] settings:
14:44:07.523: rate_control: VBR
14:44:07.523: bitrate: 40000
14:44:07.523: cqp: 15
For most cases with NVENC recording you want CQP rate control, not VBR.
14:44:07.523: lookahead: true
14:44:07.523: psycho_aq: true
These two features incur extra GPU load. You may wish to experiment with turning one or both off, especially lookahead when recording high motion content.
However, bottom line is that 144Hz does not pulldown easily to 60fps, which will introduce judder. In addition, in Windows 10 using displays with different refresh rates can also cause stutter. The only known workaround is to set all displays to the same refresh rate. If you're trying to record or stream 60fps, you're better off using 120 as a refresh rate instead of 144.