Question / Help OBS won't start with recording set to 5K

[GoldenSun3DS]

I'm using DSR to make my Windows resolution 5K (4K monitor). I'm using 100% Windows scaling.

If I set the resolution (canvas and output) to anything much above 4K (3840x2160 or 4096x2160), it gives an error message and says I should check if I have the latest driver. I'm using the NVenc encoder and I reinstalled my NVidia Geforce Experience driver.

If I use the CPU encoder, it works, but that's not an option because I have quad core i3 8350K and it instantly maxes out my CPU (as expected since I imagine you'd need a Threadripper to record 5K on your CPU).

Does the NVenc encoder just not work at 5K? Can anyone else see if this does the same thing to you?

My PC build:
https://pcpartpicker.com/user/GoldenSun3DS/saved/F76KZL

 

[Harold]

Where's your OBS log from the attempt?

 

[koala]

nvenc H.264 doesn't support anything above 4096x4096. See here: https://developer.nvidia.com/nvidia-video-codec-sdk#NVENCFeatures

There is H.265 (HEVC) support starting with Pascal GPU's with 8k (8192x8192), but you have to use ffmpeg custom output for that, and unfortunately with that multi-track audio recording isn't available. I got a 5760x3240 video this way, with a GTX 1070. The custom ffmpeg encoder is "hevc_nvenc (nvenc_hevc)", but it's difficult to tune its quality settings. I tried to get a quality-based recording (not bitrate-based) and worked out "preset=llhq rc=constqp qp=26" for Video Encoder Settings. Seems to work, but didn't research for real footage.