"Video stopped, number of skipped frames due to encoding lag: 3862/6162 (62.7%)"
^ I would say your more immediate problem is that every test had large amounts of dropped frames which is indicative of GPU overload. Make sure your computer isn't in power saver mode, ensure that the GPU is not in optimal power usage mode, and ensure that you're not pushing the GPU too hard in gaming. If you have say a 240hz monitor or a 4k monitor or something and you need to push the GPU to 90% usage to play the game. It wont encode very well. It will drop frames... and in fact may also look extra pixelated.
Also make sure the NVENC settings are set to use a keyframe every 1, 2, or, 3 seconds. Your encoder reports a keyint value of 250. At 60fps encode that's just over 4 seconds.
