I tried the Nvidia encoder and it seems to work but I still find it weird that I can't record Skyrim with the standard encoder. I'm not sure what encoder Loilo and Nvidia Shadowplay use exactly but I think my computer should be strong enough to use OBS the way other people do. I'm not even streaming. I'm just recording.
I installed CPU-Z and don't understand any of the values in the CPU tab. I'm not super technical. I don't even know what to expect. All I know is that the graphics card I'm using is not bad enough to get such horrible results with vanilla Skyrim Legendary Edition with nothing big running in the background. The only things running in the background are a few game clients that aren't even downloading anything. The CPU usage when trying to record Skyrim is around 70% for OBS alone according to task manager. And that's when the game is paused to look at task manager.
Correct me if I'm wrong, but I think part of your confusion about performance is because you think your graphics card is doing all or most of the work to run your games? If so, that's not true. Your games use your CPU as well. In fact, some games are quite CPU heavy, like Overwatch and Skyrim.
NVENC works better than x264 on your computer because x264
only uses your CPU, and so does almost everything else running on your computer, including games. I'm working on an i5-3570K, which is clocked at 3.4 GHz and is ranked faster/higher than your CPU in some benchmarks. Even so, I have issues doing live encoding and playing certain games at the same time.
Shadowplay uses NVENC, not x264, which is why that works well for you.
I'm not sure what LoiLo uses to encode, and I think I've given up on trying to find out. It caused huge performance drops in-game (lost 30-40 FPS) when I tested it, and it caused the rest of my system to lag as well.
Edit:
@Simes brings up some good points above.