You have Windows Game DVR enabled in your Windows OS settings, that is one potential conflict to resolve.
x264 encoder also uses your GPU to render the frame output, the problem is not your CPU:
22:14:13.763: Output 'simple_file_output': Number of lagged frames due to rendering lag/stalls: 43197 (27.6%)
22:14:13.764: Video stopped, number of skipped frames due to encoding lag: 16/156233 (0.0%)
The above lines in your uploaded logfile (The rendering lag) indicate you are pushing your GPU well past its limits, this is what is causing your performance issues.
I would say the encoding lag is due to your GPU as well, as it is falling so far behind that it is likely falling out of sync due to your CPU sitting there doing nothing whilst waiting for your GPU.
Some troubleshooting tips:
- If you don't already, try capping your frame rate to your monitor refresh rate, or 60fps (Using V-Sync can/will induce input lag, I prefer to use a 3rd party software tool such as MSI Afterburner or EVGA Precision. They also give you screen overlay functionality which is handy for monitoring temperatures. If you get screen tearing issues, then try a value of 61-63fps) Capping your fps will help reduce the load on your GPU, when your fps is not capped it pushes your GPU as hard as it can go, the only time when you won't have an issue is if you are CPU bottle-necked (Your GPU is not working at full load due to waiting for your CPU to finish its task)
- Reduce Graphic settings in-game, start with most taxing on GPU settings, such as Anti-Aliasing, Shadows, Lighting, Distance Objects, etc.
With your Ryzen 1700 processor, you should be able to record with the below settings (These will give you better quality than NVENC ever will):
Video:
Base Resolution: 1920x1080
Scaled Resolution: 1920x1080
Downscale Filter: Lanzcos
FPS Value: 60
Output:
Encoder: x264
Rate Control: CRF
CRF: 10-23 (Lower number = higher quality, higher file size - Under 15 difference in quality barely noticeable, file size can and will be huge)
Keyframe Interval: Auto
Preset: Veryfast
Profile: High
Tune: None (This is personal taste, film can be good for retaining detail in fast motion footage, such as FPS games. Tune profiles will add more overhead to your encoder setup)
