In your logfile I noticed you have Windows GameDVR enabled. This may be a cause of the performance issue as even when you are not using it, just having it enabled in Windows settings it will always hook into the game process and commit resources (Similar to what OBS does when OBS is open, though more aggressively as it also buffers for replay recording)
Using Monitor Capture in the same scene as Game/Window Capture can have a detrimental effect on performance as Monitor Capture is essentially 'always on':
Game capture grabs the frame directly from the graphics card for OBS to encode before sending it back to the graphics card for final rendering of the frame.
Monitor capture has the graphics card write the frame to system RAM (Which is really slow to write/read from) before OBS grabs it to encode, then sends it back to the graphics card to render for final output.
If you want to use Monitor Capture, you should create a scene for it by itself without any other capture methods included in that scene.
Your colour format should be 709 and Partial. 601 colour format should only be used for resolutions below 720p.
You can try Full colour range, though it may result in heavily saturated colours and very heavy contrast.
For your Video settings,
Base Resolution: 2560x1440
Canvas Resolution: 2560x1440
Resolution Filter: Lanczos Filtering
Frame Rate: 60
Your recording settings are not the best for recording.
In short, you should only use CBR for streaming, CRF (If using x264 CPU encoder) or CQP (h264 GPU encoder) is preferred as it will allow freedom of bitrate to retain quality output for the recording instead of being limited in average bitrate for the recording duration (This can lead to much larger file sizes at very low CRF/CQP values - <10)
Try the below settings, hopefully your RX580 can do 2560x1440 at 60fps output:
Preset: Indistinguishable (Very large file size) or Lossless (Lossless will lead to
extremely large file size)
Quality Preset: Quality
Keyframe Interval: 2
If you check your logfile after stopping the recording for the 'dropped frames due to rendering lag' line and it is there, then try lowering the quality mode to Balanced.
Can you do a few minutes' worth of recording (Doing what you would normally do in-game) then stop the recording session, then upload a current logfile please so I or someone else can further assist in troubleshooting:
https://obsproject.com/forum/threads/please-post-a-log-with-your-issue-heres-how.23074/
