How can I achieve smoothness in my videos?

[4d4m]

I am looking into how can I make my videos look smooth in 1080p 60fps. I'm new to all this and don't really know how to set each settings. I tried watching some YouTube videos but my footage was kinda pixelated.
PC specs:
AMD Ryzen 5 3600
RTX 580
Here are some pictures of what my settings look like now.

 

[BardiBard]

Use:
HEVC
high bitrate 20.000 up to 150.000
Lanczos
colour format/space/range: RGB / sRGB and Full

 

[TryHD]

Use:
HEVC
high bitrate 20.000 up to 150.000
Lanczos
colour format/space/range: RGB / sRGB and Full

You go for CBR but than full RGB?

never use a fixed bitrate for recording, instead use quantizer mode based encoding like CRF or CQP
Colorformat: nv12
color space: 709
color range: partial

 

[BardiBard]

You go for CBR but than full RGB?

never use a fixed bitrate for recording, instead use quantizer mode based encoding like CRF or CQP
Colorformat: nv12
color space: 709
color range: partial

nv12 partial = streaming

rgb full = recording

 

[4d4m]

Okay, will try, thanks!

 

[TryHD]

nv12 partial = streaming

rgb full = recording

First
CBR for streaming

CQP or CRF for recording
if you record with CBR it doesn't matter what color range you use it will always turnout bad.
Beside that most encoders don't have a efficient way to record RGB and will drop frames.

 

[BardiBard]

First
CBR for streaming

CQP or CRF for recording
if you record with CBR it doesn't matter what color range you use it will always turnout bad.
Beside that most encoders don't have a efficient way to record RGB and will drop frames.

I don't know about nVidia, but this is what is possible using and AMD card (RX 550 ~ GT 1030)

Spoiler

The footage recorded with these settings has amazing quality and thanks to RGB/Full all the colours are accurate as well. (the only "bad" thing being the AAC audio codec even with a 320 bitrate, and I have no idea how to set exactly all of this up using the "ffmpeg" option which allows different audio codecs)
The only thing that needs to be adjusted from game to game are the I-Frame and P-Frame numbers, games with lots of details (foliage, etc.) like in Fallout 4/3 will require it to go up to 28 to not have the encoder overload, which will result in slightly worse image quality where there is lots of details.

The final file size is a bit larger but who cares about that, when recording footage to edit and then upload to YouTube you'd want it to have near lossless quality anyway as well as the final rendered video will be much smaller anyway.

And judging by how many people praise nVidias encoder, I'm certian that it can do exactly the same.
I just don't happen to own a nVidia card to help with properly setting it up to record gameplay footage.

 

[TryHD]

The final file size is a bit larger but who cares about that, when recording footage to edit and then upload to YouTube you'd want it to have near lossless quality anyway as well as the final rendered video will be much smaller anyway.

Dude that is why you don't use CBR for recording. You waste bitrate in low motion scenes and than loose quality in high motion ones.