NVENC file recording encoding of high quality 4K fails on RTX 3060

G

goofyseeker3

Member
OBS claims encoder becomes overloaded when outputting 4K h.264 video file at high quality medium file size setting.
stream quality at 4K (6000kbit/s) and 4K to 1080p works fine. the OBS machine is encoding the video from a separate Xbox/PC.
when recording outputting 4K, the preview screen video runs very slow. I thought that RTX 3060 or any RTX 3000+ could handle
all kinds of 4K recording compression. so whats wrong or is RTX 3060 NVENC just slower. or does NVENC chip have a performance cap.
even recording windows desktop without any fast moving fps stuff, will get the "encoder is overloaded". what's wrong?

edit: windows task manager shows 97% video encode usage. are any other video encoder solutions (RTX or other) any better.
I think the high quality, medium file size recording setting will overload NVENC chip by default.
50 Mbps stream quality recordings should work but they only record at stream bit rate cap.
 
Last edited:
G

goofyseeker3

Member
when encoding using hardware (amd, h.264) using integrated gpu, the recording quality drops to 600kbps for high quality and medium file size. and the output quality is terrible. dont know what affects that, obs settings or amd gpu internal workings.
 
koala

koala

Active Member
Nvenc on any RTX GPU is able to encode 4k 60 fps content fine. If it doesn't work for you, you set unusual/non default settings. However, you need to post a log for us to see what is happening.
 
G

goofyseeker3

Member
according to nvidia nvenc performance table, it should be mostly fine and same for 3000 series and close to 4000 series performance. but I had to drop the quality to P4 from P7 to get the encoder to be below 100%. now its at 70%. note: default streaming to twitch at 6000kbps and similar will work. and now it works for recording at 35000kbps CBR at P4 quality. none of the not-same-as-stream defaults worked for me.

I have been able to encode 4K content (to 1080p) before just fine, now when outputting 4K quality content the problems started.
 
Last edited:
koala

koala

Active Member
Encoding 4k instead of 1080p requires 4 times the encoder resources (it's 4 times the amount of pixels), encoding with higher presets also requires significantly more resources, so find a balance with the parameters. There's nothing wrong or broken at your end. You just cannot expect infinite performance from nvenc.
 
G

goofyseeker3

Member
I expected that nvenc would just fully encode 4k in all situations no questions asked. but that was clearly not the case.
4K encoding to 1080p works in all cases tho.
 
G

goofyseeker3

Member
I wonder why nvidia and/or amd and intel, have not implement gpu encoding/decoding (as encoding/decoding chip mode), in addition to video encoder/decoder chips. or at least gpu encoding instead of nvenc/chip encoding should be an option in OBS. yep when you have the gpu doing nothing else than encoding. capturing/streaming from other source, like xbox or pc.
 
G

goofyseeker3

Member
Solution: If you have an AMD GPU, discrete or integrated, which is not being otherwise used, like in a streaming pc,
then you can use AMD HW h.264 (AVC) at maximum quality 100Mbps CBR for 4K encoding.
Unlike Nvidia NVENC, it will not claim overloading of encoding, but shows high gpu utilization.

Why is the CBR bitrate capped to 100Mbps. it should be at least 200Mbps.
Also why the AMD HW encoder produces very small bitrate video files with poor quality
when using same CQP or file quality settings as for NVIDIA nvenc encoding.
 
You must log in or register to reply here.
Top