Question / Help Cannot open the NVENC codec on Gentoo

[ashleydixon]

I've just installed O.B.S., and when attempting to record with NVENC (hardware encoder), I receive the following rather unfriendly message:

[h264_nvenc @ 0x55b47feca020] dl_fn->cuda_dl->cuInit(0) failed -> CUDA_ERROR_UNKNOWN: unknown error
warning: [NVENC encoder: 'streaming_h264'] Failed to open NVENC codec: Unknown error occurred

The most baffling thing is that when O.B.S. starts, it reports that NVENC has been detected:

info: NVENC supported

I'm very inexperienced with O.B.S. and ffmpeg in general, so please ask if any more information is required for the diagnostics. From internet-searching, it seems as though no-one has experienced this exact issue before.

 

[Harold]

Need the whole log file please.

 

[alex_crow]

Hey there,

From a terminal session, can you verify that ffmpeg does indeed have the nvenc feature compiled in it? To verify, run:

ffmpeg -codecs | grep nvenc

If that returns anything regarding h264_nvenc or nvenc_h264, ffmpeg is probably not the issue. I recently had trouble with the latest flatpak version of OBS (24.0.3-modified) detecting nvenc too. I verified that ffmpeg had the nvenc codec compiled, and still couldn't figure out what the issue was. I found a forum post somewhere stating that 24.0.3 had issues detecting nvenc for some reason, even though it was compiled in ffmpeg. The solution that I've found was to downgrade to the previous release (24.0.2). After doing so, nvenc was detected correctly and I'm now able to use the hardware encoder in OBS.

If you're using the flatpak release of OBS like I am, see the documentation at the following link for information on how to downgrade versions:

https://docs.flatpak.org/en/latest/tips-and-tricks.html

Hope this helps!

 

[sethgoldin]

Hey there,

From a terminal session, can you verify that ffmpeg does indeed have the nvenc feature compiled in it? To verify, run:

ffmpeg -codecs | grep nvenc

If that returns anything regarding h264_nvenc or nvenc_h264, ffmpeg is probably not the issue. I recently had trouble with the latest flatpak version of OBS (24.0.3-modified) detecting nvenc too. I verified that ffmpeg had the nvenc codec compiled, and still couldn't figure out what the issue was. I found a forum post somewhere stating that 24.0.3 had issues detecting nvenc for some reason, even though it was compiled in ffmpeg. The solution that I've found was to downgrade to the previous release (24.0.2). After doing so, nvenc was detected correctly and I'm now able to use the hardware encoder in OBS.

If you're using the flatpak release of OBS like I am, see the documentation at the following link for information on how to downgrade versions:

https://docs.flatpak.org/en/latest/tips-and-tricks.html

Hope this helps!

This was quite helpful, alex_crow, thank you! I was having this very issue with the Flatpak build of 24.0.2, on CentOS 8.0 with the NVIDIA 440.44 driver, and with the information you provided, I went and made sure that ffmpeg was properly installed with the NVENC codecs. Even though the NVENC codecs were indeed installed within ffmpeg, it still wasn't working. I was about to downgrade as per the documentation to which you linked, but when I went to do so, I was warned that I didn't have org.gnome.Recipes installed. So I ran:

$ flatpak install flathub org.gnome.Recipes

I saw that there was something that got installed that pertained to my exact NVIDIA driver, 440.44. I figured that before downgrading, I might as well try again, and NVENC worked again! Whatever was included in org.gnome.Recipes was enough to get it all working, and I didn't even need to downgrade. Thank you!