NVENC Video Encoder has disappeared.

[Tizye]

Hey all.

Noticed a couple weeks ago that in my Output settings, my Video Encoder (GPU: NVIDIA GeForce RTX 5070 Ti) is only showing x264 Software encoding through my CPU (AMD Ryzen 9 5900X 12-Core.) instead of showing through my GPU (GPU: NVIDIA GeForce RTX 5070 Ti.)

I was still on Windows 10, so figured that was my cue to finally upgrade. On Windows 11 now, still not showing up. My OBS build was outdated, so figured maybe I should update that as well. Now I'm on OBS 32.1.1, and it's still not showing up. I figured, hey, I've updated my GPU drivers recently, but let's give em another update just in case. Now my GeForce Game Ready Driver is up-to-date with Version 595.97, and it's still not showing up.

I checked the logs, and starting at Line 81 it seems to call the GPU, and then says
21:59:58.742: D3D11 GPU priority setup failed (not admin?)
That's weird, so I restart it, this time Run as Administrator. Now it says
22:10:14.530: D3D11 GPU priority setup success
which sounds great, except it's still not showing up. I also find on Line 118 - 120 saying
22:10:14.877: [NVENC] Test process failed: session_limit
22:10:14.877: NVENC not supported
22:10:14.877: Failed to initialize module 'obs-nvenc.dll'
Which sounds like it's just a missing file, that maybe didn't install correctly with OBS or my NVIDIA GeForce drivers?

I was getting this same issue on my main OBS build, my separate Portable builds, and now on a clean install completely. I know it's not an incompatibility with my GPU, I've used this GPU for streaming dozens of times with no issue, and this only appeared in the past month or so. I'm at a loss.

 

[Warchamp7]

I've had a couple other friends encounter this recently too, and rolling back their driver is what fixed it.

@Tizye did you ever figure this out?

 

[Tizye]

I've had a couple other friends encounter this recently too, and rolling back their driver is what fixed it.

@Tizye did you ever figure this out?

Nope, never figured it out. I've just been encoding through my CPU since then. Luckily this specific rig hasn't been used for any high intensity shows so I've been able to get away with it, but it is definitely limiting.

Rolling back their video drivers? I'd have to roll them back pretty far at this point, but might be worth a try.

 

[rockbottom]

It appears you have too many concurrent NVENC sessions running on your system.

21:29:19.637: [NVENC] Test process failed: session_limit
21:29:19.638: NVENC not supported

22:10:14.877: [NVENC] Test process failed: session_limit
22:10:14.877: NVENC not supported

 

[Tizye]

It appears you have too many concurrent NVENC sessions running on your system.

21:29:19.637: [NVENC] Test process failed: session_limit
21:29:19.638: NVENC not supported

22:10:14.877: [NVENC] Test process failed: session_limit
22:10:14.877: NVENC not supported

I struggle to think what could be causing session limit. It does this immediately upon restart of my computer.

Even in the past when I would run multiple OBS instances on my PC, it would let me set NVENC on multiple of them, but would only let me begin encoding proper on two of them at max. Now it's not even letting me select, the option simply doesn't exist.

 

[rockbottom]

If there's no other apps using NVENC, my guesses in order would be: driver, driver conflict or hardware failure are on the table. Fire up DDU & see if there's a conflict/rollback to the last driver that worked.