Question / Help Nvenc giving terrible performance vs Shadowplay

[Purplekoolaidman]

Hey everyone, Im currently getting terrrible performance using the Nvenc encoder. I have the lastest beta drivers. GPU is GTX670. I tried using the 64bit dll files for the nvenc encoding and it still just does not run even close to as well as shadowplay. The quality isnt even close to as good as using Shadowplay or normal OBS setting. When using NVenc I lose around 20 to 25 frames on my gameplay which for some games can make it unplaybale. Im only using nvenc when making local recordings. The recording details are in the text log but its at 1080p60p on High Quality. Maybe I need to use some different settings or something but I have tried everything. The text log is in the attachment. I love OBS and been using it for more than a year but I love shadowplays performance while playing games. Thanks for the help

 

[Floatingthru]

You should make a clean scene using only one Game Capture to see if that changes anything, because you are using 30? Game Captures for whatever reason. That log file isn't very long either so it's hard to see what impact OBS is having. You should also turn on CBR and CFR if you want to match the quality of Shadowplay at those settings.

 

[Purplekoolaidman]

Thank you. Ill try cbr and cfr. I turned those off cause I thought they were hindering the performance. Ill also make a new fresh scene. Could I just uncheck all the scenes? Or should I delete them?

 

[Floatingthru]

You really only need one Game Capture, use the hotkey method to acquire the game. Go ahead and delete them.

 

[Purplekoolaidman]

Wait there's a wait to hot key to pick a new game? Can you link where or how to do this

 

[Floatingthru]

Double click on your Game Capture source then select Use Hotkey and enter whatever it is that you want to use.

 

[Purplekoolaidman]

My mind. My mind is fucking blown. I feel so stupid

 

[Purplekoolaidman]

Nvm its working ill post back with the log files.

 

[Purplekoolaidman]

Its now recording properly with CBR and CFR on but the problem with performance is still there. Im going to attach the new log to the original post. I ran the batman arkham city benchmark with nvenc, without, and with shadowplay
here are the results.
Nvent - min 28 max 78 ave 51
Shadowplay min 34 max 118 avg 75
None - min 42 max 120 avg 77
Shadow play settings are on high - 1080p 60fps
Quite a large performance difference. Im not sure what the cause of this is.

 

[Floatingthru]

I personally don't see that much difference of performance between the two, but Shadowplay is using the most optimized method of capture albeit with limitations e.g. full screen only. OBS capture performance should be up there as well, though. I suppose do to the fact that OBS will use more of the GPU could have that extra impact. Have you tried different games? I'm wondering if its just Batman or if the difference is similar in all games.

 

[Purplekoolaidman]

Its actually worse in more modern games which is the frustrating part. Ive tried Crysis 2, and its poop. In rust (yes i know its alpha) I lose nearly 35 average fps. Ill leave the recording off and get an ambient fps reading by keeping the game idle. Ill then start the recording and like usual theres a drop in frames and it comes back up. But its usually around 20 frames lower. The thing I dont get is I get better performance if i just use the normal OBS local recording. Im going to try reinstalling my beta drivers but i doubt thats causing anything since shadow play runs fine.

 

[Floatingthru]

Hmm that is strange that normal encoding gives you more performance. NVENC uses a separate chip on the GPU to handle all the encoding. I don't really have anymore ideas. We have pretty similar rigs, and it was never that drastic for me.

 

[Krazy]

Hm, I'll try Arkham City tomorrow myself and see if I can reproduce. Which specific section were you at in the game?

 

[Purplekoolaidman]

I used the benchmark tool. I have the same experience with metro last light too. Ill run their benchmark and post the findings. After messing around I noticed a reasonable improvement when making sure that shadow play recording was set to manual only. Another thing I should bring up is that im not sure what I did but OBS has some pretty high cpu usage when recording at 60fps. So what i mentioned about local recording vs nvenc is wrong. Nvenc does run better but still runs reasonably worse than shadowplay. Enough worse than it can effect gameplay. I would really love to get nvenc to work like OBS mainly because of noisegate on OBS

 

[Krazy]

Have you tried lowering the bitrate to like, 25000? I seem to remember talk about NVENC choking on 1080p60fps if the bitrate is too high.

 

[Purplekoolaidman]

Alright so I tried lowing the bitrate to 25k and its just nearly as good of quality. The footage seems to be dropping frames and looks like its playing at around 30 fps but the entire time the footage is record at 60fps. Ive recorded two clips and the shadowplays quality is ridiculously better and smoother. I have done blind tests for this with friends and every picks out the OBSnvenc version as the one with worse quality. Im almost possible that Shadow play on high records at 50mbit. It must be a problem with OBS using that high of bit rate or something. When the bit rate is 25k it just doesnt look as good. I get very minor pixelation. Its hardly noticeable but what is noticeable is the frame rate. Its losing some and obs isnt showing it is. Shadowplay literally looks like im playing the game. Ill probably have to wait for another patch for this issue to come up again for OBS but like most do, ill stick with obs for streaming. Thanks for the help and any other recommendations are always appreciated.

 

[alpinlol]

what about just setting the bitrate to 1000 or whatever and the buffer to 0? what you would usually do with the normal local record settings.

also have you tried using the different nvenc presets?

 

[Purplekoolaidman]

bitrate at 1000 is completely pixelated. The bitrate should seriously be 50mbit. Its what shadow play uses at 1080p60fps. Ill try a few other presets and see what happens.

 

[Purplekoolaidman]

The only one that seems to perform anything close to shadow play is Nvencs preset High Qual Low Lat. The problem with that is it doesnt encode fast enough and i keep getting an error on OBS saying not encoding fast enough. Sometimes its as low as 15 fps but skips around even more so. Im not sure if that setting is better on performance or if its because of the lack of speed with the encoding with that preset. Recording at 30fps 1080 helps the performance during my benchmarks but it still does not run as well as shadowplay at 1080 60fps. I find that extraordinary given thats half the bit rate being used. It may also be the gamehook plugin in OBS but Idk. Ill just stick to shadowplay for my local recordings. The performance and quality is just too good to pass up.

 

[alpinlol]

thats why i said 1000 bitrate (which doesnt even matter) and buffer of 0 which allows the encoder to use as much bitrate as needed

and then i would just use the high quality or the high performance preset to record 1080p60