NVENC Performance Improvements (Release Candidate)

[dodgepong]

There's a new build available that fixes a crash when stopping the encoder, as well as frame jitter when experiencing high GPU usage: https://cdn-fastly.obsproject.com/temp/nvenc-test-8.zip

EDIT: Apparently this does not fix the crash issue yet. However, if you are not experiencing the crash, please test to make sure the issue with frame rate jitter at moments of high GPU usage is resolved.

 

[Animetic]

Largely, yes. The main issue we're trying to work with is that this build can have these stuttering issues if GPU usage is maxed out, or nearly maxed out. Perhaps a better way to fix the problem is to limit FPS in the game you are playing.

I tried the eight buffers build and it still stutters. I tried limiting my FPS in-game to 200 (down from 300) and it still stutters (no difference). My GPU usage is at about 20% when playing games. (2x1080 Ti in SLI)

It does seem to be game-related because the stuttering happens with Overwatch (and my viewers say "more often in Hero select screen"), but when I streamed Beat Saber for 5 hours it didn't stutter once.

 

[Animetic]

There's a new build available that fixes a crash when stopping the encoder, as well as frame jitter when experiencing high GPU usage: https://obsproject.com/temp/nvenc-test-8.zip

This link does not work for me.

 

[dodgepong]

This link does not work for me.

Try now, updated the link.

 

[Erick Smith]

I use two GTX 760 SLi with OBS on a dedicated PC. Would there be any specific GPU limitations with these cards? I would love to test this.

 

[AdamLop413]

Using this new encoder causes issues when game capturing Star Wars: Battlefront 2 (EA) specifically. I have replaced the DLL and other file with the digitally signed variant but this didn't seem to do anything. First let me give you specifications:

I have a 1440p Monitor, running at 65 Hz by DP 1.2, x4 Links @ 2.7 Gbps each (In case it helps). The game is running at 1080p Full-screen, DX11. My Laptop monitor is running at 1080p 60 Hz, DP 1.2, x2 Links @ 1.6 Gbps.

GPU is a GTX 965m, 2 GB VRAM, Driver 417.35, GM204 variant (not GM206) overclocked to 1477 MHz on the cores with memory bumped up by 300 MHz. I do not have issues with my overclock, it is perfectly fine, and I have used OBS before.

OKAY, now the issues:
Game Capture is rather, strange. It took 3 tries to get it to finally not crash the game when hitting the record hotkey in-game. When I got it to work, OBS was telling me my Encoder was overloaded, the frames being captured were probably 7 frames per minute but was sporadic. Monitoring the Video Engine usage, it wasn't even getting passed 2%. Changing the capture to monitor capture, Video Engine usage was at a whopping 88%, which I've never seen any application take advantage of. This utilization quickly tanked seconds later which hovered at ~20%. Frames were still being dropped, but it was about 3 times better than game capture.

I was encoding at 5650 Kbps, with preset at "Performance" saving as an MP4. My game didn't suffer any performance loss at all while recording, but I believe it has to do with the encoder not even capturing frames correctly.

However, when recording on my desktop using Monitor capture, the encoder is not overloaded and records perfectly fine. Perhaps it has to do with available VRAM? It's almost 100% utilized by the game, with GPU usage at 100% and memory controller usage at about 66% - 88%. I really love that NVIDIA is partnering with you guys, the ability for the GPU to store the frames directly onto the almost-instant VRAM instead of the terrible delay of System RAM is just ludicrous and awesome!

I hope my findings help, I have yet to try the new beta release and post an update.

 

[mendosa]

I'm having what appears to be game-specific crashing with video capture when live streaming. I'm using the original beta version (first post).

Game: Divinity: Original Sin 2
System: i7-7700k, 16GB DDR4 RAM, EVGA GTX 1080 8GB (latest drivers), 1440p GSYNC monitor, 100hz, frames limited to 100hz via 3rd party software.

My settings match this: https://obsproject.com/forum/resources/nvidia-nvenc-guide.740/

Issue: I would start recording via Game Capture (targeting game, not screen/fullsize) and about 10 seconds into the recording it would just lock up. My video capture device would continue to record but the game was frozen so I literally had a stream of a frozen game play screen and an active, moving video capture screen. I resolved this by deleting the video capture source and reseating my microphone cable on the PC. Once that was done my video capture turned black. Had to reseat cam, delete/re-add source and restart OBS.

Once this was all complete I had a fully working stream. I said this was game-specific because I streamed other games (The Story About My Uncle) and there were no issues.

 

[psifrog]

Issues with the Nvidia drivers (417.35) and fps dropping?

So.. after trying the test build the first time I was impressed. I decided to update my nvidia drivers thinking it would make things even smoother. #BrokeIt

Thought it was the game capture.. no good, 22.0.2 and slobs also having the same issues. Even went as far as to reformat my pc (was due anyways) and see if it might be disk issue.. nope.. Seems like the game is taking priority of the gpu usage.

I noticed when the game is open and obs is recording/streaming (with nvenc), my gpu usage cranks up to 100% and the fps in obs drops significantly.

I had to go back to x264 and dial back to 30fps. With x264 the gpu usage sits around 60-80% still with cpu around 50%.

Will be testing previous driver builds but I am pretty sure there is something with the new nvidia driver that is causing this issue.

(fyi) game mode is turned off.. game fps capped at 60fps.. single pc setup.. gtx970, 3770k, 16gb ram

As of right now.. I can barely stream Overwatch at 720p 30fps with mid-hi in game settings.

If there's something I'm missing please let me know! I'm going crazy here!

 

[John Chambers]

Hello Folks,

I feel like I tried everything, but same result, garbage quality video streaming...

So I want to do the job with this only one computer. I7 8700K @4.8Ghz 16 Go DDR4 @4500 Mhz RTX 2080 TI Samsung EVO 960 512 Go Nvme m2 (gaming app SSD) Samsung PRO 840 256 Go (recording SSD) MSI Optix MPG27CQ (1440p/144hz Monitor)
My Internet connection
All OBS Settings
Details from video I downloaded from my VOD library on Twitch after a Streaming Session.

New Nvenc 6000 bitrate 720p/60fps I show from the start on video the ScoreBoard, you can see my ping is one of the lowest of the server. As long as I stay static, the quality and color are great.... until I move and then it's a pixels fest... especially in dark area... I don't know why...
Come on... this is 720p... 6K bitrate... with new Nvenc, and Turing Card... I've already seen better quality on random streaming footage even in 900p/60fps... so... I don't understand.

New Nvenc 6000 bitrate 1080p/60fps Every motions (zoom, steps, look around) completely destroy the quality of the picture...
Meanwhile I'm able to see some footages here and there on Reddit with new Nvenc on PUBG with beautiful quality in 1080p/60fps...

Any advice please ?

Thanks in advance.

 

[CriticalChocolate]

Hello Folks,

I feel like I tried everything, but same result, garbage quality video streaming...

So I want to do the job with this only one computer. I7 8700K @4.8Ghz 16 Go DDR4 @4500 Mhz RTX 2080 TI Samsung EVO 960 512 Go Nvme m2 (gaming app SSD) Samsung PRO 840 256 Go (recording SSD) MSI Optix MPG27CQ (1440p/144hz Monitor)
My Internet connection
All OBS Settings
Details from video I downloaded from my VOD library on Twitch after a Streaming Session.

New Nvenc 6000 bitrate 720p/60fps I show from the start on video the ScoreBoard, you can see my ping is one of the lowest of the server. As long as I stay static, the quality and color are great.... until I move and then it's a pixels fest... especially in dark area... I don't know why...
Come on... this is 720p... 6K bitrate... with new Nvenc, and Turing Card... I've already seen better quality on random streaming footage even in 900p/60fps... so... I don't understand.

New Nvenc 6000 bitrate 1080p/60fps Every motions (zoom, steps, look around) completely destroy the quality of the picture...
Meanwhile I'm able to see some footages here and there on Reddit with new Nvenc on PUBG with beautiful quality in 1080p/60fps...

Any advice please ?

Thanks in advance.

the 720p seems alright, the 1080p footage however might need a little more bitrate, try 7000-8000 kbps and see if that is any better.

 

[Animetic]

Try now, updated the link.

This build fixes the stutterings for me. 8 hour Overwatch stream no stuttering. Very nice.

 

[AdamLop413]

the 720p seems alright, the 1080p footage however might need a little more bitrate, try 7000-8000 kbps and see if that is any better.

Yes, but they are dissappointed because that is not what this new update promises. A 1080p60 video with the new Turing NVENC Encoder would have the same, if not better, quality of a 8 Mbps stream at 6.5 Mbps all while increasing performance.

Also, I have figured out that the reason stuttering occurs for me is because in fullscreen, since my total VRAM is used up, the encoder cannot efficiently use it, which is rather strange. A 5-15 Mbps stream is not that much in terms of data. A 50 MB or even 60 MB buffer would be perfectly fine, it seems OBS uses 175 MB of GPU VRAM as a buffer. I think it would be better if OBS would intelligently select a buffer size depending on the GPU's available VRAM, or even have user-customizable variables or options.

Test 8 build seems to be working crazy, and I not only have received a massive performance increase (using Window Capture), but video quality is actually better than the regular encoder.

 

[dodgepong]

Here's another new build: https://cdn-fastly.obsproject.com/temp/nvenc-test-9.zip

This one fixes a crash caused by running multiple encoding sessions with the NVENC encoder at the same time.

 

[Fingers]

I upgraded to a RTX 2080 last week and was eager to test this. I'm running into an issue though. When the issue occurs my stream drops to single digit frames and obs reports the encoder is overloaded while in game.

The issue has odd behaviors. I ran into this first in insurgency sandstorm and noticed when I alt tabbed to my second screen obs is in the issue stopped and the stream looked fine. It resumed once I went back into the game and had mouse control back.

The issue occurred in several games with 2 exceptions, Blackops 4 has no issue with the encoder and the stream operates properly. The division has the problem, but it goes away if I open the map and have it active on the screen. Gpu usage is showing 80-97 but is the same when it works in blackops.

I've tried about every setting/option I can think of with no change to behavior.

Current hardware, 9700k, 16gig 3200 ddr4, evga rtx2080 black. Using 3 monitors, main is 1440p144hz with gsync enabled. My usual obs settings are canvas at 1440p resized to 720 in the video tab, 6000 rate and the rest of the output settings defaulted for nvenc new.

 

[Bleuzen]

Will this also come to Linux?

 

[dodgepong]

I upgraded to a RTX 2080 last week and was eager to test this. I'm running into an issue though. When the issue occurs my stream drops to single digit frames and obs reports the encoder is overloaded while in game.

The issue has odd behaviors. I ran into this first in insurgency sandstorm and noticed when I alt tabbed to my second screen obs is in the issue stopped and the stream looked fine. It resumed once I went back into the game and had mouse control back.

The issue occurred in several games with 2 exceptions, Blackops 4 has no issue with the encoder and the stream operates properly. The division has the problem, but it goes away if I open the map and have it active on the screen. Gpu usage is showing 80-97 but is the same when it works in blackops.

I've tried about every setting/option I can think of with no change to behavior.

Current hardware, 9700k, 16gig 3200 ddr4, evga rtx2080 black. Using 3 monitors, main is 1440p144hz with gsync enabled. My usual obs settings are canvas at 1440p resized to 720 in the video tab, 6000 rate and the rest of the output settings defaulted for nvenc new.

Can you post a log? This is with build 9, correct?

Will this also come to Linux?

Unfortunately not, it seems to be a Windows-only thing. That said, the next release of OBS will come with VAAPI support, at least.

 

[Bleuzen]

Unfortunately not, it seems to be a Windows-only thing. That said, the next release of OBS will come with VAAPI support, at least.

Nah, I did hope it will :/

Isn't VAAPI only for Intel / AMD? Because for Nvidia there is already the nvenc codec, so there should be no changes with the VAAPI with Nvidia cards, correct?

Does OBS use this SDK to achieve that?:
https://developer.nvidia.com/nvidia-video-codec-sdk
If so, it should be possible to port it to Linux, because the SDK supports Linux.

 

[Overflow]

Any advice for using this build with streamdeck? Maybe the problem is the build test isnt on default obs folder in my PC? I have the folder with the beta on my desktop.

 

[dodgepong]

Nah, I did hope it will :/

Isn't VAAPI only for Intel / AMD? Because for Nvidia there is already the nvenc codec, so there should be no changes with the VAAPI with Nvidia cards, correct?

Does OBS use this SDK to achieve that?:
https://developer.nvidia.com/nvidia-video-codec-sdk
If so, it should be possible to port it to Linux, because the SDK supports Linux.

That is correct, yes...It's a consolation prize if you have an Intel CPU, I suppose.

 

[Vraelon]

Thanks a bunch for this! You saved me $400 on an Elgato 4K60 Pro. I can actually record 4K60P now! I was dropping over 30%+ frames due to preview/encoding lag and had 36ms OBS frame render times with my in game FPS cut in half. Now I have 0.2ms frame render times and 0 dropped frames. However I noticed that the CQ only goes down to 14 now, and for some reason at 14 it looks VERY low quality and is only ~40mbps is there a way to lower it more? I am getting VERY noticeable color banding and blocking. I usually record at CQP 8 and transcode to CRF 16 x265 after.