NVENC Performance Improvements (Release Candidate)

[Gianluca Leal]

The AMD encoder was made entirely by a contributor and it was his decision to make it like that. But all those settings there are confusing to users, which is why they are not shown by default. Sure, something like that could be done for NVENC, but I doubt it will be a high priority thing.

I commend the contributor for his thoroughness then!
This other news is saddening though. While you won't support the full-feature list of what's possible with NVENC, I would hope that you at least put a few more advanced options than what is currently available in the beta build. It feels somewhat barebones.

It's also sad to hear that you are unable to add the "Custom Flags" section, similar to that of the x264's field for NVENC due to API limitations.
Because of this, I would hope you'll support more advanced features on release, or at least in the very near future.

Feels low priority man :(

 

[Mike321]

Then use the "simple" mode.

But simple doesnt let you do multiple audio tracks :(

 

[fsy]

I've noticed a regular, 30 second repeating glitch that briefly overloads the encoder. The recording then carries on as normal.
This is only seen in the beta9 version I've been testing. The regular OBS doesn't have this 30 "beat"

CPU: Intel i7-5820K (6c, 12t)
GPU: nVidia GTX980

Gsync: Enabled
Game Bar: Off
Game DVR: Off
Game DVR Background Recording: Off


GPU 3D usage in task manager remains approx 60% at all times. Video Encode around 10%

The encoder overload occurs continually every 30 seconds, you can set your watch by it. It's not massively related to the amount of motion going on within the game, it just drops a couple of frames every 30 seconds.

Game is software limited to 60fps (in game limiter)

This does not happen with the existing version of OBS, no such problems there.
Is this simply a matter of my hardware not keeping up with the new-NVENC? or a deeper problem? I'm tempted to try with a RTX2080 but I don't have my hands on it yet.

21:56:50.745: [game-capture: 'WoW'] attempting to hook process: Wow.exe
21:56:50.761: [game-capture: 'WoW'] shared texture capture successful
21:56:55.567: [jim-nvenc] settings:
21:56:55.567:     rate_control: CBR
21:56:55.567:     bitrate:      6000
21:56:55.567:     cqp:          20
21:56:55.567:     keyint:       120
21:56:55.567:     preset:       mq
21:56:55.567:     profile:      high
21:56:55.567:     width:        1920
21:56:55.567:     height:       1080
21:56:55.567:     2-pass:       true
21:56:55.567:     b-frames:     2
21:56:55.567:     GPU:          0
21:56:55.567:
21:56:55.581: ---------------------------------
21:56:55.582: [FFmpeg aac encoder: 'Track1'] bitrate: 160, channels: 2, channel_layout: 3
21:56:55.582:
21:56:55.603: ==== Recording Start ===============================================
21:56:55.603: [ffmpeg muxer: 'adv_file_output'] Writing file 'F:/capture/2019-01-24 21-56-55.mp4'...
21:57:15.163: warning: Stream #0: not enough frames to estimate rate; consider increasing probesize
21:57:45.164: warning: Stream #0: not enough frames to estimate rate; consider increasing probesize
21:58:15.164: warning: Stream #0: not enough frames to estimate rate; consider increasing probesize
21:58:45.162: warning: Stream #0: not enough frames to estimate rate; consider increasing probesize
21:59:15.165: warning: Stream #0: not enough frames to estimate rate; consider increasing probesize
21:59:45.170: warning: Stream #0: not enough frames to estimate rate; consider increasing probesize
22:00:15.162: warning: Stream #0: not enough frames to estimate rate; consider increasing probesize
22:00:20.807: [ffmpeg muxer: 'adv_file_output'] Output of file 'F:/capture/2019-01-24 21-56-55.mp4' stopped
22:00:20.807: Output 'adv_file_output': stopping
22:00:20.807: Output 'adv_file_output': Total frames output: 12297
22:00:20.807: Output 'adv_file_output': Total drawn frames: 12234 (12312 attempted)
22:00:20.807: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 78 (0.6%)
22:00:20.807: ==== Recording Stop ================================================
22:00:20.808: Video stopped, number of skipped frames due to encoding lag: 88/12309 (0.7%)

 

[TheRustyTigger]

is there a way to force support with streamdeck? I've replaced the original obs install with this beta build but will be unable to use it much without the support

 

[MrDoomsday]

I have 15-20% lagged frames using NVENC(new) at 100% gpu load, but x264 and nvenc (old) doesnt have this problem at 100% gpu load.
I have lagged frames even with perfomance settings and every new feature checked off.
Old nvenc have new options too, but i dont think they work.
Did i miss something installing beta version? I just replaced files over stable version.
My hardware is AMD Ryzen 1700 3.6,16gb ram and gtx 1063 OC
Here's my Log file 1 streaming with new NVENC, then set NVENC with same settings as before and no more lag\stall frames (GPU utilization were always 100% throught testing)

 

[lolmarkked]

Reporting that ive had immensely better performance using: test-9. 7700k/1080ti. In game performance is also noticeably smoother and less stuttery by large margins. I mainly play PUBG (all low settings). 1440p monitor. OBS: Canvas 1440p. Scaled resolution : 720p. Lanczos. 48fps..6000 bitrate. Quality. high. Psycho Visual Tuning checked. GPU 0. Bframes 2. One thing I noticed, I will get spikes of skipped frames + rendering lag when near the Blue circle or in a Blue circle or when in and around Smoke grenades. I will have to monitor GPU Load during these events spike into the 80-90+% usage.

FPS in game set to monitor refresh rate. 120hz.

Finding: Playing PUBG in Fullscreen (windowed) degraded OBS fps and rendering lag. Setting PUBG to only: Fullscreen - helped immensely.

Further testing:
I have found that reducing the the Preset helps with the amount of Lag spikes found via Stats. My goal was to get less than 1% throughout an entire game session.

I found that, as reported in earlier posts in this thread, if GPU utilization reaches about max, for me between 80-90%, rendering and encoding lag will increase.

 

[ATOJAR]

Do i just copy these three folders into my Streamlabs OBS folder within Program Files to install this version?

 

[Overflow]

Do i just copy these three folders into my Streamlabs OBS folder within Program Files to install this version?

This is for OBS Studio not Streamlabs OBS

 

[ATOJAR]

This is for OBS Studio not Streamlabs OBS

Ahh i see, thanks.

 

[fsy]

I did some more testing on my weird 30 second glitch.

this happens with or without a game running. Even with just me, a black . empty scene and my web camera on the screen i still get an encoder overload warning every 30 seconds.

It seems independent of the record / stream start time, ie it could occur at 15s, 45s, 75s, etc or 2s, 32s, 62s etc
but always every 30 seconds.

I note that this time in task manager "System" is briefly using 1-2% of the GPU, i'm guessing that is interrupting the encoder. I have no idea why System needs the GPU every 30 seconds. but it's messing with the new encoder.

the old OBS doesn't have this problem

Hope what i've described sheds some light on what it could be. As aside form this glitch, the beta9 is running pretty well.

 

[DrHawkeye]

Hello all,

Ive been having massive issues with OBS fps lately. Certain non intensive GPU games run solid at 720p60fps but games like CoD or the new Anthem demo absolutely destroy my OBS FPS. I have tried many different variations and cannot seem to figure out what the issue is. I did try output scaling down to 720p and rescaling output as seen below but it doesn't seem to make a difference. I plummet to 15-20 fps at most during any high movement or action or anything that isn't the basic loading screen. I am able to limit the frames to 60ish in CoD to stabilize better but I shouldn't have to play a game with terrible fps just to stream at such a basic quality.

PC Specs:
8700k @ 4.8
16g 3000 RAM
RTX 2080 Founders
~11Mbps Avg Upload


I am not sure what to do anymore. Seems like no matter what I look up or change in settings nothing changes for the stream with the new Nvenc and the old and with x264. It is all the same. It's like my 2080 can't handle playing a game and encoding at the same time.

EDIT: My preset is at high quality, I also tried x264 again and it is stable 60fps but is using 15-30% cpu

 

[Jaybonaut]

No. The only way the performance improvement in this patch can even happen is if OBS is running on the same GPU as the encoder, and you also need to run OBS on the same GPU as the game you're running to game capture it. Generally speaking, it's not possible to use one GPU for encoding and one for gaming.

Well, technically it might be possible to program, probably, but would actually result in significant performance degradation, not improvement.

Curious: what about the NDI plugin on a 2 PC setup? I typically have the gaming PC do all the setup/overlays and send 1080p over as 720p to the 2nd PC which only overlays a viewer count but does the actual encoding and sent to Twitch. Settings on the gaming PC within OBS do not affect the output via NDI plugin at all supposedly, so quality settings don't do anything, and I leave it on x264 medium or whatever I want, but on the streaming PC I set it to NVENC blu-ray (is that the best settings in the current official version?)

 

[DIRTY CES]

Osiris,

I've attached the log. Now I have a i7 7700k and an 2080ti. My Xbox is connected to my elgato hd60 pro Internal card.

When I use the nvenc new my stream gets blurrier than when I was using x264. same bit rate 8,000.
I'm also recording at the same time using nvenc new same settings but with bitrate at 40,000. My upload speed to around 950 mbps.

Hope you can find an isssue. This was last nights stream on Black Ops 4. Every time I ran or moved it got blurry and also can't even make out my face. It got like that on the i7 7700k but not as bad

 

[fsy]

OK I have got to the bottom of my problem; i'm going to describe the root cause below as to aid any future troubleshooters.


The 30 second overload on my NVENC-new encoder was actually caused by an old Image Slide Show source with a 30 second refresh on it! I don't even use it anymore, but it was still there in OBS as I hadn't deleted it.

Even though I'd blocked the slideshow source as unseen in the one Scene where it was used; OBS was still dutifully loading a fresh high resolution .bmp file from the HDD into the scene every 30 seconds and that was hurting NVENC-new.

Now it's an open question as to why does this hurt NVENC-new and not old NVENC's encoding, but for now that problem has a work around. If you're having clockwork like issues like i had, perhaps you should delete a slideshow source.
(delete sources you don't use anymore)

 

[Bonezz]

I have 15-20% lagged frames using NVENC(new) at 100% gpu load, but x264 and nvenc (old) doesnt have this problem at 100% gpu load.
I have lagged frames even with perfomance settings and every new feature checked off.
Old nvenc have new options too, but i dont think they work.
Did i miss something installing beta version? I just replaced files over stable version.
My hardware is AMD Ryzen 1700 3.6,16gb ram and gtx 1063 OC
Here's my Log file 1 streaming with new NVENC, then set NVENC with same settings as before and no more lag\stall frames (GPU utilization were always 100% throught testing)

Yeah people in this thread are trying to push that nvenc simply will not work over 90% GPU usage yet if you use the old codec, any other version of OBS or streaming through shadowplay itself this issue literally doesn't exist.

 

[Osiris]

@Bonezz you seem to forget that the new nvenc works a bit differently then the old nvenc, so you can't really compare the two.
Comparing OBS to shadowplay is even more weird, shadowplay doesn't do scene compositing, so less GPU usage.

 

[LiaNdrY]

Hello all,

Ive been having massive issues with OBS fps lately. Certain non intensive GPU games run solid at 720p60fps but games like CoD or the new Anthem demo absolutely destroy my OBS FPS. I have tried many different variations and cannot seem to figure out what the issue is. I did try output scaling down to 720p and rescaling output as seen below but it doesn't seem to make a difference. I plummet to 15-20 fps at most during any high movement or action or anything that isn't the basic loading screen. I am able to limit the frames to 60ish in CoD to stabilize better but I shouldn't have to play a game with terrible fps just to stream at such a basic quality...

You need to close OBS, and delete the file "%AppData%\Roaming\obs-studio\basic\profiles\Untitled\basic.ini" or the entire folder "%AppData%\Roaming\obs-studio\", then run OBS and reapply configure for yourself.

 

[Osiris]

Hello all,

Ive been having massive issues with OBS fps lately. Certain non intensive GPU games run solid at 720p60fps but games like CoD or the new Anthem demo absolutely destroy my OBS FPS. I have tried many different variations and cannot seem to figure out what the issue is. I did try output scaling down to 720p and rescaling output as seen below but it doesn't seem to make a difference. I plummet to 15-20 fps at most during any high movement or action or anything that isn't the basic loading screen. I am able to limit the frames to 60ish in CoD to stabilize better but I shouldn't have to play a game with terrible fps just to stream at such a basic quality.

Unfortunately you will have to play the game at a reduced fps (or reduced quality), if you run the game at an fps where it's using almost all of the GPU's power, then there will be nothing left for OBS to do it's thing, resulting in the game and OBS fighting for resources. You can see that something like this is happening because there are lagged and skipped frames in your logfile.
There are also other causes that appear in your log:

1. Display capture and game capture in the same scene, this is usually not a good idea.
2. Running webcam at 1080p can also increase gpu usage a bit, usually it is not necessary to run it at 1080p if the cam is just a little box in the corner.

 

[DrHawkeye]

You need to close OBS, and delete the file "%AppData%\Roaming\obs-studio\basic\profiles\Untitled\basic.ini" or the entire folder "%AppData%\Roaming\obs-studio\", then run OBS and reapply configure for yourself.

Ok ill try, what does this do if I might ask?

Unfortunately you will have to play the game at a reduced fps (or reduced quality), if you run the game at an fps where it's using almost all of the GPU's power, then there will be nothing left for OBS to do it's thing, resulting in the game and OBS fighting for resources. You can see that something like this is happening because there are lagged and skipped frames in your logfile.
There are also other causes that appear in your log:

1. Display capture and game capture in the same scene, this is usually not a good idea.
2. Running webcam at 1080p can also increase gpu usage a bit, usually it is not necessary to run it at 1080p if the cam is just a little box in the corner.

I usually keep the Display capture hidden, does it still cause problems just being there if its not actually on? I didn't think about the webcam either, Ill turn the quality down



Edit: So I redid the configuration and scenes and such as recommended, Seems to be slightly more stable than previously, still getting encoding overloads but not as bad. It's also weird that if I tab out (borderless max) during the heavy encoding bits the encoder seems to have no problem and everything is smooth but if I click back in it overloads again.

If I am being honest here should I just use x264 until I can afford a 2 PC setup so I don't stress out the GPU on tougher games like Anthem or Blops 4? Is my PC just not strong enough for the juice needed?

 

[Selphadur]

Greetings everyone,
Just got RTX 2080 and started testing OBS. Performance was great on current build and quality was ok (usual nvenc). So decided to test new rtx beta 9 build. Copied files from zip to OBS folder, launched. It runs ok. But. There is but. That is why I decided to create account and post here. The quality. Oh... It is terrible! Worse than software ultrafast! No matter what quality preset (tested quality and max quality), psycho tuning on and off, look ahead on and off. I can stream 4k60 fps now with current OBS build at 51k bitrate no problemo. But beta build. Wow. I expected at least somewhat better than current build. Here are my specs (also no dropped frames, no frame lag, no dropped (network) frames, frametime around 0.5-1ms):
Win 10 Pro Newest Build fresh install (installed two days ago).
Newest Nvidia drivers
9600k CPU clocked at 4.7Ghz (watercooled)
16gigs ddr4 dual channel Corsair 14CL 2400mhz ram
256gig nvme, 500gig ssd, 256gig ssd
MSI RTX 2080 DUKE clocked at ~1900 core (stock) and 15ghz mem clock
GPU usage around 65% while streaming
If you need something else let me know!
Maybe there is some kind of a glitch.
Thank you!
Update:
Tried Max Performance (With same settings as above, meaning 1080p60 9000kbits, psycho and la on) and what do you think? Quality is same as Max Quality or Quality preset! Even worse usage is higher and it introduces some microstutter in game!