NVEnc Feedback thread (2014)

Status
Not open for further replies.
Do you have anything to show us besides "encoder initialization errors"? OBS log file, crash log, screen shots, courtroom drawings?
My 32Bit OBS lets me use Shadowplay, however my 64Bit OBS does not. However my 32Bit OBS doesn't seem to play well with Rivatuner... even when I list it to not detect, it keeps giving me "hook errors" that I have to alt-tab to ignore so I'd much prefer to use the 64Bit OBS.
 

Boildown

Active Member
Use High Performance preset instead of High Quality and instead of Low Latency anything. And if you still have problems after that, make a new thread and post a log file.

That said, if you're using NVEnc to feed Twitch its going to look bad, not enough bitrate, use x264 instead.
 
Use High Performance preset instead of High Quality and instead of Low Latency anything. And if you still have problems after that, make a new thread and post a log file.

That said, if you're using NVEnc to feed Twitch its going to look bad, not enough bitrate, use x264 instead.
I tried different presets but the issues till persist.
The "motion blockness" isn't an issue for now.
 
Not sure if this would even be helpful but after a brief lapse of reason (experimenting with x264 instead of NVEnc) I ended up going back to NVEnc. x264 is very nice with OBS but I just don't want the CPU load on the gaming rig. The 2 presets I've been using lately are NV Default and HQLL (High Quality Low Latency). Both of these have very high quality output for me - but here's the kicker -- at 50 megabits. I don't stream directly to Twitch from OBS at "typical streamer bitrates" because of my past experiences with NVEnc output at such low bitrates (looks "blegh!") and because I'd rather do transcoding on a 2nd box where it doesn't impact game FPS, and where I have complete control over the ffmpeg x264 options used to transcode down to those typical streamer bitrates to get the best quality possible.

Anywho, all this really just to say that NVEnc is very capable of delivering high quality output at 60fps. I have had some problems here and there with my OBS setup, two in particular that are driving me nuts currently, but NVEnc can definitely do the job with OBS @60fps if the rig it's running on can keep up and the game isn't doing something counterproductive.

The trick with NVEnc in my experience is pretty simple: Don't expect high quality video output from *ANY* of the NVEnc presets at bitrate levels that Twitch and other similar services recommend to stream at - you will NOT get it. I can't answer why NVEnc is seemingly so inefficient with its bitrate budget -- or at least as I have seen with OBS (the only NVEnc app I've ever used, so I'm not blaming OBS) -- but I have seen many, MANY people say don't use NVEnc because "its quality sucks". My response is simply: "you aren't outputting a high enough bitrate for it to look good".

I would agree with the notion that OBS using NVEnc is probably not a good solution for people who don't want to mess with a 2nd transcoder box, and just want to output from OBS directly to Twitch or other services at their recommended bitrates. But it is exactly what I want for what I'm trying to do.

Admittedly I am not a full time streamer, more of a hobbyist at this point -- but I am sort of a perfectionist or whatever other noun you'd like to substitute. I look at this way: A "new kid on the block" has a lot better chance of gaining regular viewers with high video quality than mediocre or crappy video quality. I have been tuning OBS, game graphics and ffmpeg transcoder settings, etc etc. in my not-so-copious spare time for a few months now to get the absolute best quality video I can get out of the setup I have. I've also streamed other games in the past with older versions of OBS that supported NVEnc a couple years ago.

Strangely enough, I actually have problems using 30fps output (XBMC calls it 20fps, plus or minus 2-3fps, and discards about 1/3 of the frames in the stream). 60fps output is pretty darn solid/stable for me as long as I don't have certain options in the game above "internet documented" levels. But that's a different conversation.

EDIT: Currently using OBS 64-bit 0.655 beta. Gamer rig is an OC'd 5960X with SLI'd 980Ti's, running Windows 7 x64 SP1.
 
Last edited:
TackMeister, I've got a similar setup and I'm trying to do the same thing. However, I'm trying to crop my Nvidia surround (6000x1080) down to 1920x1080, but so far but running NVENC encoding seems to still cause issues.

Do you mind posting your settings for NVENC encoding? I've been tinkering with 50K on bitrate and 50K custom buffer size, but a lot of the other settings are baffling. Thanks.
 
Hi Beast96GT,

Ah, so you're using surround monitors. I've never done that before. This is just a guess but I'm not sure 50000 would be enough for 6000x1080. 50000 works well for me but I'm only game- or window-capturing one 1920x1080 monitor and downscaling that in OBS to 1280x720 (or 960x540 lately) with lanczos. For THREE monitors' worth of 1920x1080 frames at your selected framerate, I wouldn't be surprised if you need to raise that max bitrate and buffer size up higher.

I don't have a lot of details on the maximum bitrate/buffer sizes of different NVidia cards - I'm sure it's out there somewhere. I do seem to remember reading a couple years ago that the Kepler (higher end 6xx series) GPU chips could handle higher NVEnc bitrates than older/lesser generations. I had used values as high as 100000 without problems with then-current OBS versions and my older Core2 Quad (Q9650) and an EVGA GTX680 FTW to do 1280x720@30fps, and I don't think 100000 was the upper limit on the GTX680. I'm using 2 GTX980Ti's in SLI now with an overclocked i7-5960X, but 50000 is easier on my transcoder rig than higher values at the slower x264 ffmpeg presets I prefer/need to use for $this_game.

It it was me, I'd try 150000 bitrate and 150000 (non-custom) buffer size and work down from there.

Anywho, here's some details of a 50mbit 1280x720@60fps config that is probably the best I came up with for $this_game.

Encoding tab
  • NVidia NVENC
  • Enable CBR checked
  • Enable CBR padding checked
  • Use Custom Buffer Size unchecked
  • Max Bitrate (kb/s): 50000
Video tab
  • Base Resolution: 1920x1080
  • Aspect Ratio: 16:9
  • Resolution Downscale: 1.50 (1280x720)
  • Filter: Lanczos (best detail, 36 samples)
  • FPS: 60
  • Disable Aero: unchecked
I generally use game capture as a source, although window capture also seems to work equally well. It's Windows 7 and I'm not using Monitor Capture so there is no good reason to turn the Disable Aero checkbox on.

Advanced tab
  • Use Multithreaded Optimizations checked
  • Process Priority Class: Normal
  • Scene Buffering Time: 700
  • Disable Encoding While Previewing checked
  • NVEnc Preset: High Quality Low Latency
  • Encoding Profile: High
  • Keyframe Interval: 2
  • Use CFR: unchecked
  • (everything else in this tab is unchecked - the default)
All the usual disclaimers apply - YMMV - what works well for you will depend on your system hardware, peculiarities of your OS, what game you're playing, what graphics settings you are using in the game, what other shizzle you have running, and possibly the phase of the moon and solar particle count. You will need to plan on tuning the output to your game, system, graphics settings, etc. until you are happy with it.

PS: I would recommend also saving a local copy of the file under the Broadcast Settings tab (Automatically save stream to file) while you are troubleshooting, so you have a way to know if a problem you are seeing in the final output from your 2nd PC (transcoder box) was also present in the original 50mbit stream coming from NvENC/OBS or not. (ie, are the encoder settings on the 2nd PC causing the problem or was the problem already there before the stream was transcoded?) Just remember these files will be huge.

Good luck!
 
Last edited:

Boildown

Active Member
The maximum resolution defined by H.264 is 4096 in either direction, so make sure you don't try to encode something bigger than that, it won't work.
 
Thanks for the responses, guys. A little clarification, I'm not trying to capture 6000x1080, I'm trying to capture the middle 1920x1080 region out of that resolution. This is because I want to play locally using my Nvidia Surround G-sync displays, but only stream the middle region. Cropping seems to be done after capture, so the performance issues I'm having occur no matter what I set the crop too.

Thanks Boildown, that's good information and certainly the reason for poor performance--I didn't know that.

TackMeister, thanks for posting your stream settings for NVENC, that will really help. I've got a similar setup, but I've only got an i7-4790K CPU with the SLI 980ti's. However, I bought an i7-5820K for my streaming PC--but that might be better served as my main CPU. Ultimately, I'd like to use NVENC to offload to the streaming PC.

Here's my plan:

TL;DR; - I'm trying to modify the code so it only captures a max width of 1920 (adjustable). As resolutions get larger, I think this will become more relevant.

I'd like to talk to Jim to see if this is even possible, but apparently he's got a lot going on. How much does he charge for an hour on the phone?? :) That said, I'm personally modifying the source code to attempt to only capture the 1920x1080 region from the GPU instead of the entire 6000x1080. So far I've had some success in the D3D11 Capture C++ code, but now I broke something and my test DX11 program crashes every time I try and inject capture code into it. I'm pissed because I was just messing around and didn't set up my own local git repository, and now that's it's broke and I can't revert. Doh! Fail. But I'll keep trying until I hit a wall--of spikes--because I've already hit a regular wall. lol.
 
Well I think I've done it. Jim thought the problem was definitely in the Capture code. After working on it all night, I finally have it working properly and only capturing 1920! I tested the performance, and noticed quite an improvement.

Thanks for the settings info, TackMeister.. those seem to be great... However, do you still get messages that your encoder has high CPU usage?

Anyway, the code is not ready for prime-time yet as there will be a process I'm sure, and there needs to be support added for API's other than DirectX11. Plus code reviews can be brutal. ;)
 
Does anyone know what preset would produce the best quality in regular OBS? This is at a bitrate for Twitch mind you. I realize x264 is better looking and hardware encoders are trash, but I can't use x264 as it just causes issues for my i5. So far after a fair bit of searching, I left with mixed results. Any pointers would be appreciated :)
 
Beast96GT --

Wow, congratulations that is quite a trick you pulled off there - ie changing code and getting what you wanted out of it! :)

Sorry it took so long to respond to your question in the last post - I just now saw it.

When I was still using the original OBS and NvENC I did still see the red message at the bottom of the OBS window sometimes, giving the warning that the encoder had high CPU usage. But I don't think I ever really saw any problems or lag when that was happening, and the video output was still good. I'm not sure what makes it think the CPU utilization is in the red zone when NvENC is generating the output - but it would be interesting to know what makes it detect the condition.

I am using x264 now with OBS-MP. I am not sure offhand what the reason was that I decided to give it a try, but I knew there was no NvENC support in it for me right now.


I do miss the CPU utilization (or lack thereof) with the combo of original OBS and NvENC output, but it turns out the x264 output at a higher bitrate (and VeryFast) from my gaming PC is not that much of a hit on my gaming PC and the transcoder rig it streams to first is still doing its thing pretty much the same good job. If/when NvENC becomes an option for OBS-SMP, without having to run Windows 10, and if it reviews well, I'll probably check it out again.

Kudos on your capture success!

@asianboy --

There are probably different opinions here on what preset will produce best quality, and I've seen more than a few good how-to's on stuff like that here in the forums. There are also some pretty good youtube tutorial-type videos out there that cover a lot of stuff like this too.

But if you can't use x264, then before you worry about which preset to use, you will need to decide which video codec/encoder you are going to use instead.

You might want to start with finding out 1) what "hardware offload" capabilities your system has (like NVidia NvENC, Intel or AMD others, etc.) and what kind of capabilities they have of producing quality output and 2) what formats you can even submit to Twitch in if you don't plan on sending x264/H.264.
 

oyoy

New Member
Jump in just to say how much i appreciated the hard work you guys are doing. With my potato CPU (I5-750 @4GHz) + GTX 770 OC 4GB I used to stream only trough Shadowplay but now with the NVENC + new presets I'm pretty excited that everything works great (15-30% CPU + 30-45% GPU) ..Wow. :D

NVENC + OBS plugins + Features such as layout composition for my 16:10 to align 1080p to one side (right) and use the black space (left) for overlay stuff is a game changer vs. Shadowplay (can't do any layout composition at all).

Now, after testing every single preset for live stream (FPS) I found that "Streaming 2 Pass" actually gives me the best result (Quality/Performance/Latency/Lag).
Encoding tab
  • NVidia NVENC
  • Enable CBR checked
  • Enable CBR padding checked
  • Use Custom Buffer Size unchecked
  • Max Bitrate (kb/s): 2600
Video tab
  • Base Resolution: 1920x1080
  • Aspect Ratio: 16:9
  • Resolution Downscale: 1.50 (1280x720)
  • Filter: Lanczos (best detail, 36 samples)
  • FPS: 30
Advanced tab
  • Use Multithreaded Optimizations checked
  • Process Priority Class: Normal
  • Scene Buffering Time: 700
  • NVEnc Preset: Streaming 2 Pass
  • Encoding Profile: High
  • Keyframe Interval: 2
  • Use CFR: unchecked
My Questions:
1. I read people saying "Don't use Streaming 2 pass" ..is there's a technical reason behind this suggestion?
In all Nvidia documentations I couldn't fine any mention of "Streaming" or "Streaming 2 pass".
2. Can anyone elaborate a little about those 2 presets please?
3. How the hell Shadowplay still show 10-15% better quality? Is there anything else that NVENC/OBS is missing?

This is not a deal breaker of course, just a simple wondering.
 

Boildown

Active Member
1) Two pass encoding is designed to go over all the content twice. Which isn't even physically possible for a live encoding session. If it even lets you use a two-pass preset at all, it probably only does the first pass and not the second, which probably means shitty quality. So just don't use it.
2) See 1)
3) Maybe it isn't a fair test. What bitrate and preset is NVEnc using for each encode when you make this assertion? How is the quality being measured?
 
I posted a problem with the released version of OBS Studio in the support forum in regards to NVEnc won't record at 4k resolution unless I go down to a 2K resolution. Not sure if anyone here reads the post over there.

Aside from that, will there be support NVEnc HEVC/H.265 support in the future release?

I am not a streamer, so local recording is most important to me.
 
Status
Not open for further replies.
Top