Question / Help GTX 970, NVENC, Quick Sync, Bitrates, x264. And more. Probably.

[SporksAreGoodForYou]

Right then. I've been trying to find a definitive answer to this riddle.

I've been uploading game guides for a while. Fraps did the job, I could capture raw and compress to 25Mbps, upload takes a few minutes, great 1080p30 videos. I'm not fussed about 60 frames.

A few of my viewers have been asking for live stuff recently, and so I had a play around. Went through Wirecast Play (awful, just awful, except the YouTube integration, which is lovely), and didn't get on with xsplit. OBS is goldilocks.

Anyway, tried playing a couple of games, and the x264 software compression just collapsed performance. Frame rates dropped from 120 to ~18. i5-4690K overclocked to 4.8GHz. Stable. So, it really should be ok, but whatever. That's fine.

I looked into NVENC and Quick Sync. After finally getting Quick Sync enabled (thanks to Yuri for that: http://www.gamelkoorn.com/how-to-setup-obs-for-twitch-using-quicksync/), and both (obviously) let the CPU do its thing
Problem solved. Except it's not. Quality is, as we know, not great.

So, I'm streaming to YouTube, they support up to 6k ingestion rate. I can push a solid 1080p30 at that bitrate, but quality is impacted. I can't find any hard number on what I should expect at 6k, or how that would compare to x264 at the same bitrate. nvidia published a thing, but it's by them, and therefore untrustworthy. They claim at ~6.5k, there's virtually no difference.

I'd like to get to the bottom of best performance between NVENC and Quick Sync at 1080/30 @ 6k, and the comparison to x264. Is there an easy way for me to do this, and then publish the results here for you guys, so we can settle this for once and for all. Is it just a question of recording these things, and then eyeballing them?

On a sidenote, NVENC on the 970 has a ridiculous amount of encoding options. Any idea what the difference is? 2 pass will just smooth the bitrate, so that's covered. But quality vs performance? Both claims to be able to handle multiple streams in parallel, so I don't see why anyone would ever pick performance if they're just encoding a single stream. What's Bluray Disk? What's low latency? What about lossless? It seems, from the names alone, that High Quality Low Latency 2 pass would be the chap to go with. But is it? Really?

On another sidenote, is there any reason not to choose 'best quality' on Quick Sync?

Finally, on an unrelated note, is there any reason why the audio boost features wouldn't do fractions? 2x mic input seems like overkill.

 

[FerretBomb]

NVENC is lowest-grade quality. Very poor compression, needs a ton of bitrate to look not-crappy.
QSV is a small step up. Far from good.

Hardware encoding options are mostly meant as a workaround; a band-aid for those with potato-grade CPUs to still be able to stream, or to chuck truckloads of bitrate toward for local-recording only (such as to record gameplay at higher resolution/framerate for later editing and upload to YouTube). There's not really anything you can do to make them look good (aside from aforementioned truckloads of bitrate), just less-bad.

A Core i5 generally will run out of gas much past 720p@30fps on the x264 Veryfast preset. They're great for gaming-only rigs. But generally-speaking they aren't going to be able to handle 1080p@30, full-stop, outside of a dedicated encoding-only machine (and even then it tends to be iffy). If you want to encode 1080p@30fps video in real-time as with livestreaming, you pretty much need the raw grunt of an i7.
In short, dial it back a notch. You have a midline machine and real-time video encoding is murder on a CPU.

Far as the mic boost goes, use the bar graph to the left of the mic icon on the main OBS screen. It's a slider. Clicking the mic mutes it. Likewise with the speaker (system channel) to the right.

 

[SporksAreGoodForYou]

Ferretbomb, thank you for this. First up, the easy one!

The mic bar! Hurrah! Thank you so much.

So, it seems like, for whatever reason, Life Is Strange is massively CPU intensive, or the encoding requires extra CPU. Which seems insane, given the game, but whatever. Alien Isolation can do x264 on this rig in real time without even breaking a sweat. I did the same 2 minutes of footage with nvenc 2pass high quality low latency, quick sync, and x264 (superfast and veryfast), all at 6k.

I honestly can't tell the difference between them, visually, from just watching. They're here, if anyone wants to check for themselves.
https://www.dropbox.com/sh/6a733z1eeznouoh/AAB0dzRfakcai91SGqRMd0hMa?dl=0

x264 screws with the colour balance (doesn't do studio RGB to computer RGB) so everything comes out a little darker. And, if pushed, I'd have to say this is the worst of the lot.

And, for some reason, on NVENC, I can't seek using the progress bar. It just gives me a black screen. no idea why that is.

So, based on my purely personal opinion, I'd say NVENC @ 6k 1080/30 is more or less identical to x264. But they're all pretty awful, comparatively.
Even if you can do real time encoding of 1080/30 content, unless you find a site which will ingest at 10k+, it seems like you're going to take a pretty significant quality hit, simply due to the raw number of pixels.

 

[Boildown]

The thing you need to worry about is Quality Per Bitrate and how much CPU it takes to encode at that quality.

NVEnc basically uses next to zero CPU to do what it does. But its quality per bitrate is really bad. So NVEnc is bad for streaming, but if you crank up the bitrate (20-30000 at 1080p60) then it can make very good quality recordings to your hard drive. Use High Performance preset if you have a Kepler series GPU, High Quality for Maxwell. Don't bother with the others.

x264 is massively configurable. If you have a shit CPU, you can set it to SuperFast or UltraFast and record using not much CPU but with poor quality for the bitrate you generate. If you have an "ok" CPU, set x264 to VeryFast preset and you'll get average quality per bitrate and hopefully not have too much of an impact on your game. If you have a top-end i7 or extreme series CPU, with 4 or more cores and hyperthreading, you can set x264 to Faster, Fast, maybe even Medium preset, and have very good quality videos for the bitrate you use. This is as good as it gets for live streaming purposes.

Quicksync is between the two above, generally. Better quality per bitrate than NVEnc, but uses more CPU than NVEnc as well. Depending on the settings used, works out to be around x264 Very Fast or Faster quality per bitrate, but with using less CPU. Quicksync caps out though and can't achieve the quality per bitrate that x264 can on better presets. This is assuming you have a Haswell (or later) Intel CPU; Ivy and Sandy Bridge Quicksync were significantly worse than the Haswell generation. You can use Haswell's Quicksync to stream with, but I wouldn't stream with Ivy or Sandy Bridge Quicksync, only save to hard drive for those.

My advice is this:

If you're just saving to your hard drive, use NVEnc with a bitrate in the 20000-30000 range.
If you're live streaming with a Haswell CPU, experiment to see if Quicksync or x264 on its various presets gives the best results. If your game plays fine (you can't "feel" the encoding lag), the look at the OBS log file for the "duplicated frames". If more than 1%, your CPU is too stressed and you need to lower your settings or try another encoding method. If the duplicated frames are around 0.1% or less, then you can probably increase the encoding settings to increase quality. If duplicated frames are more than 0.1% but less than 1.0%, then you've got it tweaked well.

Edit: One more thing, for color balance issues, be sure to enable "Encode in Full Range" on the Settings > Advanced page.

 

[SporksAreGoodForYou]

Aha! Full range is amazing. Thank you Boildown.

For local, for me at least, fraps does the job. I've got a dedicated 5tb array and record uncompressed then postprocess. I heavily edit the offline stuff anyway and put voice in afterwards, so using OBS to record is only necessary if I'm trying to see what the output quality will be like for streaming.

I played around with a couple of x264 settings, and as you may have seen in the footage I linked, It's more or less the same across all 4 at 6mbps. I certainly couldn't spot any meaningful difference. Quick Sync doesn't seem to have a noticeable impact on my CPU, but I only did limited tests. The 4690k is Haswell, so I'm in good shape there.

Duplicated frames: 0 across all 4 tests (QS, NV, X @ super and very). But as I said, Alien Isolation seems to not hassle the CPU (uses < 10% most of the time), which is why I thought it would be a good test. Plus, there's a lot of similar colour walls, so it may not be the best way to test.

I guess I'll just tinker within those excellent guidelines you've both provided, and set profiles for each game based on testing.

I guess, though, that this means hardware accelerated post-encoding is using the NVENC encoder, and therefore suffers the same pitfalls?

 

[Boildown]

I guess, though, that this means hardware accelerated post-encoding is using the NVENC encoder, and therefore suffers the same pitfalls?

I don't understand this question.

 

[SporksAreGoodForYou]

I don't understand this question.

Sorry. I mean, for example, if I use hardware accelerated encoding on a program such as Sony Movie Studio, is it running x264 on the GPU using custom code, but instead using the worse-quality-per-bit compression.

Or does that entirely depend on the method I choose?

 

[Boildown]

Hardware encoders use the h.264 technique to do their compression, but they don't use x264, as that's a specific software implementation of h.264.

Read this: http://www.avidemux.org/admWiki/doku.php?id=general:common_myths#software_vs_format_format_vs_codec

If you're doing non-real-time encoding, then you should pretty much always use x264, as it offers the best quality per bitrate. Use the slowest x264 preset that you don't mind waiting for. Sony Movie Studio doesn't have the x264 encoder built in, I think they use Mainconcept, so the best thing to do IMO is write a lossless intermediate file out of Sony Movie Studio and use Handbrake to make the final encode to something you'd upload to YouTube.

Here's a good video for that part: https://www.youtube.com/watch?v=rWMX5lSvEgY

 

[SporksAreGoodForYou]

Thank you again for this.Just did my first live stream using Quick Sync, and it went pretty well.
Interesting advice on Handbrake. I'll run some tests and see what happens. (and yes, they use Mainconcept)

 

[Caio Faustino]

Read this: http://www.avidemux.org/admWiki/doku.php?id=general:common_myths#software_vs_format_format_vs_codec

That link was EXTREMELY helpfull, thanks a lot. I'll check the video when I get home. You said that setting a high bitrate for recording with NVENC should give good quality. But my tests with the presets gave me hugely different results.

If you could take a look at my thread and see if there is something I might be missing there I would be very gratefull.
https://obsproject.com/forum/threads/wich-nvenc-preset-for-high-quality-high-file-compression.34644/

I'm still learning a lot about the technical differences in videos, so apologies for my early mistakes.

Also setting the Encode in full range option made my video colors very weird and dark.