Question / Help Streaming hurts game fps + x264 settings optimization

[SHNS0]

Hey guys, I just registered because I've been experiencing a little bit of problems recently.
I started a Twitch.tv account wanting to stream some Touhou superplay and practice, but it's been all but smooth.
For starters, my bandwith's pretty limited. I can only spare 700kbps in upload, yet the games are very fast paced and I can't really skimp on framerate or video quality. Since I have a pretty good pc and a decent knowledge of x264, I've been tinkering around with the settings and came up with this:

CBR/Buffer @700kbps, CBR padding ticked, AAC audio @64kbps
1920x1440 monitor capture downscaled to 640x480@30fps
Multithread ticked, process priority @ high, scene buffering time @ 400ms (btw, what happens if I tweak the scene buffering? Couldn't find an explanation in the techincal guide here)
x264 CPU preset slow, CFR ticked (this seems to help with keeping the video smooth, but maybe it's just placebo)
Custom x264 settings: preset=slower tune=animation bframes=12 deblock=1:1 rc-lookahead=50 aq-mode=2 aq-strength=0.6 partitions=p8x8,b8x8,i8x8i4x4 psy-rd=0.4:0.0 no-dct-decimate threads=10

My specs are:
- 2560x1440 monitor
- i7 3820 (currently at stock)
- GTX 570
- 16gb RAM

This has worked pretty decently up until now, giving an massive boost in video quality, 100% smooth videos, and the CPU load had peaks of 60-65% which is pretty good (the games itself require very little resources, both CPU and GPU, even with all the encoding there were no slowdowns or problems).
Thing is, I've now started using the vsync patch for the games which helped me quite a lot with imput lag, which can be a real problem for very fast-paced games, and now streaming makes my game framerate unstable! Not by much, it's a few frames here and there, but it's enough to drop my concentration and to make me play like crap.
It's not the x264 settings, even without them the gameplay tends to get quite chunky pretty often. There shouldn't be any conflicting software since I close everything I don't really need before streaming. The recording itself is smooth since everything's converted to 30fps, but having a live stream that's smooth and a game that's frustrating and choppy is pretty absurd.
So, yeah, any ideas for this? Removing the vsync patch is the absolute last thing I want to do since it helps me both with gameplay and with capturing, too; Touhou games run only at 4:3 and at fixed resolutions, can't be captured in fullscreen because they lower the whole desktop's resolution when they start. The only workaround is capturing a screen crop with the game windowed. Since the game's resolution is pretty low in windowed mode, I first used ShiftWindows to center the window and to resize it at full height, but the vsync patch does this automatically (while SW is a bit of a pain) and removes the title bar and the window borders too, so it's really useful.

And by the way, if there are any x264 gurus out there, a quick check on my custom settings would be really appreciated since I should have some spare headroom for more hardcore settings and they could need some further optimization.

Thanks guys

 

[R1CH]

Put your preset back to veryfast, delete your custom settings and set your process priority back to normal (or lower if you want the game to have priority).

 

[Kharay]

• Go back to Veryfast, it's default for a good reason.
• Drop the Process Priority Class down to Above Normal
• Show us a log of a session when this phenomenon occurred.

In fact, just follow my troubleshooting guide and don't go with anything advanced for the time being. Stabilize the stream before improving on its quality.

 

[SHNS0]

Thing is, it's been stable up until I've used this vpatch. That's why I don't get it.
I've tested it a bit and it looks like what cripples gameplay smoothness is the x264 CPU preset, getting worse and worse with every slower preset. Is this setting the same thing as setting --preset=veryfast while encoding? If I for example set it to veryfast on the OBS settings list, but I write preset=slower on the custom ones, which one does x264 choose?

 

[Kharay]

As far as I know, the last one that was specified. Which in your example would be Slower. But, honestly, just leave it at Veryfast. And if you must pick a slower one, don't drop it below Fast. But, really, Veryfast is more than adequate for streaming purposes and in and of itself is more than capable of making a game look 99% game accurate. Depending on the title, obviously.

 

[SHNS0]

As far as I know, the last one that was specified. Which in your example would be Slower. But, honestly, just leave it at Veryfast. And if you must pick a slower one, don't drop it below Fast. But, really, Veryfast is more than adequate for streaming purposes and in and of itself is more than capable of making a game look 99% game accurate. Depending on the title, obviously.

The difference in quality:compression between veryfast and slower isn't really pennies as far as I know, it isn't like going from veryslow to placebo which almost doubles the time for a few percents of improvement in compression. But one thing's encoding shows and movies like I usually do and one thing's encoding live i suppose so I should probably do some tests. Still, the fact that slower was perfectly fine before the patch kind of bugs me.

Oh, another thing. What does opencl=true do? I didn't know x264 had such a preset. Hardware acceleration?

 

[Kharay]

http://en.wikipedia.org/wiki/OpenCL

Yes, hardware acceleration. Whereas it will not move all of the encoding process to the GPU (which at this time simply is impossible and/or infeasible), it does help to balance the load between CPU and GPU. Obviously, the faster the CPU, the less the gain is, since the CPU in question would have little issue as-is but still, it doesn't hurt.

As far as quality vs compression and such; remember -- streaming is a one-pass encode no matter what so there simply is a limitation on the level of compression and quality that can be accomplished.

 

[SHNS0]

Ok, here's a funny thing. Apparently, forcing preset=slow in the custom settings gives much better results in the gameplay than having preset=slow set directly on OBS. So after a bit of testing I found out that setting preset=slow in OBS results in pretty much the same inpact framerate-wise as something between preset=veryslow and preset=placebo in the custom ones.
What kind of sorcery is this?

 

[Kharay]

That actually... is kind of weird. Since as far as I know the dropdown essentially does the job of specifying a preset for you. So, you'd imagine the dropdown Slow being exactly the same as the manual Slow. How quaint.

 

[SHNS0]

Absolutely. I've done a bit more of testing and it's kind of strange. With heavy presets the game gets a bit unstable framerate-wise when there are massive spikes on the cpu load, which is pretty obvious considering how the placebo setting gets spikes of 95% load. But the OBS slow setting barely gives me 15% spikes and the slowdowns are more random. In some moments where tons of bullets come out it's perfectly fine, other times where it's just 3-4 enemies shooting it gets choppy. In comparison, custom preset=slow doesn't do this as far as I can see, and it actually looks pretty stable which is cool.

As for my former custom x264 settings, do you think I should tweak something? tune=animation was kind of obvious, but I'm not really sure about the other settings since I'm kind of familiar with them but I'm hardly a guru.

 

[R1CH]

Setting the preset from the dropdown or custom settings is identical. Custom overrides the dropdown, but that's all.

 

[WayZHC]

Even if you had stronger cpu like 3970K (or even E5-2687) you shouldn't go below fast preset. With 3820 playing heavy games on multiplayer is nearly impossible if you had the preset on slow. Looks like you play really light games since 3820 isn't that powerful cpu really. Try any heavier game and you'll find out. But if you play only those light games it might be okay then.

 

[SHNS0]

Touhou games are 2d shmups, you could probably run them at max settings and 60fps on a 15 years old system. The only lags I get ingame now are when there are MASSIVE spikes of cpu load and when/if load reaches 95/100%

 

[SHNS0]

I've pretty much eradicated the slowdowns, there are a few of them now but it's not a problem anymore. I guess it's just the patch's problem that escalated under the wrong circumstances.
As for custom settings, after some testing I am currently going with:

preset=slow tune=animation bframes=16 b-adapt=1 ref=5 deblock=1:1 rc-lookahead=50 aq-mode=2 partitions=p8x8,b8x8,i8x8,i4x4 direct=auto me=umh merange=48 subme=10 psy-rd=0.5:0.0 trellis=2 no-fast-pskip threads=10 resize:width=640,height=480,method=spline

And it's amazing. Low cpu load (45% max, almost always stable between 30 and 40), massive improvement in quality. Here's a pic of a seriously bit-starved scene, left is with settings on and right with settings off: http://i.imgur.com/DYr6zSe.jpg

What do you think of these settings? I think I went overkill with merange=48 and subme=10, but to be honest it's working smooth as silk so I'm not really concerned about them. Also, would setting profile=high10 actually do something? 10 bits video should deliver a bit more quality with the same bitrate - but do Flash, Twitch or OBS even support it?

 

[WayZHC]

preset=slow tune=animation bframes=16 b-adapt=1 ref=5 deblock=1:1 rc-lookahead=50 aq-mode=2 partitions=p8x8,b8x8,i8x8,i4x4 direct=auto me=umh merange=48 subme=10 psy-rd=0.5:0.0 trellis=2 no-fast-pskip threads=10 resize:width=640,height=480,method=spline

I'm totally noob at x264 commands but what gives you 10 threads on 4c/8t cpu? Shouldn't it be set to 8 or am i missing something i should know about x264?

 

[SHNS0]

preset=slow tune=animation bframes=16 b-adapt=1 ref=5 deblock=1:1 rc-lookahead=50 aq-mode=2 partitions=p8x8,b8x8,i8x8,i4x4 direct=auto me=umh merange=48 subme=10 psy-rd=0.5:0.0 trellis=2 no-fast-pskip threads=10 resize:width=640,height=480,method=spline

I'm totally noob at x264 commands but what gives you 10 threads on 4c/8t cpu? Shouldn't it be set to 8 or am i missing something i should know about x264?

http://mewiki.project357.com/wiki/X264_Settings#threads

Default: auto (frame based threads: 1.5 * logical processors, rounded down; slice based threads: 1 * logical processors)

My cpu has 8 logical processors, so if what the mewiki says it's true then threads should be set max to 12. I just took out 2 because my game is really light but I still don't want x264 to completely use the CPU. But I haven't done any testing with other threads=x settings so I don't know if it actually works or not, I'm just following the guide.

 

[Kharay]

Actually, if I read that correctly, it should default to 12 for you. In other words, you're currently depriving it of threads it normally would have had.

 

[SHNS0]

Actually, if I read that correctly, it should default to 12 for you. In other words, you're currently depriving it of threads it normally would have had.

Read above, I took away 2 to be kind of sure that encoding wouldn't take everything for itself, slowing down the game. Which isn't really a problem since my cpu load is now 45% at peak

 

[Kharay]

No, you read. ;) The default would be 12 for you. Default threads aren't an issue. I have threads at default and do not have a CPU anywhere near as powerful as yours and it never even comes close to maxing out my CPU. My maximum (short peaks) total CPU load while streaming is something like 75%, which is while capturing a piece of software far, far more CPU heavy than the games you're playing (a 64Kb intro). In fact, 64Kb into's are a great way to benchmark your streaming performance, since they are really CPU and GPU heavy.

 

[SHNS0]

No, you read. ;)

Oh, sorry, I didn't mean to sound arrogant or anything, it just came out that way. I'm actually grateful for the help.

I'll try and see if anything changes setting threads to 12 or just leaving them to default. Thanks again. What do you think of the other settings? Something else I should take a look at, or that could need further optimization?