Question / Help Sharing Experience of x264 (seeking perfection suggestions)

[BigYuckFou]

Hello All, Sharing Experience of x264 (seeking perfection suggestions)

attached - (Stats, Output, Advanced, Audio, Video, CPU/GPU/Mem usage, Twitch Inspector, r-1.ch-Inspection)


OBS Log File - https://obsproject.com/logs/LEjXrEUCcFZ1rOQF

I wanted to share what I was just able to achieve using x264 and OBS. After 400+ hours of testing and long hours reading. I was able to achieve using the following:

------
4k inputs (2) via capture cards (1x 1080 via capture card)
3840x2160 base canvas
3840x2160 out under video
1080p out->rescale output via CPU under stream settings
profile->high
preset->placebo (although heavily modified)
keyframe->2
90%cpu
render time->3.8/4.4
not one single dropped frame lag
---OBS x264 Options---
threads=28
level=4.2
b_adapt=2
direct-pred=spatial
me=umh
trellis=2
bframes=2
ref=6
subme=9
analyse=all
rc_lookahead=60
me_range=20
------

PC Specs:
Intel 10940x (14core/24thread) All Core Overclock at 4.7ghz using 1.155vcore
Nvidia EVGA 2060 KO Ultra GPU
Corsair AX1200i PSU
32gig Gskill Memory @3200
Avermedia Live Gamer 4k x2
Avermedia Live Gamer Ultra (x1)
water cooled CPU

 

[BigYuckFou]

I sincerely thought that this team of experienced OBS enthusiasts would love to respond to a post with in-depth information and a chance to interact and provide feedback on how to get this even better, or tell me there is something I did wrong to help.

but not a single response.

please anyone I would love some feedback. pretty please :)

 

[Banyarola]

Don't take offense but, I have no idea what any of your posting means...
Others here with more experience then I may but I don't have a clue...

 

[BigYuckFou]

I just appreciate your honesty. just looking for tips on how to make it even more crispy on Twitch and if I have any room for improvement.

---Update---
I did make a few adjustments to the settings and here are the settings and video results to Twitch
Twitch Broadcast test, 6000kbps 1080p@60fps

test image from stream at full quality

I was told previously that 6000kbps @ 1080p60fps didn't need level 4.2, is that true?

threads=26 (reduced to 26 instead of 28)
level=4.2
b_adapt=2
direct-pred=spatial
me=umh
trellis=2
bframes=2
ref=6
subme=9
analyse=all
rc_lookahead=60
me_range=20
aq-mode=0 (reduced to zero from 1)

 

[vapeahoy]

How about you just keep to one thread? what settings you can use will depend on the use case. Ie if streaming a slow game as hearthstone you can use heavier settings, if streaming high motion fps games you should use less. I posted the links to the x264 documentation, all the answears are in there, you simply have to adjust. Everyone has their own system and so what works for somebody else will probably not work for you.
It's like asking what shampoo you should use based on how your hair looks like. I don't know if you have underlying issues with your scalp tissue, the same way i dont know how optimized your latency is overrall on your pc, despite i can see your components.

You *actually* have to figure out stuff on your own.

 

[BigYuckFou]

so you posted all of that and didn't even provide an answer to the question you think i need to stick to one thread for. thanks bud. Precisely why I made an independent thread for the single question.

If you did post links, I don't see them in your response.

even this page is unclear to me what needs to be selected. but hey. im a noob.

H.264 profiles and levels | Inside & Outside MediaCoder

 

[Banyarola]

so you posted all of that and didn't even provide an answer to the question you think i need to stick to one thread for. thanks bud. Precisely why I made an independent thread for the single question.

If you did post links, I don't see them in your response.

even this page is unclear to me what needs to be selected. but hey. im a noob.

H.264 profiles and levels | Inside & Outside MediaCoder

Because he is only pretending to be smart while I was just being honest..

 

[vapeahoy]

so you posted all of that and didn't even provide an answer to the question you think i need to stick to one thread for. thanks bud. Precisely why I made an independent thread for the single question.

If you did post links, I don't see them in your response.

even this page is unclear to me what needs to be selected. but hey. im a noob.

H.264 profiles and levels | Inside & Outside MediaCoder

If you can't figure out what are your own threads, perhaps you should learn to use the forum search function.
I posted the links as u can see here:

Question / Help - Encoder overload (x264) CPU usage at 35% - why?

Hello fellow OBS members, Can anyone help tell me why I would get the "encoder overload" while attempting this? I use a 14 core Intel with hyper threading running at 4.7ghz all cores.Dang good PC. 32gig memory and a 2060 KO Ultra GPU. I am using x264 CPU encode. Slow preset. I do have set some...

obsproject.com

Because he is only pretending to be smart while I was just being honest..

So because i dont want to say what settings i think he should use because i dont want to be a baby sitter i am pretending? Seriously?

 

[BigYuckFou]

Hi @
vapeahoy
please refrain from responding to my posts thank you. good day. I am finding this ignore feature super handy in this situation. goodbye.

 

[BigYuckFou]

To the rest of you fine folk, I am Simpy seeking guidance on if there are any errors I am making. these settings work wonderfully for me. however in another post it was determined via my log files that I had an issue or two, I have taken care of them and hope someone polite enough to respond will do so.

 

[koala]

Unfortunately, it isn't possible to comment on your settings, because they are 12 out of infinitely much more settings you are able to set for x264. I don't like telling bad news, but nobody will check what all these settings actually do in addition to the chosen profile and nobody is able to evaluate all this. It's too tedious. For this, the presets (slow, medium, fast, etc.) exist. These presets are carefully crafted sets of sensible settings for varying CPU and quality requirements. In addition, there are profiles (baseline, main, high) that contain carefully designed sets of decoder requirements to be able to support different kind of devices.

We want to just use the program and stream/record. We don't want to test and configure the encoder for hours. So we choose the profile that fits best our hardware and we're done. There are quite some articles on the web that deal with comparing the presets against each other in terms of quality, file size and CPU demand. They show how the encoder will scale with the presets. They also show there is really no need to hand-craft differing settings - it's all already within the presets.

So, if you seek guidance, listen to my advice: don't use any special x264 encoder settings. Choose profile high and the best preset your CPU is able to support for given resolution and fps.

There is also an actual error you made: you activated rescaling output to 1920x1080 in the encoder settings. This setting is very CPU consuming, because it's taking place in CPU space. Uncheck this and instead rescale your output to 1920x1080 in Settings->Video->Output (rescaled) resolution. This rescaling is taking place on the GPU and using no computing resources, because the GPU has extremely efficient operations to handle this rescaling.

 

[FerretBomb]

Additionally, optimal settings vary based on the game being played, and even the level you're on. When recording, if you want perfection you can run a ton of test encodings and run them through VMAF against the original (gigantic) lossless recording to pick out which one is the closest.
When streaming, you don't have that option. You tend to go with whatever is close-enough, and don't stress about screwing with the individual x264 options all that much... especially when trying to stream 1080p60@6000 which is always going to look not-great no matter what you do.

If you REALLY want to do something quite that time-consuming, you'd losslessly record 10-20 5 minute clips of a variety of varying motion/visually different games/levels/etc, re-encode them using a variety of options, VMAF them all, then plot the options that get the highest scores for each game type or individual game.
Likely to find out that the settings closely match one of the existing presets, maybe with one or two minor tweaks. Streamlabs did that. The changes were on average one or two points off-preset for a single option.

I really don't mean to rain on your parade. Really I don't. You clearly put some work into coming up with those settings.
But really, the difference is going to be minimal at best given the bitrate/resolution*framerate in use, and likely not visually noticeable unless you were running two streams side-by-side showing the exact same content. Even then, chances are good that it'd be pretty hard to tell.

Even worse, the gains between Turing NVENC on your GPU and taking all that time are going to be near-zero as well... Turing swings on-par with x264 Slow, and the reducing rate of returns between Slow and not-Placebo makes that line even thinner.

It's awesome that you're paring down like that. Yes, there may be some other enthusiasts trying to turn the dials on 1080p60@6000 to 11. But really, for normal use, it's going to be a wash. Anyone running 1080p60 on Twitch is likely going to use the 'unofficial word-of-mouth' 8000kbps 'known to work' rate, at which point all the settings go back out the window.

There's a chance of finding someone else around here who's also into that, absolutely. But the 'not a single response' self-bump? Yeah, don't expect too many here to have at-present top tier hardware and the kind of free time to sink in to corroborate, much less improve and give feedback on your results. Especially when they're just a list of settings you're using, with no qualitative metrics.
I'd point you toward a couple of video editing forums with a more technical-core base, but none of them are going to be imposing the 6000kbps restriction, and will just ask why you're bothering.

 

[BigYuckFou]

The reason I used the CPU downscale instead of the GPU on is I was encoding on the CPU and apparently something gets whacky downscalling on the GPU then encoding on the GPU as I drop frames on the encoder if I do that, but doing it all on the CPU I drop zero frames.

 

[BigYuckFou]

I agree about the GPU encode. using the 2060 KO Ultra it looks dang good... and its much less of a headache.

The reason I went this route to get "better settings" was because while I can do slow preset I cannot do slower due to the higher bframes and reference frames. the encoder just lags...

One thing I don't understand is why with 40% cpu usage and x264 with CPPU downscale, or even GPU downscale, why I get encoder dropped frames.

It seems odd with such CPU overhead that the encoder at 3.0ms render time would drop frames... any idea?

 

[FerretBomb]

Not really, no. You'd have to wait for one of the folks super intimate with the x264 stack to answer that one. You might try exiting OBS and checking the timing tree at the end of the logfile, see if anything particularly stands out.
But yeah, multi-pass encoding generally causes issues during live recording. And with the reducing rate of returns on the slower preset, Medium was the 'good enough' standard for the longest while until Turing NVENC cranked that down to Slow-equivalent.

 

[BigYuckFou]

I used the suggestions above, went with the GPU downscale instead of CPU re-scale. it made a significant difference. basically just getting the downscale content to the CPU for encode was the most important thing...and with Turing the quality is superb too.

The one thing I believe we all struggle with is high motion content, my goal has been to deliver high motion with butter smooth playback with no lost frames. I believe after following the suggestions and making a few tweaks I have been able to get this working pretty flawlessly with very high motion footage. Call of Duty.

I realized that reference frames are heavy on the CPU and were the cause of the encoder lagging mentioned above with low CPU usage, throwing high reference frames at a CPU gives the encoder a heart attack even at low cpu usage and seemingly low render times. B frames offer significant bandwidth steadying as does lookahead that offers the better spread of those bits across frames. Also there are several settings that greatly affect how x264 looks for motion vectors and those changes. like "me=tesa"... side note** direct-pred=temporal or auto looked way worse and lost frames ensued.

Here is what I ended up with and its pretty dang flawless. you can see 50 bframes, 1 ref frame. and a lookahead of 180 (3 seconds)

me=tesa, rc_lookahead=180, and bframes=50 made the largest difference in high motion, and ref=1 made the dropped frames stop entirely along with the manual thread count. Note the thread count is just three lower than the (cores+hyper threading) for me which total 28 threads, so I set it to 25 to provide a few for lookahead and other programs.

If you have the hardware I suggest you give it a shot with high motion streaming. not for recording.

x264 options -> placebo preset(heavily modified) -> high profile
threads=25 b_adapt=1 direct-pred=spatial me=tesa trellis=2 bframes=50 ref=1 subme=4 analyse=all rc_lookahead=180 me_range=24 aq-mode=2

This also sets the level automatically to 4.2 instead of 5 like before. which in and of itself is an improvement. got down to 60% cpu usage and 2.8-3.2ms render times.

hope this helps out someone that wants to stream high motion content streaming

Here is a sample of the outcome, to me the block-iness from fast motion is mostly gone...

Twitch

Twitch is the world's leading video platform and community for gamers.

www.twitch.tv

 

[koala]

It may be you're deceived by the name of the profile. Why not choose the best plain profile the CPU is able to support your encoding? It looks as if are choosing "the best" profile (placebo) just because of its name. To make your CPU able to use it, you tune down some of its parameters, so you don't get encoding lag. But, by doing this, you bring the overall operation of the encoder into the same range of the less demanding presets. I will not compare the settings in detail, but it may be very well you just rebuilt the settings of one of these other profiles.So you're actually use one of these, and it may be the only thing that remains from the placebo profile is its name.

As last comment: you have a Nvidia RTX GPU. It has a very sophisticated hardware encoder (nvenc). Its quality is between the quality of the medium and slow preset of x264. Given the fact this encoder uses next to no CPU and GPU resources, because it's a dedicated hardware circuit, it's a viable alternative for everything. You're not playing and capturing a game on your PC, just grabbing a console, so this may not apply to you. But if you're playing a CPU heavy game on this machine, this is a viable option to x264 to give the game the full CPU.

 

[BigYuckFou]

Certainly cannot disagree that Placebo is only left in name.it is heavily modified.

I have done hundreds of hours of tests to arrive at.. starting at say slow and then increasing to the settings I use offers a bit less quality due to settings within the placebo preset that are left untouched I don't use or understand enough yet not present in slow.

Using placebo was a choice not by the name but due to the settings within I don't yet grasp completely. It offers with heavily modified optional settings the quality for fast moving content I am seeking. There is something in this profile I am missing in the others, something seemingly affecting fast movement.

The reason I won't use slow preset in its natural form is that it is heavy, and oddly enough slower lags the encoder. plain and simple slow preset does not even touch on the main settings that take care of fast moving content (hex, high ref frames, low bframes low merage, low lookahead etc...). Not to mention as you go from slow to slower or slowest the ref frames increase and in my tests they are simply not necessary nor is the huge lag impact that they provide on the encoder (for streaming). as I use one ref frame. that is it.

Also, while I love the Nvidia NVENC and award it great! overall, again it goes blocky and/or blurry upon fast moment. plain and simple its great. but not better than specific settings in x264 that directly affect fast movement like:

me=tesa, rc_lookahead=180, and bframes=50 merange=24

brames -> allow good quality and can reference the prior and after frames from their location
rc_lookahead -> stabilize the bits @180 across three seconds
me=tesa-> determines motion estimation and how search for motion vectors (important for fast motion)
merange-> range of the motion search within tesa/esa/umh/hex (again important for fast motion)

if you use me=hex merange is locked at or below 16...(not enough)

also, I am if going to use the NVENC limited to 4 bframes which I am perplexed by this limitation in OBS and its terrible it cannot be raised as Bframes are truly what stabilizes the quality and bits in fast movement along with lookahead. the "me" settings at UMH is good.. but TESA takes it to a whole different level when fast movement is engaged. they say NVENC is like medium. the video I provided as a sample is by far better than that. I think you would agree, even better than slow by a good margin.

do you know of a way to enable more bframes on NVENC? I would really like to test that?

 

[koala]

How did you create and compare your videos? Create manually and compare visually, or by creating an automated test series and computing the psnr of the encoded videos? Comparing visually is not objective. You're deceived by your expectations and your eyes.

To be able to compare the same stuff against each other, create a 2 minute lossless reference video with slow parts as well as fast parts, then use this as sole input for the test series. As measure of the quality of an encoded video, use the mse/psnr (mean squared error/peak signal to noise ratio) by applying the psnr filter of ffmpeg. The psnr is simply a number that tells how different the frames are between two videos, ideally the input video and the encoded video. Then you can sort videos from a test series by psnr to get a rating.

For this, use the cli version of ffmpeg to do encoding tests of the reference video, because it has the psnr filter, and you can grab the psnr values from the resulting logfiles. You call ffmpeg in an automated way (shell script) as many times as you like.

I did some comparison work in this thread: https://obsproject.com/forum/threads/comparison-of-x264-nvenc-quicksync-vce.57358/ and since you are using x264 only, I recommend you use only ffmpeg instead of OBS, because it's much easier to automate. I used autohotkey to automate OBS, but only to support Quicksync and nvenc that were not available in the ffmpeg command line binaries at the time.

 

[BigYuckFou]

well, I did a visual test as this is what I am going for. the numbers do not matter to me. if it looks better overall that is the goal for me especially when fast motion occurs.

by the way, great thread you linked to. wish there were 6000kbps tests too.

what is mse score?