Question / Help Automating recording tests of different OBS settings

[Roy777]

Hello friends,

I'm trying to figure out if there is a way to automate recording tests of different OBS settings.

I'd like to record a large amount of videos and would like to have the best quality/file size ratio, I'd like to try recording using different profiles and settings and evaluate the results.

I could change the settings manually, but I would at least like to record a specific duration of time accurately, let's say 1 minute. and also to capture the same thing on screen each time, so it would be the same content in the files for a correct comparison.

The manual way to do this would be to change the settings-> put the right content on screen -> start recording-> stop recording (on time)

Is there a known way to automate this or a part of it?

Thanks a lot,

Roy.

 

[FerretBomb]

To stop the recording there is. In the Tools menu, there's a Timer function.
Just don't forget to disable it when you're done recording testing samples, or you'll be pretty frustrated when OBS refuses to stream/record for more than a minute.

The rest, not so much. You'll be stuck doing that part by hand.

 

[koala]

What you can do depends on what goal you intend to achieve. Especially what settings you intend to test within OBS. In terms of encoding, OBS is almost completely relying on ffmpeg, so if you intend to just test different encoding settings supported by ffmpeg, you might as well go with the commandline ffmpeg tool. This can be completely automated. However, if you want to vary settings not present in ffmpeg, you still need OBS.

The first step is to create a lossless reference video of the thing you intend to test. This reference video is your sole source of all your tests. Make sure this is perfect in terms of content as well as no lags, no dropped frames, and exactly the fps you want to create. It should be the native resolution of the thing you intend to record, no scaling. It should also be the same color space you want your videos in, usually NV12. NV12 is not lossless in comparison to RGB, but since all your recordings will probably be in NV12 color space as well, this isn't a constraint.
Use the "lossless" quality in simple output mode for creating the reference video. It should be about 1-2 minutes in length and contain scenes of varying motion and should be a reference in general for the content you intend to create.

Once you have a reference video, you can do with ffmpeg alone:

• encoding tests with encoders supported by ffmpeg. x264 is supported, nvenc (h.264) probably as well. Quicksync is in theory supported by ffmpeg, but I've yet to find a binary with Quicksync compiled in.
• make encoding tests with several x264 or nvenc profiles
• several output resolutions (use the ffmpeg resize filters)
• several bitrates

Not possible with ffmpeg alone:

• make tests with encoders not supported by ffmpeg. AMD VCE is one of that, at least with the Windows version of ffmpeg.
• video filters specific to OBS
• overlaying multiple sources in addition to the reference video. This is in theory possible with ffmpeg, but it's probably not the same as with OBS. But you can create the reference video with composited sources in the first place.

I did such a thing in the past (see here), but I didn't go the ffmpeg way, because I wanted the result be a 100% OBS product, and I didn't find a Quicksync-enabled ffmpeg. There is no difference between ffmpeg encoding and OBS encoding, but to avoid people saying "but OBS itself would produce different quality, so your tests are invalid", I used OBS instead of ffmpeg in the first place. I used Autohotkey to automate the necessary clicks for OBS to create the different configurations and start/stop playback of the reference video and of the recording. I did even more, probably too much for you: I did a frame-by-frame comparison of the resulting videos and the reference video, then computed the MSE to get a similarity value, so I was able to sort by MSE, thus sort by objective quality. Not by perceived quality. You know, the human eye can be deceived, but math cannot.
The autohotkey script is linked in that thread, but it is not directly usable today, because the UI of OBS changed much since then. You have to determine all the button positions anew.

 

[Roy777]

Thank you @FerretBomb It's good to know (: and Thank you @koala for all the details! that's interesting and also very good to know.

I eventually created a script using autoit, that starts the recording in OBS, automates the same actions and motions in the 3D design program I captured and stops the recording, all in 30 seconds, then I manually changed the settings in OBS to try different things.

I was testing and testing but got a little lost.. I'm trying to get the best quality/lowest file size, to be streamed later through a website. what I've tried so far is using NVENC- resulting in very good quality and big file sizes and x264 resulting in ok quality with a smaller file size using slow, while not loosing any frames in both. using x264 slowest in real-time, I'm losing about 70% of all frames but the frames that are left look much better than slow.

so I have tried to create that best quality video file as you @koala mentioned (the one captured with the NVENC was great for it) and tried to re-encode it with x264 slower -not in real-time using handbrake, following @FerretBomb advice from a few days ago. but from some reason no matter what I try, even x264 slowest/placebo don't look that great. trying to encode with VP9 produced much better quality but took much longer time to render. I think I'll be going for the VP9 eventually. I was still learning about bitrates and file sizes of 1080/60p for streaming on demand and how it's being served from a website like youtube and got a little lost there too actually.. (: but i'll get there..

Thank you so much for your help!

Roy.

 

[koala]

If you intend to record, postprocess and later upload to a website, you don't have the constraint to record live the best quality and the best compression ratio at the same time.
The best compression ratio is x264 with the crf rate control.
The best quality can be achieved with every encoder (except AMD VCE), it's just the file size that has to be inflated enough.

So I propose you record your videos in simple output mode with nvenc as encoder and "indistinguishable quality" as quality setting. This will produce videos you will not really be able to distinguish from the raw source.

In a postprocessing step, you cut your material and render it with the encoder that produces the best size/quality ratio, that is x264 with CRF. The x264 crf parameter defines the quality, which you have to set to a value that makes the video small enough but still retain as much quality as possible.
Rule of thumb: decreasing the crf value by 3 about doubles the file size, and increasing the crf value by 3 about halves the file size.
Another rule of thumb: if you have 2 videos, one encoded with nvenc CQP (this is what simple output mode will use for recording) and one with x264 CRF, and both have about the same quality, the one encoded by nvenc is about double the size than the one encoded by x264.

Having said all that, it might or might not be advisable to use CRF for a video intended for streamed download from a website. You might encounter pausing and buffering in high motion scenes due to the variable bitrate. You might be forced to use the CBR rate control and give a fixed bitrate instead of CRF with a quality parameter. In this case, think about the available bandwidth of the persons who will download and watch the video and choose a not too high bitrate. If you choose a too high bitrate, the video may be pausing and buffering during view.