Any reason to record a stream with a faster bitrate than "stream" ?

[Meanmom]

Hi all, my first post here!

I just found OBS a few days ago, and wow i'm blown away. Went thru the usual challenges like the "encoding overload" which gave me cause to clean my windoze down to bare metal hah, all is well now. Absolutely love simplewall (replaces windows firewall) now I can see every time anything tries to access networking. Followed this brilliant piece of work here to clean up windows:

How to Speed up Windows 10 (Effective Methods)

Use our effective ways to speed up Windows 10 PC. From basic to advanced steps, we have covered everything to make your Windows 10 PC faster.

beebom.com

Anyways, I've been testing OBS (totally thrilled btw) recording a stream, and I'm playing around with the bitrate settings, to see differences in playback quality. I recorded a 1080p60fps stream at "stream", "HQ" (high quality, medium file size), and "IQ" (large file size) to compare. Saved those 3 mp4 files to my Emby server and watched them on my HD tv (via Roku), and I noticed something curious... all 3 look pretty much the same! I'm old, so maybe my eyes ain't what they used to be.

That got me thinking... if it records a higher bitrate than the stream (if it is recording just the actual stream), where are all of those extra bits & bytes coming from? Over sampling the stream? Why do that?

I'm a typical older frugal person, "waste not and want not", just wondering why people would oversample, since OBS can record the actual stream.

Thanks for reading, and I'd love to hear your thoughts on this!

Cheers

 

[Suslik V]

When recording quality in OBS set to other than "Same as stream" then another (second) encoder started that compresses raw data separately and saves them on the disk. OBS renders images that uncompressed at all, so before they saved or streamed they needs to be compressed. Thus, no oversampling - both encoders has raw data for input. The "Same as stream" setting - reuses streaming encoder (copies already compressed data), so almost twice less load to your hardware. With two standalone encoders it is possible to re-scale the data only for the stream to lower resolution (to 320x240 for example, in case of bad/busy network etc.) and save full resolution video (as is) on the disk to upload it later manually.

 

[konsolenritter]

@Meanmom

Its like this: If you're about to drive by your car 600 miles, then you would say "Oooh, the quality of these old stereo cassettes work for me. The motor is that loud... the old tapes last for that."
If you go to your basement studio with friends making music, you would say "Lets take the best tapes we can get to catch these epical moments forever." (analogue example given by intention)

The thing with the digital domain is this: Digital compressed storage (like here for videos) isn't really lossless, quite the contrary.
For streaming the people go for the best bitrate that their connection and provider can handle safely and stable. Slogan: "More quality isn't possible for going live." For recording on the other hand you would go a little bit reputable and conservative, because you say: "Possibly i'm going to edit and cut this recording in a couple of days for future use." - So while every stage of editing compressed material enforces further loss of quality (each stage enforces de-coding and re-encoding), you want to start with a good one quality level, so high bitrate to think ahead of possible loss. (Back to analog: If you copy from your answering machine five times, you will barely understand the word. If you copy from good studio tapes five times, your chances are quite better. ;) So if you know that you have to copy and spread the word, you wouldn't start with an answering machine tape.

The biggest difference in used bitrate (sure, ...the disk space... =D ) - but no, the biggest difference in used bitrate will be seen on really demanding material. This never means steady or calm picture content, but fast movement and lasting pan shots eat up bitrate really fast!

(To tell a secret - one of the most demanding test cases for video algorithms - aside of modern gameplay - is fireworks on dark sky: fast spreading pixel movements allover the picture, permanent changes in most regions of the picture.)

 

[Meanmom]

Thank you both @Suslik V @konsolenritter for the response!

If I understand this right...

When recording using the "stream" option, it's really just a minimal raw sampling (that is not loseless), but during the encoding process, more loss occurs, before saving to disk, and then later, when decoding the saved file, more loss occurs. So your first (and only) chance to capture as much as possible, to minimize loss during encoding and decoding (and transcoding when converting to lower resolution), the more you initially capture, the less loss during actual final playback. Thanks, that makes sense, if I got that right.

So choosing how much "oversampling" in the beginning is all about preventing data loss later. If you have limited bandwidth (transmission), disk space (storage), or cpu/gpu (encode/decode/transcode), then resource limitation becomes the priority. If not limited in resources, then preventing loss becomes the priority.

So for practical purposes, recording a stream at the "stream" level is the absolute minimum data capture, which is basically better than nothing. You'll get enough audio and video to see the original stream, but it won't be as sharp or clear, especially during extreme picture changes. This setting is good if you just want to watch the event once, like a news broadcast, social gathering, church sermon, etc.

Capturing at the HQ level "high quality, medium storage" is an excellent attempt to reasonably preserve the original data, even after the encode/decode/transcode process. However, extreme changes in video are likely lost forever. This setting is good if you or others will watch repeatedly, but can live without the finest detail.

Capturing at the IQ level "indistinguishable quality, large storage" goes the extra mile to ALSO capture the extreme changes (such as fireworks on a black sky) with greater chances. This setting is good for preserving a great movie you will enjoy over and over on the big screen.

Lastly, the LQ level "lossless quality, tremendously large file size" is grab everything, let God sort it out.

So, it's a judgement call. For most situations, where resources like disk space, bandwidth and cpu/gpu are reasonably available, it boils down to how bad do you want that extra bit of detail.

Which brings me to my next question..................................

I noticed OBS prefers to record a stream to mkv first, then afterward, convert to mp4. I take it recording to mkv format is easier/faster/less loss? I guess I need to read up on mkv, never really used it.

Thanks again, appreciate the discussion!

Cheers

 

[konsolenritter]

Nearby all you said is right.

Differences are:
Recording using the settings "like stream encoder" means half the processing power, because the recording will just be written from the multiplexed stream that is encoded for streaming (one encoding to two targets). So you are bound by the restrictions given by the streaming environment (upload speed, streaming provider,...) Using different settings for stream and recording enforces encoding two times. But with appropriate hardware this isn't a problem. The modern gpu graphics have typical more than one encoder, so it means two encoding processes without doubt in the same time.

Oversampling is technical a complete different thing. It has nothing to do with encoding here. Encoding is (in any form) an algorithm following the question what information from the raw data can be removed while maintaining (a certain degree of) visual quality.
Encoding is a oneway street (or better highway?) from raw data to compressed data. Decoding is vice versa.

The mkv thing is as follows: Its the "final" file format. The encoder encodes the streams, the multiplexer brings encoded video and audio together and then its saved. The mp4 format is good, but has a clear drawback: Part of the needed metadata are written into the file at closing time (when the generation of the file is complete). If your recording lasts long, and there are certain possibilites that your recording is broken (due to freezes, unwanted shutdowns of software or machine, running out of space,...) the mp4 file will be broken and unusable without professionel data recovery.

The mkv file format is more provident for such cases. The type of encoding and size will be absolutely equal (or comparable regarding size) but there are needed metadata written in between or refreshed in the header during the recording. So whatever unconditionally brings down your recording, the mkv file should keep readable. So its best to record into mkv format and - after ending the recording - can be remuxed very fast into the final mp4 format by the obs menu top left.

 

[Suslik V]

The .mp4 container for media files in OBS was selected as most popular (most supported in video editors).
Later, it turns out that the most popular implementation of this container has disadvantage of storing some very important data in the RAM of the PC until the file writing was complete (finalized). And thus, in case of power failure, BSOD or program crash the whole recording (many hours) became lost.
Second most popular media container to store the video and audio data was .mkv format. It was robust. So, it was chosen as default in OBS.
Today, OBS records into .mkv and for users who prefer old good .mp4 files it can re-multiplex (remux) recordings into .mp4 format.

To record directly into .mp4 with minimal damage to the file in case of the failure you rather need to look at the: Stopping recording never ends

...the Muxer Settings as:

movflags=frag_keyframe min_frag_duration=16000000

will help to withstand the crash.

If your editor doesn't supports fragmented files (Muxer Settings above creates such files with fragment length 16 sec) you may remux recording in the main menu of OBS before using it in your editor.

and similar threads.

 

[Meanmom]

Great info, thank you both!

I do have a high end pc (fast intel cpu and fast radeon gpu) so not worried about performance issues, especially since I shut down all unneeded processes, and promoted OBS priority to above normal. Fast Internet and wifi too, no bottlenecks here.

Right now I'm testing highest quality of recording an inbound stream of 1080p 60fps and 5.1 audio, using the IQ setting "indistinguishable quality, large file". To improve quality, I also turned off gpu (AMD) utilization, both in OBS and the browser, so they both just use the cpu, h.264 software encoding. So far in testing, even over 20,000 kb/s bitrate, cpu barely tops 60%, and zero skipped/missed frames. I'll watch this version compared to the minimal "stream" version, and look for differences in panning scenes and such. Should be noticeable, if my eyes still worked good hah :)

Thanks again, I'm really glad I found OBS, it's amazing software. Developers can be proud. They earned it.

 

[koala]

If I understand this right...

When recording using the "stream" option, it's really just a minimal raw sampling (that is not loseless), but during the encoding process, more loss occurs, before saving to disk, and then later, when decoding the saved file, more loss occurs.

This is not the point of the stream option. The difference between streaming settings and recording settings is much much simpler.

Streaming needs constant bitrate due to network behavior and requirements. This means, the encoder needs to encode the video in a way that while you download and view the video, always the same amount of bytes per second are transferred. This is called constant bitrate (CBR) rate control. Since high motion parts of a video needs more data and low motion parts of a video needs less data to encode, the encoder is removing detail from high motion scenes to squeeze the larger data within the narrow bitrate requirement for streaming. Low motion scenes can have less detail removed.

Recording doesn't have this constant bitrate requirement. The local hard disk has nearly unlimited throughput and unlimited space if it comes to video processing. So it's possible to encode high motion scenes with more bitrate than low motion scenes and have both with the same quality. This is called constant quality rate control (abbreviations vary: CQP, ICQ, CRF). The overall quality is much better than a CBR encoding.

This is the whole difference. You can of course use streaming settings (CBR) for saving a local recording, but you waste quality this way. You will do this only, if you want to stream and at the same time save this as recording to disk and your machine isn't capable to run 2 encoding sessions at the same time. If your machine is powerful enough, you can use 1 encoding session with CBR for streaming and a second one with CQP for recording. If your machine isn't powerful enough for 2 encoding sessions, you can re-use the compressed data created as CBR for streaming and save this as recording to disk.

If you use simple output mode of OBS, OBS automatically chooses CBR for streaming and CQP for recording. Exception if you use "use stream encoder" for recording, you re-use the streaming data. But if you don't stream in the first place but record only, you don't need to (and should not) choose that setting.

 

[Meanmom]

@koala

That makes perfect sense. I love technical detail like that. I'd like to learn more, if you can point me to some good reading, I'd appreciate it.

Thanks

 

[Vaesive]

This is not the point of the stream option. The difference between streaming settings and recording settings is much much simpler.

Streaming needs constant bitrate due to network behavior and requirements. This means, the encoder needs to encode the video in a way that while you download and view the video, always the same amount of bytes per second are transferred. This is called constant bitrate (CBR) rate control. Since high motion parts of a video needs more data and low motion parts of a video needs less data to encode, the encoder is removing detail from high motion scenes to squeeze the larger data within the narrow bitrate requirement for streaming. Low motion scenes can have less detail removed.

Recording doesn't have this constant bitrate requirement. The local hard disk has nearly unlimited throughput and unlimited space if it comes to video processing. So it's possible to encode high motion scenes with more bitrate than low motion scenes and have both with the same quality. This is called constant quality rate control (abbreviations vary: CQP, ICQ, CRF). The overall quality is much better than a CBR encoding.

This is the whole difference. You can of course use streaming settings (CBR) for saving a local recording, but you waste quality this way. You will do this only, if you want to stream and at the same time save this as recording to disk and your machine isn't capable to run 2 encoding sessions at the same time. If your machine is powerful enough, you can use 1 encoding session with CBR for streaming and a second one with CQP for recording. If your machine isn't powerful enough for 2 encoding sessions, you can re-use the compressed data created as CBR for streaming and save this as recording to disk.

If you use simple output mode of OBS, OBS automatically chooses CBR for streaming and CQP for recording. Exception if you use "use stream encoder" for recording, you re-use the streaming data. But if you don't stream in the first place but record only, you don't need to (and should not) choose that setting.

Apologies for the necro but just looking for some explicit clarification: When you are talking about "use same as stream encoder" are you saying "as long as the settings of the stream and recording outputs are the same (in advanced mode) it'll use a single encoder" or is is strictly the "simple" mode with "use stream encoder" only. I ask because the simple mode does not have the Lookahead and Psycho Visual options for NVENC so I cannot set these settings unless I am in Advanced mode.

I'm running a 10900 and a 2080ti and have been reliably using 1080p FPS stream and record simultaneously flawlessly for months now until the latest OBS update that changed the NVENC encoder settings to the P1-P7 format and it's suddenly struggling to do 720p 60fps reliably -.-