Support Splitting MP4s to Help Reduce Corruption

[FartyGuy101]

Hi, the title might be a bit confusing but I'll try to explain. First, I love OBS and I think it's a wonderful program, to which I would love to support financially someday. :-)

I'm currently using a computer that occasionally hard reboots, or shuts down on me. This is annoying, but I'm poor and can do nothing about it until I find a source of income.

Sometimes, I will stream for 5-10 hours at a time, and most of the time, I don't crash. However, that one time I do crash, really sucks because it corrupts my entire 5-10 hour MP4 file.

If OBS had an option to split MP4 files after a certain amount of gigs, or a certain amount of time, I would have several perfect MP4 files... and the very last one would be the only corrupt one.

As of currently, the entire 5-10 hour MP4 file corrupts, which makes it a nasty effort to repair, and so far I've had no luck.

Also, splitting MP4s could be useful for other reasons. Perhaps, editing in popular editing software would be easier, because chunks could be dragged into the timeline, rather than 1 huge massive 10GB MP4 file.

Lastly, this *might* help with audio/video sync issues. I've noticed that after about 6-7 hours of streaming, the audio will desync by about 1 second. This only happens occasionally.

I do, of course, suggest this feature be OPTIONAL, because some may not want to use this, which is totally understandable. :-)

The reason this is important to me is because I do 3d game design in various programs, and I like to make timelapse videos of my progress for youtube. If only 10% of the entire stream is unrecoverable, that's ok. :-)

I'm aware that I'm asking for a feature, that would be unnecessary if I had a more stable PC, and the solution is to fix my PC. Please, don't patronize me for this. I'm well aware this request is backed by unusual logic.

Hopefully this is considered. :-)

Thanks!
-Neil

 

[R1CH]

Save to FLV instead.

 

[Videophile]

Save to FLV instead.

I think you should recommend this in OBS. I use to save my videos in MP4, but whenever OBS crashed, my stream crashed or anything happened, the file was corrupted. I only recently learned about what FLV can do, now I only record in FLV.

 

[Jack0r]

Why should I record in flv format?
OBS by default has flv selected by the way.

 

[FartyGuy101]

Thanks. I crashed again today, but this time I was able to mux the corrupted FLV into an MP4 to recover seekability.

Is there anyway you can make MP4 behave the same way as FLV? Some programs allow storing the MP4 information at the beginning of the file, not the end. Would this help prevent corrupting MP4s?

 

[Jack0r]

Are these live recording programs? I guess you are talking about a software that reencodes a finished video and thus has the necessary informations at the start?

 

[GodlessGeek]

Are these live recording programs? I guess you are talking about a software that reencodes a finished video and thus has the necessary informations at the start?

Yes, the video has to be finished. I believe he is referring to the moov atom in mp4 files, which contains the header information. It must be placed at the beginning of the file to enable progressive downloading. Otherwise, a video player can't start playback until the entire file has completely downloaded.

At one time, most encoding software placed the moov atom at the end of the file. You could run the file through a program called MP4 FastStart and it would move it to the beginning. Nowadays, many encoders have an option to place it at the beginning. Apple's Quicktime has an option for it. Sony Vegas and Sony Movie Studio Platinum allow you to insert the text command "qtfaststart" at the beginning of your video. And the Handbrake program allows you to set the "web optimized" (or something like that) option.

 

[inkBot]

I support the idea of having an option for OBS to split recordings, but for a somewhat different reason.

I use OBS to record lossless video and those files become huge if run at longer lengths. So huge that they become unworkable. Being able to set up OBS to split recordings into chunks of [X]MB Could be alterable for personal needs and wants. For me 5GB would be a good size to split at.

 

[ZeroWalker]

Wondering, as this is similar to my other question and same answer.

What about MKV?
It has Stream support, so should it also work like FLV?
Cause i know you can play from an MKV file while it's writing to it, and at some points the length may even work (guessing it depends on what kind of frame was last written).

 

[8BitCerberus]

So I've been using FLV lately to keep longer recordings from being corrupted, but post production time almost triples with them since nothing really out there for video editing can work with FLV directly, not even Premiere Pro which you'd think would be a no brainer on Adobe's part.

So instead after a recording session I have to extract the H264 and AAC out (doubles the space required) then merge those back into a single MP4 (now tripling the space needed, though I can afterwards remove the intermediate files, I keep the FLV around just in case something went wrong merging the MP4 and I don't find out about it until editing time) and that process can take about half the time it took just recording the session.

Does anyone know of any video editing software that will work directly with an FLV container?

Otherwise I gotta throw in my request for an option to split MP4 files either by file size or by time. The reduction in post production time, alone, would be worth the minor hassle of stitching the pieces back together on an NLE timeline. I did some recordings early on with Shadowplay and noticed it was splitting files at 4GB, which translated to about 11 minutes (1080p/60 @ 50Mbps), with OBS that'd be about 30 minutes (1080p/30 @ 25Mbps because I don't really see a need for more than that with YouTube at the moment), so an option to split at 10 minutes, regardless of file size, would be fantastic. If a piece gets corrupted, it's only 10 minutes missing, not the entire 1+ hour(s) session.

[edit]Jack0r, do you know how long ffmpeg takes to convert the FLV to MP4? Sounds like it's on-the-fly, which should be somewhat faster than the two step process I'm doing now, extraction, then re-combination. Does it at least hit the CPU and make use of all your cores? I think that's the big problem with what I'm doing now, it seems like they barely hit the CPU at all, and the result is taking far longer than it should to get the job done. I'd gladly have all 8 cores maxed out for 30 seconds with ffmpeg versus 1+ hour process with only a 5% CPU usage.

 

[Sapiens]

@8BitCerberus Remuxing with ffmpeg sounds like it would be faster and less painful, though it would still double the storage space needed.

 

[R1CH]

Converting FLV to MP4 is entirely I/O bound. You should use different disks for reading and writing if possible. If you're serious about video editing, a couple of RAID-0 scratch arrays are very useful.

 

[8BitCerberus]

I'm not quite at the level of serious video editing, yet, but I do have plans to set up a RAID scratch at some point. I actually had one going for about a month before one of the drives started dying (it was a ~7 year old pair of drives so I kind of expected they wouldn't be lasting much longer anyway). I just recently had to update all of my storage HDDs, so that put a slight damper on the RAID plans for the moment.

Thanks for the replies, and I'll be doing a recording session later tonight so I'll give ffmpeg a go to see if that helps speed things up a bit. Even I/O bound it's got to be faster than the two-step conversion I've been doing.

 

[8BitCerberus]

So I can confirm that the FFMpeg approach is significantly faster. I just converted a 3 hour session (didn't get to record the other night) in about 3-5 minutes where the previous two-step method was taking a bit over an hour.

 

[FartyGuy101]

Just wanted to post an update on this thread after such a long time.

First, I've been saving to FLV ever since this problem has occured, and it has been a life saver. I've crashed numerous times over the months, and every single time, the FLV is easily recoverable simply by muxing to MP4.

For those of you who choose a complicated process to mux to MP4, stop. Use XMedia Recode and set Audio and Video to Copy. Check "Streaming - Fast Start" Set output to input directory. It's super fast, it doesn't create intermediate files, and it works, literally every time. I've been converting my FLVs back to MP4 files, and so far it has never once failed, or created a corrupt file.

You can even use Xmedia Recode to splice little segments of your video for posting to dropbox or youtube by simply choosing a start and end timecode from the preview tab. It's fucking easy. (Excuse my french) :)

Also, dragging an FLV file into Sony Vegas, or some other video editing program is bad. Don't do it. It will make your video editing take forever, and might even fail altogether due to unforeseen issues. Always convert FLV to MP4 before dragging the files onto your video editing timeline, you'll be amazed at how much more responsive your video editor will become.

But this really begs the question - why is FLV so much better for recoverability in case of corruption, whereas MP4 is not? Is this due to negligence on behalf of the OBS developer (no offense)...or is this due to some other reason, such as MP4 and FLV working differently? Would putting the header information at the beginning of the MP4 file - allow playback/conversion of corrupted mp4 files? If so, can you please implement this if it hasn't been implemented already?

The reason is, bypassing FLV altogether would cut out the need to mux back into MP4, which would save countless (combined) hours of unnecessary waiting.

Again, appreciate all of the hard work that's put into OBS. :) I look forward to it's continued development and improvement.

Thanks!

 

[dodgepong]

FLV and MP4 are different containers for h.264 video + AAC audio. FLV's container is very simple and designed to be stopped "on the fly" since it's for streaming. MP4 is not designed this way, and has a lot more data that is included in the container. When a video file is finished encoding and is being saved as an MP4, there are a few last-minute things the muxer has to do to "finalize" an MP4. If the program crashes before that can happen, it can corrupt the MP4. That is why FLVs are resilient to crashes and MP4s are not. It has nothing to do with negligence.

 

[FartyGuy101]

What about using the wkv/webm container? Would that be a viable solution? - or could the MP4 container be augmented to behave like an FLV - much the way AVI has been augmented over the years to do things it was never intended.

 

[Osiris]

OBS multiplatform can already save to a couple of different containers. I doubt any new ones will be added to OBS classic.

 

[Jack0r]

There are differences between the flv and mp4 container which lead to the problems. For example you might notice that skipping through an flv file can take time in a video player while skipping through an mp4 is pretty fast.
As far as I know, for an mp4 file to work, it has to save variables like the length of the whole recording at the end of finishing the video. If this information is missing, you have the problem of not being able to use the file. (That is what happens if your PC crashes for example. The file is not properly closed) Now the flv file does not need this information and if a software remuxes the file it can write the information after going through the whole file as it then knows its complete length.

As Osiris mentioned, OBS-MP supports different output formats, but it still depends on the format if its crash save or not.(We also have to support the specifications of the different containers or you wont be able to watch the videos anyway)