Question / Help Webcam Mic Not in Sync (Solution?)

[ZeroWalker]

My Webcam and my Mic is not in sync by 100ms.

Now i wonder, what is the solution for this:

1: Delay the Mic 100ms to make it in sync with the Webcam
2: Minus Delay the Webcam 100ms (Can't be done currently?) to make it in sync with the Mic


I am guessing i should sync the one which is truest to realtime.

In this scenario, i actually thing the Webcam has quite a delay on start, which means it's the Mic which is closer to "Reality" then the Webcam, but sadly i can't Minus Delay the Webcam picture, or rather i can't skip about 3 frames.

 

[Jack0r]

Yup, delay the microphone. A negative buffer is not possible if I remember correct.

 

[FerretBomb]

Unfortunately, while you can negative-buffer the mic (if the cam is too fast) you can't negative-buffer the cam. Per Jim it'd take up too much video memory to keep a second of uncompressed video with full layers in VRAM.

Still kind of hoping this gets added (even just to the rewrite), with a warning about it eating video memory. 100-300ms (3-10 frames at 30fps) seems like it'd be workable... even up to a full second or two maybe with larger memory video footprint cards (seriously, does anything even USE 4GB of textures at the moment?). Doubt it'd be able to be spooled off to system memory (bottleneck the hell out of the bus probably, with the bi-directional traffic) but if it was, it'd be INCREDIBLY handy... webcam delay is just long enough that you sound behind when singing along with any in-game music, which is maddening when you have to keep forcing yourself to stop.

 

[ZeroWalker]

Why would it eat memory, can't it just discard 3 frames, and then let it run freely?

I am pretty sure VirtualDub does that, it looks at the timing on the Video and Audio, and if the Video starts before the Audio or something, it will just duplicate the last frame until the Audio is at the same time, then run both so they are in sync.

No need to save the first 3 etc frames in memory for the rest of the recording.
It just needs to wait until everything is at the same time, then begin recording.

 

[FerretBomb]

Because this isn't just audio sync. It's compositing and then encoding time-shifted video sources together in real-time, with live sources instead of preexisting dead files. It'd need to 'hold' the game and any other sources in VRAM, wait for your webcam's video to come in, then composite the other video layers from however long ago (in-game, other sources like Skype/videos/etc) with the incoming webcam feed.

There's a pretty major difference between what VirtualDub is doing, and what OBS is doing.

 

[ZeroWalker]

Ah, so it can't just Blank the entire video layers, wait for the Webcam to come into the same time, then start the recording?

Thought it would be possible, but then again, as you say it must have continues information flow of several layers.

What could be done however is to have a Temp file with the Webcam and write there, instead of using VRAM, and just write, let's say 1 sec of video, over and over (Similar to Fraps Continues recording thing).

And copy that to the Video recording layer.

However, it seems a bit inefficient, but you would be able to prevent using VRAM.

However, as you said earlier, a webcam used for gaming would be like 640x480/320x240, and that is extremely love in size even when uncompressed, and some frames of that in the VRAM is next to nothing.

However, if you are doing some serious stuff with 1080p then it would be more of a problem, but even then it wouldn't really matter as long as it's not more than 1 second.

RAM and VRAM are about 2-8gb these days, and 1-100mb of just keeping Video data is nothing really, but it is indeed wasted space no matter how you look at it.

 

[FerretBomb]

It doesn't work that way. Again, you're not dealing with a dead file that you can just look ahead on, it would have to retain those frames in memory. And not just any memory, but video card memory.

Again, you can delay the WEBCAM just fine. You can add a positive offset to this without any problem. The issue comes in when trying to put a NEGATIVE offset on a webcam... since the frames don't yet exist, OBS would have to keep all the frames between the active gameplay and the webcam's negative buffer in VRAM (as it doesn't write to system memory, or even slower to disk) in uncompressed, un-composited form to allow the scene to be properly flattened and then encoded for each frame as it came in from the webcam.

Again, it's not a problem with keeping the webcam in memory and adding a delay onto that (after all, that's already coming into system RAM and can just be buffered right there). It's keeping EVERYTHING ELSE in memory (especially the game video which never touches system memory until it's already been composited into a completed scene) to time-shift it forward that's the problem.

So yeah, I have 48GB system memory. As the game video is being stored in my video card's memory and never leaves until it's in a composited, final frame, it really doesn't help me. Spooling the game video off to system memory would just flood the bus in both directions... once writing the frame out to system memory, and once reading it back in for GPU accelerated scaling/compositing.

 

[ZeroWalker]

I see, so it would need to save every information available for that negative offsett, all captures that's going on?

If so, that that indeed will be quite the problem, still it will work, but it will be quite a hit to always retain all that in the VRAM and flush it every new frame (meaning if it's 3 frames behind, on the 4th flush the first etc).

I thought it could use the HDD as a temp solution if the VRAM didn't work, but it would probably quite the inefficent all around way to do something like that.

Hmm, would have been one thing if everything was handled differently, but then again, the system doesn't work that way so not much to do there.

Just thinking loud now, but what happens if you positively delay the Gamecapture and audio to make it sync with the webcam, instead of the other way around?

I think it will restult in the same thing, but as you said, positive = good, negative = bad, as the words originally means funny enough;P

 

[Jack0r]

The option to add positive delay to "any" source might be available in the rewrite at some point, it will definitely be possible to add the feature to obs at least, if someone invests the time to do it.

 

[FerretBomb]

There's no 'flushing' involved. Again, not VirtualDub. Doesn't work the same. At all. Two completely different things. Forget VD. It'd remove the buffered frame-layers from VRAM after each frame was composited and moved out to system memory as a finished frame for compression by the CPU.

Using the HDD would be INCREDIBLY slow, trying to write a buffer of 30-60fps at full resolution, then re-reading it back in, even with an SSD. Again, if system RAM is too slow (and it probably is, due to the PCIe bus bottleneck again, even if you have speedy system RAM) then a HDD isn't even moving. On-card VRAM (rocket), system RAM (jet), HDD (guy with one leg hopping down the road) as far as speeds go.

Again, that's exactly what we're talking about up above. You would HAVE to buffer the game capture, all layers, and the system audio forward to match up with the webcam (keeping the frames to-be-synced in uncompressed, uncomposited form in VRAM until the webcam frame was available), unless you have a wormhole camera that can literally see 300-1000ms into the future (again, it's a LIVE CAM, not a video feed that you can just read ahead as with video editing software).


Don't get me wrong, I'd love to see this for those of us with large-memory video cards, and even moreso a test build that would buffer the frames to/from system RAM just to see how much of a hit it would really incur. But assuming 1080p@30fps, that's 178MB per second (assuming 30fps, 24-bit color depth, 1920x1080 base resolution) per layer assuming someone isn't clever about only providing the full workspace for layers that are actually that size, and don't just keep static layers (overlays, watermarks) in memory as a reference item. Personally I'd be more than happy to give up half a gig of VRAM for three seconds' video sync delay (especially as in my case I only actually need about 300ms, so maybe 70MB of VRAM for a single game layer, and perhaps another 30MB for other static assets and game captures). Doesn't really sound like much, but apparently the logistics behind it would be prohibitive.

It'd take work to put it in too, and Jim is kind of working on the rewrite and has a bunch of bugs to otherwise stomp.

 

[ZeroWalker]

Jack0r, Great, really like how the aim for it is Modular making it future proof!

FerretBomb, with "flushing" i simply meant discarding, as you won't have the entire video in the buffer, meaning if i play 1 hour, it won't have 1hour - the delay, in the buffer, that would make it impossible.

I will have to have the buffer , the layers as you say, in the VRAM, which would take, for webcam and game capture, i guess, webcam size + game size * frames, or something, and if more layers are added, it will be even more of course.

And yes i know HDD or even SSD will be extremely slow, i just meant that if VRAM would be impossible because of space, HDD temp may be possible, but you would probably need to have quite the way to do that, compression must be part of it, as uncompressed read/write will be impossible, and even with compression it would need quite the system and quite the waste resource wise still.


Yeah noticed after i wrote, i thought that if you just did something like this:


You have a layer with the webcam which needs, let's say 500ms to be in sync (negative),
And you have a Monitor layer which is always in sync (nothing needed, 0ms).


I though, why not simply do like this.

Start the webcam on Recording, let it record and save like normal, let the Monitor layer be blank (Nothing there, no image, no buffer etc).

When it hits 500ms, then activate the Monitor layer.


When i think of it fast, it sounds like it will work, but as you say, it won't, ant with that is my limitations to understand the complexity of the system itself.

 

[FerretBomb]

Again, you're thinking in virtualdub terms. This is NOT dead-files. It doesn't work like that. Time won't 'catch up' with itself. As soon as that monitor layer goes live, it will be showing the CURRENT video on the monitor capture, with zero delay, regardless of how much black you just show. Again, FORGET video editing. Doesn't work like that in real-time, because we literally cannot look into the future and pull back stuff that hasn't happened yet (as you can with editing dead video files, with relative time).

 

[ZeroWalker]

Yeah i know, just said that when i think it at the time, it sounds like it will work, but as you say and explain, it doesn't.

Offtopic, i see you have c920 webcam, how does it work in darkness with just the monitor?
I got c525 and it's not that good, making it impossible to see yourself if the monitor is dark (Dark games etc).

 

[FerretBomb]

No matter what, you'll need a good lighting setup unless you go for an IR camera (I use one of those for horror nights) which isn't generally an option as with IR there is no color, just a black and white image (which has literally scared some of my viewers away on its own, 3spooky5them).

There's only so much a cam can do with gain amplification and exposure increase (which will lower your refresh rate just on its own). Spending on lights is at least as important at what camera you get, and you aren't going to get anything but crap results on any cam setup with just the light from a monitor, until you get into the professional (read: multi-thousand-dollar) arena. Probably not even then, without some major glass.

 

[ZeroWalker]

Ah, thought so, but i know some are extremely bad dark, some are better, but i guess ours are pretty much the same.

IR camera would be nice, but as you say, B/W looks scary in itself;P

Searched on "3spooky5them", didn't find anything though;S?

Yeah well have worked on that exposure part as well, and as you say, refresh rate goes down, and i don't really find it worth it, perhaps for some occasions, but 30fps vs 20-15 is just way to different.

Ah well that's good to know, will have to fix something, got a strong light, sadly it's directional, i try to mess it up with paper to make it glow around more, as i don't want to have a pure directional beam of light in my face;P

 

[FerretBomb]

3spooky5them = more than 2spooky4them.

Yeah, if your camera normally records at 30fps under optimal conditions, with half as much light it'll need twice the exposure to get the same image (or to raise the gain and introduce sensor noise), so 15fps. With just a monitor providing light, I'd expect closer to 3-5fps at best, including a gain amplification. Plus you'll probably smurf-face a lot, with the variable white balance coming off the monitor.

Use a reflector. Get a shiny silver car sun shade, point the light away from you, and use the silver material to bounce it back and scatter it. You can also deflect it and bounce it off just about any surface (I use a nearby white wall as a reflector). You can also use a diffuser box which will make it less intense. Both techniques will soften shadows on your face though, leading to less definition on-cam. You'll still want at least one light shining directly on your face to add sharp shadows for visual depth. That's getting into lighting theory though, and there are a TON of websites out there for photography/videography lighting technique. Spend a few hours browsing them, get some ideas.

 

[ZeroWalker]

Ah, now i get it;P
Thought it was some kind of username;D

As true as you say, it will be a slideshow showing a rainbow in form of a face.

Ah, will look it up, i have a white wall, but as it's to close it doesn't work as expected, i could perhaps use (http://www.sikkemaverpakking.nl/uploads ... klein1.JPG) not sure of the name, but it's Folie here.

Ah, though over lecture it may be good to know, you can never know to much at a given scenario, it's better to be prepared.