OBS output delay

[jason ki]

Hello,

I'm having an issue where there is about 1.1 second delay between output versus displayed media. I'm not sure if this is a bug with some sort of timestamping in the encoder. I've tested this using some intricate trigger test using photodides.

can someone guide me to understanding how the obs encodes the output.

 

[dodgepong]

What do you mean by "output vs. displayed media", exactly? Are you comparing the OBS preview to what is being displayed on stream and noticing there is a delay to the stream?

 

[jason ki]

What do you mean by "output vs. displayed media", exactly?

Here, I'm referring the output as the recorded .mov file, and displayed media, I'm referring to the OBS preview.

Are you comparing the OBS preview to what is being displayed on stream and noticing there is a delay to the stream?

And for the second question, i'm comparing what is displayed on the display monitor using OBS preview to the recording that was output to file? I noticed that there is a around 1.15 second delay.

In my experiment, I play a 180 second video with 30fps which contains white "flash" frames every 30th frame with rest of the frames being black. I play this video using the media preview on OBS as I concurrently record in OBS.

To perform a definitive comparison of what was played on the screen to the recorded output, I record the flash activity on the monitor using a photodiode. At every flash, the photodiode writes out a marker to the disk. For the OBS software, I wrote a modification in the line of code below in obs-output.c file which sends a frame count of the video using. The two marker modalities are sent to the same output file using lab streaming layer (LSL, a C API for sending time events). This system ensures that all markers are time stamped and aligned.

static inline void send_interleaved(struct obs_output *output){
...
if (out.type == OBS_ENCODER_VIDEO) {
        output->total_frames++;
        double frame_marker[1];
        frame_marker[0] = (double) output->total_frames;
            send_lsl_trigger(&obs->obs_lsl_global->outlet, frame_marker);
        }
}
...
}

After the recording, I do some post processing on the recorded file to index the frames where the flash occurred in the mp4. And based on the markers above, I calculated an average difference of 1.15 fixed delay in the recorded output. I've attached a simple plot displaying the trigger occurence.

 

[jason ki]

Can someone help me with this issue?

 

[dodgepong]

Video isn't encoded at the same moment that it is rendered to the screen. The encoding process happens after rendering, and there are some internal buffers in OBS in the encoding process that are necessary to make sure everything stays in sync. What are you doing that requires the rendered image to be written to disk at the exact moment that it is displayed?

 

[jason ki]

Video isn't encoded at the same moment that it is rendered to the screen. The encoding process happens after rendering, and there are some internal buffers in OBS in the encoding process that are necessary to make sure everything stays in sync. What are you doing that requires the rendered image to be written to disk at the exact moment that it is displayed?

Thank you for the reply!

We are doing a neuroscience experiment where we have subjects play a video game (supertuxkart) while we record their brain activity. We are hoping to use OBS to do screen capture of the gameplay. Brain response has high temporal sensitivity to visual stimulation ( < 10ms), our hope is to align the video displayed on the screen with eeg. So I'm sending trigger mark for every screen that is captured.

 

[jason ki]

If all goes well, we were thinking of releasing a plugin dedicated for the neuroscience use cases. I find OBS perfect system for presentation and recording modality for neuro/ cognitive experiments. It is a wonderful software overall in my opinion.

 

[Lain]

That internal latency of OBS itself is normal, especially when accounting for h264 (which always operates about 20 frames behind), audio latency, and buffered video frames on the GPU. It's not a bug -- OBS has to have latency to be able to do any sort of processing; rendering, capturing, and especially encoding and audio processing. OBS having approximately 1 second latency between the captured video data and the output video data is completely normal and expected.

I might not be fully understanding what you're doing here -- if I'm reading correctly you're outputting metadata along with the video frames associated with that flash or something. The data in the portion of code you're modifying is always going to have a delay (or rather latency) because it's in the encoding/output pipeline, not the rendering pipeline. You will have to store a system timestamp of your event (with os_gettime_ns right as the event happens) and then compare that time to the time of the video frame (encoder_packet::sys_dts_usec * 1000), and then wait until you have the frame closest to that timestamp, and then output your metadata for that event. Or store it in the frame data as metadata pretty much right as it's rendered and have that metadata go through the whole graph, which would probably be more difficult.

Don't try to fight against it -- instead, work with it because it's completely normal. Just store a list of your events and wait for the frames to catch up to those events, and then output your metadata. Fighting against buffers will only cause pain. You will never get 10ms latency out of something like OBS, because there has to be buffering of things from capture to output. A single 60FPS frame is 16.6 milliseconds -- one frame of buffering alone will shatter that desire by itself, so trying to fight against the internal latency of the subsystems is completely futile.