Grabbing frame from source (video capture device)

[Wesley Osborn]

I'm currently grabbing a frame with obs_source_get_frame but I'm having issues pointing to the array of pixels from there. The image is needed to process a facial feature live. To do that I need to pass the width(check), height(check), color format(check), and pointer to the pixel array(access violation). Any help would be appreciated.

 

[Lain]

I really can't recommend using that function to read async video frames. That function is really more of an internal function. Create an async video filter to read frames. See plugins/obs-filters/async-delay-filter.c for an example of an async filter.

 

[Wesley Osborn]

Thank you for the push in the right direction! cheers

 

[Wesley Osborn]

Video format returned is YUY2. I noticed you have I420 for packed and BRGA for uncompressed. If i wanted to make a return of the I420 inside of video-fourcc.c I could use the MAKE_FOURCC #define to bitshift in the correct position? Little confused on how the format conversion works.

 

[Lain]

The video format is whatever video format that's coming from the source. It can be RGB, YUY2, I420, NV12, etc. Any of the formats listed in libobs/media-io/video-io.h. There's no way to really control what it uses because it's only designed to convert to RGB for compositing, and that's not done on CPU either. If you need to use a specific format from there you'll have to convert it manually in your plugin. You can either use FFmpeg directly to do that or use it via libobs/media-io/video-scaler.h

 

[Wesley Osborn]

Appreciate the input. I was able to grab the frame data and then convert the format as soon as I received it from my filter. It was actually very simple to convert from YUY2 After doing some reading into formats, I noticed the format RGB was actually returned as RGBx (x meaning no alpha, an opaque image). Currently everything works as intended, thank you!
p.s. love your teams code structure

 

[Lain]

You're welcome. Glad to help. By the way, if you're thinking of distributing, I'm going to try to get a plugin manager hopefully for 0.16.0. I wanted it for 0.15.0 but it's not going to happen that soon. The plugin manager will simplify plugin installation.

 

[Wesley Osborn]

Awesome! I will keep an eye open for it.

 

[Marcos Rapoport]

Hi, i'm new to this forum and obs plugins development. I really want to intersept a frame, manipulate it, and send it back into the obs pipeline. I want for example to be able to do a sort of mirror filter, that mirrors de left half of the image on the right side of the image. How and when should I get the frame data? Should i call it from the render method of the filter? And how can i return it into the pipeline?

 

[Tam66662]

Hi @Jim and Wesley,

Can you please share the implementation, or a pointer to the conversion process for taking the YUV data and converting it to RGB? Any help would be greatly appreciated. I simply want to take the data from the obs_source_frame, and save it to disk as a png or jpg file. Just as an exercise to understanding the conversion process.

 

[Garvey]

I too would like some help here, I tried looking at the async filter, however I unable to even get that to work from a simple Display Capture source.

I just would like to create a filter that can read per frame pixel data from a display capture source, if possible