Capture Frame Data - Output Pixel RGB

[Michael Vo]

Hi!

Project: I would like to capture the frame data from a video source (game capture device) and simplify it down to pixels and their RGB color. I can use this data as a crude way of recognizing what is happening on the screen.

Questions:

• Is something like this even achievable?
• What areas of the code should I be looking at?
• Would something like this be a plug in / module (obs-module.h) or an output (obs-output.h)?

Background:
I am a software developer (mostly web). I am comfortable with compiled languages like Java, C#, as well as interpreted languages like PHP. I don't typically write C code besides some Arduino micro controller stuff, however I am not afraid to dive in and read, research, learn, and experiment!

Thank you for the help!
-Mike

 

[dodgepong]

The main issue is that OBS renders each frame on the GPU VRAM, and moving that back out to system RAM for you to read it is very slow. It would be better for you to write a program that could take the FLV output from OBS that is written to disk when you record and analyze that.

 

[Michael Vo]

Hi Dodgepong,

With that approach in mind, is there a way to just take jpg screen captures of a live video source at specific intervals? Then I can just analyze the appropriate image files?

Can the FLV output be processed live while it is being written?

Thanks!
Mike

 

[dodgepong]

OBS doesn't have a screenshot function, no, but you should be able to read in an FLV stream. That is what they are built for, to be decoded in a streaming context.

 

[Michael Vo]

Hi dodgepong,

Thank you for your direction! With some research I realized I can achieve what I want using FFMPEG while the stream is recording to the .FLV file. That combined with some ImageMagick and I have what I need!

Thanks again!
Mike

 

[dodgepong]

Cool, would be interested to see what you came up with...I've seen a number of people wondering if they could accomplish the same thing.