write captured frame to decklink SDI output

[kayvan]

Now, I am trying to get the captured frame then write to decklink SDI output,
Which interface or data struct should I use to get the frame before conding?

In the source code, i found List<VideoSegment> bufferedVideo but is used in EncodeLoop().

Any Suggestions are very helpful

 

[Lain]

What specifically are you trying to do? Take the raw data and output it? Honestly it's still quite difficult with this version. I would have to see an example of the code you're writing in more detail.

I would probably just recommend waiting for https://github.com/jp9000/obs-studio - because doing something like this in the new version is super easy and is designed specifically to allow new and multiple outputs

 

[kayvan]

Hi Jim,

Thank you for your timely response!

I want take the raw data and output it, here is the position I want to get the raw video frame.
OBSVideoCapture.cpp --> MainCaptureLoop() -->
if(bEncode)
{
......
if(SUCCEEDED(result = prevTexture->Map(0, D3D10_MAP_READ, 0, &map)))
{
memcpy(videoFrameBuffer, (LPBYTE)map.pData, frameWidth * frameHeight * 4);
.......
}
.......
}

Here is a question:
what's the color space of pData in struct D3D10_MAPPED_TEXTURE2D, BGRA or other?
Then, what is the size of pData? I just found that RowPitch is 7680.

typedef struct D3D10_MAPPED_TEXTURE2D
{
void *pData;
UINT RowPitch;
} D3D10_MAPPED_TEXTURE2D;

Any Suggestions are very helpful, thanks!

Kayvan

 

[kayvan]

Now, I found :
http://msdn.microsoft.com/en-us/library/windows/desktop/bb205319(v=vs.85).aspx

D3D10_MAPPED_TEXTURE2D mappedTexture;
if( SUCCEEDED( pTexture->Map( D3D10CalcSubresource(2, 0, 3), D3D10_MAP_WRITE_DISCARD, 0, &mappedTexture )))
{
    D3D10_TEXTURE2D_DESC desc;
    pTexture->GetDesc( &desc );
   
    // Compute the width and height of the third mipmap level
    const UINT WIDTH = desc.Width >> 2;
    const UINT HEIGHT = desc.Height >> 2;
   
    FLOAT* pTexels = (FLOAT*)mappedTexture.pData;
    for( UINT row = 0; row < HEIGHT; row++ )
    {
      UINT rowStart = row * mappedTexture.RowPitch/4;
      for( UINT col = 0; col < WIDTH; col++ )
      {
        pTexels[rowStart + col*4 + 0]; // Red
        pTexels[rowStart + col*4 + 1]; // Green
        pTexels[rowStart + col*4 + 2]; // Blue
        pTexels[rowStart + col*4 + 3]; // Alpha
      }
    }

    pTexture->Unmap(D3D10CalcSubresource(2, 0, 3));
}

I can get RGB picture but do this work is too Inefficient.

 

[Lain]

There should be nothing in OBS that has any mipmaps unless their width/heights are both pow-2 sized.

Those for loops feel rather superfluous, aka they don't do anything. I'm confused about what specifically you're trying to accomplish. If you're trying to get the texture, you should just do a memcpy for every row with width*4 as the size.

for  (UINT row = 0; row < height; row++)
{
    UINT outputPos = row * width * 4;
    UINT texturePos = row * mappedTexture.RowPitch;
    memcpy(output + outputPos, pTexels + texturePos, width*4);
}

Again, I would probably suggest waiting for obs-studio on this particular sort of thing, as it's insanely easy to get the full frame output from that via an output plugin.

 

[kayvan]

Hi Jim,

Thank you for pointing out my mistakes, I don't have enough experience on Direct3D Graphics. and I still want to use current code to finish the work in present stage.

Decklink SDI supports output formats:

enum _BMDPixelFormat
{
bmdFormat8BitYUV = 0x32767579,
bmdFormat10BitYUV = 0x76323130,
bmdFormat8BitARGB = 32,
bmdFormat8BitBGRA = 0x42475241,
bmdFormat10BitRGB = 0x72323130
} BMDPixelFormat;

how to get the raw frame which fits decklink from a mapped D3D10_MAPPED_TEXTURE2D ?

Thanks,

 

[Lain]

Have it use ARGB or BGRA and just basically do that code I listed above with a texture of the same format, unless I'm misunderstanding something

 

[kayvan]

Hi, now it can work, I convert the NV12 frame to UYVY422 by ffmpeg then write to decklink,
thank you for your help, we are waiting for obs-studio.

                    if(!bUsing444)
                    {
                        profileIn("conversion to 4:2:0");

                        if(bUseThreaded420)
                        {
                            for(int i=0; i<numThreads; i++)
                            {
                                convertInfo[i].input     = (LPBYTE)map.pData;
                                convertInfo[i].inPitch   = map.RowPitch;

                                if(bUsingQSV)
                                {
                                    mfxFrameData& data = nextPicOut.mfxOut->Data;
                                    videoEncoder->RequestBuffers(&data);
                                    convertInfo[i].outPitch  = data.Pitch;
                                    convertInfo[i].output[0] = data.Y;
                                    convertInfo[i].output[1] = data.UV;
                                }
                                else
                                {
                                    convertInfo[i].output[0] = nextPicOut.picOut->img.plane[0];
                                    convertInfo[i].output[1] = nextPicOut.picOut->img.plane[1];
                                    convertInfo[i].output[2] = nextPicOut.picOut->img.plane[2];
                                }

                                SetEvent(convertInfo[i].hSignalConvert);   

                                // get NV12 frame, then convert to UYVY422.

                                memcpy(g_videobuffer, convertInfo[i].output[0], convertInfo[i].width*convertInfo[i].height);
                                memcpy(g_videobuffer + convertInfo[i].width*convertInfo[i].height,
                                       convertInfo[i].output[1], convertInfo[i].width*convertInfo[i].height /2);
                                .......
                                .......

 

[kayvan]

Hi,
Now it can work, I try to convert the NV12 frame to UYVY422 by ffmpeg then write to decklink,
and this convert logic consumes additional 15% CPU with Intel(R) Core(TM) i5-3470 @3.20GHz.
thank you for your help, we are waiting for obs-studio.

                    if(!bUsing444)
                    {
                        profileIn("conversion to 4:2:0");

                        if(bUseThreaded420)
                        {
                            for(int i=0; i<numThreads; i++)
                            {
                                convertInfo[i].input     = (LPBYTE)map.pData;
                                convertInfo[i].inPitch   = map.RowPitch;

                                if(bUsingQSV)
                                {
                                    mfxFrameData& data = nextPicOut.mfxOut->Data;
                                    videoEncoder->RequestBuffers(&data);
                                    convertInfo[i].outPitch  = data.Pitch;
                                    convertInfo[i].output[0] = data.Y;
                                    convertInfo[i].output[1] = data.UV;
                                }
                                else
                                {
                                    convertInfo[i].output[0] = nextPicOut.picOut->img.plane[0];
                                    convertInfo[i].output[1] = nextPicOut.picOut->img.plane[1];
                                    convertInfo[i].output[2] = nextPicOut.picOut->img.plane[2];
                                }

                                SetEvent(convertInfo[i].hSignalConvert);   

                                // get NV12 frame, then convert to UYVY422.                    
                                unsigned long int length = convertInfo[i].width*convertInfo[i].height;
                                memcpy(g_videobuffer, convertInfo[i].output[0], length);
                                memcpy(g_videobuffer + length, convertInfo[i].output[1], length / 2);
                                .......
                                .......
                                 }
                        }
                   }