Bug Report Crash upon stopping recording when using AMD-Encoder

[Carsten Scholtes]

Hi!

Whenever I select as Encoder "AMD Video Coding Engine (VCE) H.264 Encoder (Media Foundation)" (instead of "x264" in Settings -> Output -> Streaming -> Encoder), I get a crash immediately upon stopping recording (no streaming). (No such problem exists with Encoder "x264", except maxed out processor cores.) The system does not contain a dedicated graphics card. Onboard Graphics AMD R7 of the AMD processor A8-7600 are used.

The enclosed reports (Crash and Log) were generated by starting OBS, starting a recording of a single window (Minecraft) to a file and stopping the recording after a few seconds. Immediately upon stopping the recording, the crash occured. The recorded file appears to be (nearly?) complete.

Kind regards

Carsten Scholtes

 

[Jack0r]

Unfortunately at the moment we are using Media Foundation to allow users to use VCE (and other hw encoders). So I am not sure that one of the devs will be able to fix this before they can add a native implementation of said encoders.

 

[Carsten Scholtes]

Just a quick guess: According to the crash report, the error occured within the Destructor ~H264Encoder(). Its implementation can be found in the currently distributed sources obs-studio-0.13.2, file OBS-Studio/plugins/win-mf/mf-h264-encoder, lines 72-94:

H264Encoder::~H264Encoder()
{
    HRESULT hr;

    if (!descriptor->Async() || !eventGenerator || !pendingRequests)
        return;

    // Make sure all events have finished before releasing, and drain
    // all output requests until it makes an input request.
    // If you do not do this, you risk it releasing while there's still
    // encoder activity, which can cause a crash with certain interfaces.
    while (inputRequests == 0) {
        hr = ProcessOutput();
        if (hr != MF_E_TRANSFORM_NEED_MORE_INPUT && FAILED(hr)) {
            MF_LOG_COM(LOG_ERROR, "H264Encoder::~H264Encoder: "
                    "ProcessOutput()", hr);
            break;
        }

        if (inputRequests == 0)
            Sleep(1);
    }
}

Shouldn't the line

while (inputRequests == 0) {

actually read

while (inputRequests != 0) {

?