Bug Report Problem with colorkey and capturing window.

[Zirgotv]

Good day Jim.

I was fixing up a scene where I want to capture part of an application screen and adding a colorkey to it, in this case green in the color picker. I expected that colorkey is the same as setting a chroma key so some color in the captured screen will be filtered out and transparent. While I did so and added the capture to the scene I tried to preview it and the application hung up on me and generated an error log. It reads as follows:

Open Broadcaster Software v0.432a - 64bit (　^ω^)
-------------------------------
CPU Name: AMD Phenom(tm) II N830 Triple-Core Processor
CPU Speed: 2094MHz
Physical Memory:  3578MB Total, 1589MB Free
stepping id: 3, model 5, family 15, type 0, extmodel 8, extfamily 0, HTT 1, logical cores 3, total cores 3
------------------------------------------
Adapter 1
  Video Adapter: AMD Radeon HD 6500M/5600/5700 Series
  Video Adapeter Dedicated Video Memory: 1064222720
  Video Adapeter Shared System Memory: 1607938048
=====Stream Start=====================================================================
  Multithreaded optimizations: On
  Base resolution: 1280x720
  Output resolution: 1280x720
------------------------------------------
Loading up D3D10...
------------------------------------------
Audio Encoding: AAC
    bitrate: 64
Warning -- DShowPlugin: Could not create device filter
Warning -- D3D10Texture::CreateTexture: CreateTexture2D failed, result = 0x80070057
------------------------------------------
Video Encoding: x264
    fps: 30
    width: 1280, height: 720
    quality: 10
    preset: superfast
    i444: no
    max bitrate: 5000
    buffer size: 5000
------------------------------------------

Exception Fault - Stack Trace:
    OBS::MainCaptureLoop -> Scene::Render

Hope there can be some sense in this info to make it work. I have the latest DirectX installed, tried it just to make sure that wasn't the problem.

 

[Zirgotv]

Noted that this has been addressed as a problem in the current version and will be fixed in the next version? Probably it is the same error that you have been getting and trying to fix?

 

[Lain]

hmm, are you using this on a device, or on a software capture source? the log seems to indicate a device, so forgive me if I'm confused.

 

[Zirgotv]

I am using software capture source, capturing a software window I created myself. In the app I have defined a panel with the colour of clLime. In OBS i use the same colour as color key and capture an area in my software. When trying to preview this area in OBS, the error occurs and the app closes.

Maybe worth mentioning is that I didn't have the device (video camera) connected in this scene so the "CAMERA" layer I defined shows up as black. Could it be that I need to have it connected? This didn't prove a problem before when previewing the scene.

 

[Lain]

no that's fine, I was just confused by the device warning.

anyway, I'll take a look at it, and see if I can get it to crash. is there any way for me to replicate? otherwise I may ask you to get me a dump file or something so I can find out what specifically is causing it to crash

 

[Zirgotv]

To replicate is following these steps.
-Create scene
-I have created the software capture as a global resource. The application I want to capture is custom by me so I set the property of the "panel" component in my application to clLime or bright green.
-I don't capture entire app window but use inner window and set it up to capture an area within the window that has the clLime ( with the idea that what else that isnt that color in the area will be visible and rest transparent.
- So I enable the colorkey and choose the color clLime. The other properties I have left untouched, like sensitivity etc.
- Apply and exit global resources.
- Add the software capture from global resources to my scene.
- Try to make a preview by pressing the "preview stream" button.
- App hangs up and generates above mentioned error log.

That should be the steps needed to recreate this error. I haven't tried the app or color key out in any other way than this so can't say if I will get the same results in other cases as well. I am using the 0.432 alpha 64 bit version of OBS.