Bug Report Window capture has artifacts

[codl]

Window capture seems to introduce some artifacting around edges, as a result harsh colour / chroma keying does not look right. As an example, here is a window I am trying to color key:

Captured with desktop capture and colour keyed, it looks fine:

And here it is captured through window capture and colour keyed, note the magenta edge:

Sorry about the compression on these last two, I don't know how to get an uncompressed frame from obs's output.

Both of these are using the same magenta colour key filter with 1 similarity and 1 smoothness

 

[gelisam]

The reason this happens is because OBS is scaling the capture using a scaling algorithm which interpolates the colours between the pixels. Since your source has sharp pixel-aligned borders, that scaling causes the pixels at that border to be in-between the pink and non-pink colours. As a result, when you filter away the pink, those pink-ish-but-not-actually-pink pixels are not filtered away.

The solution is to use a different scaling algorithm: right-click on your source / Scale Filtering / point. Unfortunately the Color Key effect's preview window seems to use a hardcoded scaling algorithm, but don't worry about the artifacts in the preview, they won't occur in the actual output if you use point scaling there.

Alternatively, I have found that adding a Sharpen filter after the Color Key filter is sometimes an easy way to eliminate the artifact. It depends on whether the rest of your source is negatively-affected by that filter.