Recording Transparent Video with VP9

LordHenry17

New Member
First off, my intention: I want to record Videos with a transparent background, where Chroma keying does not work (Specific examples: Vtuber models with semitransparent parts or colors that would mess with chroma keys, for Voice-overs).

One way it works for sure:
- Advanced Settings: Color Format: BGRA; Color Space: sRGB; Color Range: limited
- Output (Advanced Mode): Container Format: mov; Video Encoder: prores; Audio Encoder: aac (any other Settings did not have a noticable Impact)

This will let you create Video files with an Alpha layer the only problem with that method is that it creates huge files (~200mbits).

So the Idea was to use another codec that supports an alpha layer and has higher compression. Codecs I have tried so far:

- DNxHD: Background not transparent, huge files (~150mbits)
- png: did not work at all
- GoPro Cineform: Background not transparent, huge files (~600mbits)
- qtrle: works, huge file size (100-300mbits)
- vp9: somewhat works, small file size (more below)

Now to the actual reason I am posting this. I tried using VP9 to record with a transparent background and it works fine to a certain degree.
First off my settings:
SettingsVP9.png
Settings VP9-2.png

For the Audio Encoder: libvorbis lets you record but obs often crashes when the recording is finished, also DaVinci Resolve wont get the audio
Color Space: Rec.709 and Rec.601 also work. (The other two might work, but I didnt test them since they heavily desaturate the image in the preview already)
Color Format: BGRA is the only one that will keep the transparency, P416 does not keep transaprency and does turn the video into green goo when opened in the windows player (looks fine when imported in DaVinci)
Color Range: full will not record transparency
Container Format: webm also works for the transparency, but doesnt keep the audio (DaVinci doesnt like either libvorbis nor libopus, windows will play the audio though). Also the problem I am having still exists with webm. mp4 obviously does not keep the transparency.

The Problem:
Screenshot 2025-02-09 172833.png

(Sorry, I cant provide a bigger Image due to file size limitations, but the problem is still visible)
Since I was too lazy to properly set something up myself I just recorded the livestream I had open and scaled it to a quarter of the canvas (so 3/4 of the video should be transparent with the recorded livestream appearing in the top left corner) however when I put the transparent video over another video in DaVinci
almost all pure black parts of the foreground video are also transparent instead of being black. In my example the anime girl and the dark strip to the left belong to the foreground video and the space between should be completely black (its the black border from a YouTube Video in vertical format), but the background is showing through.

Can anyone help me with why this happens and possibly how to fix it? I dont really know what I am doing and only got to this point through a lot of googling and trial and error.

(Also I know this is the wrong forum for this, but when i import the vp9 encoded .mkv in DaVinci it shows the media offline error in the preview thumbnails, when scrubbing and when playing the video backwards, but not when playing forwards at normal speeds or in the rendered file. Also the errors appear in different parts of the clip everytime you scrub through, so not at consistent timestamps. I already tried most of the recommended fixes.)
 
Top