Question / Help How to record a video with transparency?

[Gene90]

Hello again to all OBS users and developers! I'm not entirely certain whether I'm posting this question in the right category, and I've done some searching on Google but haven't found an answer to my question (perhaps due to not defining the actual question in proper words). I'll try to explain as best as I can what exactly I want to achieve, so here's the overall picture:

I want to make some footage of a game that causes tremendous amounts of pixelation (due to foliage) when it is recorded in OBS, even with a high bitrate. Due to this, I record the game on different software (DxTory) with with custom lossless codecs that are really impressive in terms of quality. However, unlike OBS, DxTory doesn't provide an option to include a video capture device whilst recording the game. So what I wanted to do instead is record the gameplay in DxTory and record a webcam separately in OBS and then just add the whole webcam thing in the process of video editing.

This is where the problem starts, and I hope that I will be able to express my thought properly. Now, I have a greenscreen and the transparency works quite well in OBS, given that all the different sources are all recorded in this wonderful program. But if I just record my webcam with the chroma key filter enabled, the black background in a recorded video stays black after I try to view it or edit it or do whatever to it. My question is - is it possible to record a webcam video with chroma key enabled and make the black background transparent, as in for an example, (although I might be saying something silly now) your typical PNG image file? This is the only conceivable way of retaining both the DxTory codec quality and having a transparent webcam video from OBS in one gameplay footage. Any help and advice is really appreciated! Yours sincerely,

Gene90

 

[FerretBomb]

OBS cannot record while preserving the alpha layer, as far as I am aware. Alpha-encoding is also EXTREMELY bitrate-demanding, and does not compress well at all, resulting in HUGE filesizes.

A better way would be to just record the game and webcam in OBS, and use a quality target based recording method, rather than a bitrate target. Namely, using CRF or CQP.
Go into Simple output mode, set the quality to Indistinguishable, and use a hardware encoder if your system has one (NVENC, AMF, QSV). It'll allow the recording to use as much bitrate as it needs at the moment to produce a visually-lossless recording.

If you use Advanced mode for multitrack audio support, swap to CQP/CRF instead of CBR and use a target value of 14-18 (lower is better quality, but larger filesizes; never use zero, and only use lower than 14 if you have a specific reason and need).

 

[Gene90]

Thank you for the prompt reply, FerretBomb, I will try this method out and will see how good the quality and performance is. Also, this is completely offtopic, (my apologies but I'm very curious to know), is it possible for OBS to include custom codecs in the future just like DxTory? I'm asking because I am very impressed with the Magic YUV RGB Codec, it records really smoothly on my rather outdated rig and the quality is honestly just almost (if at all) indistinguishable. While OBS might provide the same quality with the abovementioned quality target recording method, I do not know just yet how smoothly it will operate on my PC.

 

[FerretBomb]

No worries. You can set local recording to use extremely poor compression (x264 Ultrafast, for example) to minimize performance impact, and let the quality-target throw a ton of bitrate at the problem to compensate. That will result in large filesizes, but you can re-encode it later with a better quality compression via Handbrake or similar to bring the filesizes down when real-time performance isn't critical.
Works well even on lower-end machines just to get the recording, then let it crunch away later.

I don't know if OBS will allow custom codecs later; the already-implemented ffmpeg output may, but I don't touch that as it can be very easy to run into problems unless you're very conversant with ffmpeg, and I'm not.