Question / Help Using one Video Source as Alpha for another?

jashan

jashan

New Member
I'm new to OBS and would like to use it both for Mixed Reality recording and streaming for a Virtual Reality game that I'm currently developing. The SteamVR Unity plugin has a new feature for this that will render output to four cameras, one in each quadrant of the screen (top left, bottom left, top right, bottom right).

The way this is meant to be used is to use the bottom right as background (that's a full in-game camera that always at the same location as the "Webcam"), then a Webcam with green screen (that records the actual physical player playing the game), and then the in-game camera that only captures things that are in the game world between the Webcam / in-game cam and the player himself.

For the last one, there's one quadrant with the actual footage, and another one with the alpha mask.

So, what I'd need to do is use that alpha mask (from the video) in an alpha mask filter for the actual footage. I did find the alpha mask filter and I think this would be exactly what I need - except that apparently, it only takes images. So while I could use this for a static alpha mask that never changes, there does not seem to be a way to feed another source into that.

Did I only not find this, or is this something that's not possible with OBS (Studio) at all?
 
R

RebelPhoton

New Member
This is something that is bringing many new people to OBS.

http://steamcommunity.com/app/358720/discussions/0/405694031549662100/

https://www.youtube.com/watch?v=5BmiFIvII7Y

http://uploadvr.com/mixed-reality-videos-get-official-support-with-unity-plug-in/

https://www.vrfocus.com/2016/06/epic-games-developing-mixed-reality-support-for-unreal-engine-4/


The unity plugin right now outputs a four-quadrant render. One on the quadrant is the foreground elements meant to be composited on top of the player's live chroma keyed stream. Another quadrant is the Foreground alpha.

How can we interpret this in OBS so that one live cropped layer is used as another live cropped layers' alpha channel?

Thanks for the help
 
zinefer

zinefer

New Member
I too need this feature. I don't even know how to do it live with _any_ program so if someone knows of a way that would be appreciated.
 
T

Tom Goethals

New Member
+1 for this feature, exact same requirement.

Don't understand why the Valve devs would not put a real alpha channel in the top left quadrant. (if you do game capture on the top right, (the mask) you don't need to do any keying the black is already "alpha" 0. (if you enable transparancy in the game capture) So if they would put that on the top left quadrant there would be no need for this.
 
T

Tom Painter

New Member
+1 for this, I'm another person wanting it for Mixed Reality.

Jashan I saw your post on the steam forum talking about piping around different screens/channels.
Im in the middle of getting our setup together at the moment. Im thinking....

machine 1 runs the game and outputs the 4 quadrant views

machine 2 has 2 capture cards (we need 2 to get 2 HMDI in ports right? or maybe there are cards with 2 I havent researched it yet), one accepts HDMI input from the video camera, one from machine 1's game view.
Ideally we have alpha working nicely for this foreground layer, and OBS can produce a nice comp, we then mirror this comp on a 7" field monitor display attached to the camera so our camera operator can see in mixed reality as filming.

now...currently I have some problems before we can make 1080p mixed reality footage
do I need a 4K monitor to output 4K footage from machine 1?
is my geforce 970 going to be able to render 4k and a 90FPS view on the HMD (sounds like this is very unlikely)

For now Im focusing on 540p mixed reality, more thoughts include...
would there be a way for machine 2 to create the 'comp' but also be dumping out all the raw footage too? this way OBS gives us a good pre-visualisation of what we're trying to do, we can use this OBS footage to select our cuts for further refinement in apps like After Effects where we have more refined control over alpha etc.

well theres my brain dump, thanks for listening guys and

cheers
Tom
 
T

Tom Painter

New Member
a picture speaks a thousand words (and several thousand pounds!)

This is my plan, if anyone can see any problems with this please give me a shout. running multiple instances of OBS I havent tested yet but a forum said it was possible

its a XEON workstation with tons of cores so should handle it ok I think.
 

Attachments

  • Screen Shot 2016-08-25 at 12.48.53.jpg
    Screen Shot 2016-08-25 at 12.48.53.jpg
    283.5 KB · Views: 476
T

Tom Painter

New Member
hey steffan thanks so much
sorry it took me 2 months to see this, there was no notification
cheers mate will check it out soon
tom
 
N

nonvr

New Member
did some one have the OBS mixed reality alpha plugin (blend_sub_filter.filter) because OBS update have erase mine and it's not downloadable from OBS plugin section anymore. some one can send a download link of he's one version?
 
Steffan Donal

Steffan Donal

New Member
For anyone else who might come across this thread in their searching - the filter has been built in to the LIV Client, on Steam.

It's a free VR mixed reality compositing tool that I built at LIV!
 
W

Waffles13

New Member
Steffan Donal said:
For anyone else who might come across this thread in their searching - the filter has been built in to the LIV Client, on Steam.

It's a free VR mixed reality compositing tool that I built at LIV!
Click to expand...
Can't thank you enough, not only for developing this (and for free, no less), but also for taking the time to let the thread know. Saves me an afternoon trying to recompile the entire SteamVR client just to add the alpha channel to the foreground.
 
S

Sagar998

New Member
Waffles13 said:
Can't thank you enough, not only for developing this (and for free, no less), but also for taking the time to let the thread know. Saves me an afternoon trying to recompile the entire SteamVR client just to add the alpha channel to the foreground.
Click to expand...
So, is that plugin finally working in OBS or LIV?? if working pls let me know the process..?
 
S

Sean Fulton Hall

New Member
It was difficult to pry open OBS plugin building in one hour, so I opted to overwrite an existing shader in OBS to accomplish mixed reality streaming for SteamVR in OBS.

Okay so if you want to use OBS to use the color of the top-right quadrant of a source as the alpha on the top-left quadrant of a source, simply replace the contents of any .effect file in obs-studio/data/obs-plugins/obs-filters/ with

uniform float4x4 ViewProj;
uniform texture2d image;

uniform float4 color;
uniform float contrast;
uniform float brightness;
uniform float gamma;

uniform float4 key_color;
uniform float similarity;
uniform float smoothness;

sampler_state textureSampler {
Filter = Linear;
AddressU = Clamp;
AddressV = Clamp;
};

struct VertData {
float4 pos : POSITION;
float2 uv : TEXCOORD0;
};



VertData VSDefault(VertData v_in)
{
VertData vert_out;
vert_out.pos = mul(float4(v_in.pos.xyz, 1.0), ViewProj);
vert_out.uv = v_in.uv;
return vert_out;
}



float4 OffsetPixel(float4 rgba, VertData v_in)
{
float2 shiftedUV = v_in.uv;
shiftedUV.x += 0.5f;
return image.Sample( textureSampler, shiftedUV ) * color;
}

float4 PSColorKeyTRAppliedTLAndCroppedTL( VertData v_in )
{
float4 rgba = image.Sample( textureSampler, v_in.uv ) * color;

if ( v_in.uv.x < 0.5f && v_in.uv.y < 0.5f )
{
float4 offsetRgba = OffsetPixel( rgba, v_in );
rgba.w = offsetRgba.x;
}
return rgba;

}

float4 PSColorKeyRGBA(VertData v_in) : TARGET
{
float4 rgba = PSColorKeyTRAppliedTLAndCroppedTL( v_in );
return rgba;
}



technique Draw
{
pass
{
vertex_shader = VSDefault(v_in);
pixel_shader = PSColorKeyRGBA(v_in);
}
}


and then use the filter of the same name inside of OBS. I recommend not writing over an effect file for a filter you are already using, and to back up your original effects files before overwriting one or more of them.
 
You must log in or register to reply here.
Top