Try to delete all comments, and put the script into neutral directory like: c/users/owner/documents. This seems like a syntax error. LuaJIT is builtin in Windows.
Tips and tricks for Lua scripts
thx , it was totally my mistake :p
Rendering 3D models in OBS Studio.
Although the newest version 30.1.0+ of the program now supports capturing textures with the alpha channel, it maystill be useful to work with geometry inside the program, e.g. creating a 3D logo, rendering sources to texture on the model etc.
So the first step is to set up vertex buffer data, it is a relatively slow operation and it is done at the initialization stage.
Lua:
S.gs_render_start(true) -- enters the render context
S.gs_vertex3f, S.gs_texcoord, S.gs_normal3f -- functions to call inside that context
data.my_buf = S.gs_render_save() -- buffer is saved
S.gs_load_vertexbuffer(data.my_buf) -- loading geometry so we can draw it
S.gs_draw(S.GS_TRIS, 0, vertex_num) -- this will do itsteps must be executed. The depth buffer testing, correct camera to Z, and clear color and depth buffers gracefully.
Sounds like a lot but it is actually just 4 lines of code.
Lua:
S.gs_clear(bit.bor(S.GS_CLEAR_COLOR , S.GS_CLEAR_DEPTH), clear_color, 0, 0)
S.gs_ortho(0.0, data.width, 0.0, data.height, -2000.0, 2000.0)
S.gs_enable_depth_test(true)
S.gs_depth_function(S.GS_GREATER)which can handle that if we pass the GS_Z16 flag. So we do the drawing inside of this texrender context, and finally we pass the texture to the output.
Here is the link to the script which can load and show .obj files.
3D model viewer (.obj format) for OBS Studio
3D model viewer (.obj format) for OBS Studio. GitHub Gist: instantly share code, notes, and snippets.
gist.github.com
Hi!
No sure if this is the right place to post this, but I figured this might interest some of you.
I recently started writing lua scripts for obs, but I could not find anything related to support of obslua with the lua language server. I think that having this sort of support would make the overall development experience quite a bit more fluid.
I made a post for this on the obs ideas & suggestions website (https://ideas.obsproject.com/posts/2557/support-for-lua-language-server). If you guys also think this sort of thing would be nice, feel free to upvote the post :)
No sure if this is the right place to post this, but I figured this might interest some of you.
I recently started writing lua scripts for obs, but I could not find anything related to support of obslua with the lua language server. I think that having this sort of support would make the overall development experience quite a bit more fluid.
I made a post for this on the obs ideas & suggestions website (https://ideas.obsproject.com/posts/2557/support-for-lua-language-server). If you guys also think this sort of thing would be nice, feel free to upvote the post :)
DirectX C API access via ffi
Tested on OBS Studio version 30.1.0, but should work on earlier versions too.
OBS Studio Direct3D feature level is 11.0, and shader model version is 4.0.
Now, before we start, why even bother to writing raw DirectX at all instead of using libobs graphics wrappers?
While it is has an extensive list of supported features, it is mostly tailored to composition, filtering and capturing.
So if you want to do some effects that are outside of that paradigm, you may not be able to.
For example, you can't use Vertex Texture Fetch inside a vertex shader. Its resources are not set in the rendering pipeline.
But I will show you how I did it, the end result, without the important ffi bindings is only 4 lines of code in the rendering context.
The tricky part is how to define these API calls. Actually it is very straightforward if we know the underlying model that DirectX uses.
COM - or Component Object Model is the main driving force behind it. I will not go deep into it, but here is a good 10/10 overview.
The other important link is related to modding old games, the solution there is done with LuaJIT ffi and DirectX version 9.
Similarly, libobs provides an almost identical function to get device pointer.
It is technically possible to load d3d11 and create a device yourself, the same approach would work in a standalone interpreter, but I digress.
I'll add correctly indexed and full vtables later, for now let's focus on the glue code.
As you can see, it looks very similar to the DirectX 9 link approach, experimentally it is found that we must use index instead of a name in our vtable.
Now the routine is to find an interesting function, and write it's binding to the following template:
data.VSSetShaderResources = ffi.cast("long(__stdcall*)(void*, unsigned int, unsigned int, void**)", data.pContext2.lpVtbl[25])
data.VSSetShaderResources - this is a permanent location on stack that survives garbage collector
"long(__stdcall*)(void*, unsigned int, unsigned int, void**)" - return type, calling convention, first argument is vtable, the type and number of arguments
ffi.cast("...", data.pContext2.lpVtbl[25]) - the index in the vtable
GetImmediateContext - MSDN says this will increment the reference count, so we must release it when we are done with it to avoid a memory leak, this is done in the destroy callback.
data.Release_pContext(data.pContext2)
--data.Release_pDevice(data.pDevice)
I commented Release_pDevice because we did not create this resource, so I think it is not our responsibility to deal with it.
So this is basically it, the lost device case is left as an exercise for the reader.
From my tests, this final code does not crash during execution and at exit nor it does it leak memory, the gist features texture reading in vertex shader passing results to fragment shader, simple color cycle.
Looking forward to what you guys can do with all of this, some ideas :)
- Thread-safe texture reading from GPU to RAM
- Stateful particle system
- D3D11 clear screen, triangle
Vtables. I have found out the Odin programming language vendor sources to be a reliable and readable source of vtables information for DirectX,
Although you can find enum lists and indexes on the net, I think it is fair to include a full description here.
Device
Device context
Tested on OBS Studio version 30.1.0, but should work on earlier versions too.
OBS Studio Direct3D feature level is 11.0, and shader model version is 4.0.
Now, before we start, why even bother to writing raw DirectX at all instead of using libobs graphics wrappers?
While it is has an extensive list of supported features, it is mostly tailored to composition, filtering and capturing.
So if you want to do some effects that are outside of that paradigm, you may not be able to.
For example, you can't use Vertex Texture Fetch inside a vertex shader. Its resources are not set in the rendering pipeline.
But I will show you how I did it, the end result, without the important ffi bindings is only 4 lines of code in the rendering context.
Lua:
while S.gs_effect_loop(data.effect, "Custom1") do
S.gs_load_texture(data.tex1, 0) -- sets pixel shader resource view
data.PSGetShaderResources(data.pContext2, 0, 1, data.pRes)
data.VSSetShaderResources(data.pContext2, 0, 1, data.pRes)
S.gs_draw_sprite(nil, 0, 2560, 1440)
endCOM - or Component Object Model is the main driving force behind it. I will not go deep into it, but here is a good 10/10 overview.
The other important link is related to modding old games, the solution there is done with LuaJIT ffi and DirectX version 9.
Similarly, libobs provides an almost identical function to get device pointer.
It is technically possible to load d3d11 and create a device yourself, the same approach would work in a standalone interpreter, but I digress.
I'll add correctly indexed and full vtables later, for now let's focus on the glue code.
Lua:
data.device = obsffi.gs_get_device_obj()
data.pDevice = ffi.cast("struct d3ddevice*", data.device)
data.GetImmediateContext = ffi.cast("long(__stdcall*)(void*, void**)", data.pDevice.lpVtbl[40])
data._arg1 = ffi.new("unsigned long[1]")
data.pContext = ffi.cast("void**", data._arg1)
data.GetImmediateContext(data.pDevice, data.pContext)
data.pContext2 = ffi.cast("struct d3ddevicecontext*", data.pContext[0])
data.Release_pContext = ffi.cast("unsigned long(__stdcall*)(void*)", data.pContext2.lpVtbl[2])
data.Release_pDevice = ffi.cast("unsigned long(__stdcall*)(void*)", data.pDevice.lpVtbl[2])
data.VSSetShaderResources = ffi.cast("long(__stdcall*)(void*, unsigned int, unsigned int, void**)", data.pContext2.lpVtbl[25])
data.PSGetShaderResources = ffi.cast("long(__stdcall*)(void*, unsigned int, unsigned int, void**)", data.pContext2.lpVtbl[73])
data.VSSetSamplers = ffi.cast("long(__stdcall*)(void*, unsigned int, unsigned int, void**)", data.pContext2.lpVtbl[26])
data._arg2 = ffi.new("unsigned long[1]")
data.pRes = ffi.cast("void**", data._arg2)Now the routine is to find an interesting function, and write it's binding to the following template:
data.VSSetShaderResources = ffi.cast("long(__stdcall*)(void*, unsigned int, unsigned int, void**)", data.pContext2.lpVtbl[25])
data.VSSetShaderResources - this is a permanent location on stack that survives garbage collector
"long(__stdcall*)(void*, unsigned int, unsigned int, void**)" - return type, calling convention, first argument is vtable, the type and number of arguments
ffi.cast("...", data.pContext2.lpVtbl[25]) - the index in the vtable
GetImmediateContext - MSDN says this will increment the reference count, so we must release it when we are done with it to avoid a memory leak, this is done in the destroy callback.
data.Release_pContext(data.pContext2)
--data.Release_pDevice(data.pDevice)
I commented Release_pDevice because we did not create this resource, so I think it is not our responsibility to deal with it.
So this is basically it, the lost device case is left as an exercise for the reader.
From my tests, this final code does not crash during execution and at exit nor it does it leak memory, the gist features texture reading in vertex shader passing results to fragment shader, simple color cycle.
Looking forward to what you guys can do with all of this, some ideas :)
- Thread-safe texture reading from GPU to RAM
- Stateful particle system
- D3D11 clear screen, triangle
Vtables. I have found out the Odin programming language vendor sources to be a reliable and readable source of vtables information for DirectX,
Although you can find enum lists and indexes on the net, I think it is fair to include a full description here.
Device
C:
struct d3ddeviceVTBL {
void *QueryInterface;
void *AddRef;
void *Release;
void *CreateBuffer;
void *CreateTexture1D;
void *CreateTexture2D;
void *CreateTexture3D;
void *CreateShaderResourceView;
void *CreateUnorderedAccessView;
void *CreateRenderTargetView;
void *CreateDepthStencilView;
void *CreateInputLayout;
void *CreateVertexShader;
void *CreateGeometryShader;
void *CreateGeometryShaderWithStreamOutput;
void *CreatePixelShader;
void *CreateHullShader;
void *CreateDomainShader;
void *CreateComputeShader;
void *CreateClassLinkage;
void *CreateBlendState;
void *CreateDepthStencilState;
void *CreateRasterizerState;
void *CreateSamplerState;
void *CreateQuery;
void *CreatePredicate;
void *CreateCounter;
void *CreateDeferredContext;
void *OpenSharedResource;
void *CheckFormatSupport;
void *CheckMultisampleQualityLevels;
void *CheckCounterInfo;
void *CheckCounter;
void *CheckFeatureSupport;
void *GetPrivateData;
void *SetPrivateData;
void *SetPrivateDataInterface;
void *GetFeatureLevel;
void *GetCreationFlags;
void *GetDeviceRemovedReason;
void *GetImmediateContext;
void *SetExceptionMode;
void *GetExceptionMode;
};
struct d3ddevice {
struct d3ddeviceVTBL** lpVtbl;
};
C:
struct d3ddevicecontextVTBL {
void *QueryInterface;
void *Addref;
void *Release;
void *GetDevice;
void *GetPrivateData;
void *SetPrivateData;
void *SetPrivateDataInterface;
void *VSSetConstantBuffers;
void *PSSetShaderResources;
void *PSSetShader;
void *SetSamplers;
void *SetShader;
void *DrawIndexed;
void *Draw;
void *Map;
void *Unmap;
void *PSSetConstantBuffer;
void *IASetInputLayout;
void *IASetVertexBuffers;
void *IASetIndexBuffer;
void *DrawIndexedInstanced;
void *DrawInstanced;
void *GSSetConstantBuffers;
void *GSSetShader;
void *IASetPrimitiveTopology;
void *VSSetShaderResources;
void *VSSetSamplers;
void *Begin;
void *End;
void *GetData;
void *GSSetPredication;
void *GSSetShaderResources;
void *GSSetSamplers;
void *OMSetRenderTargets;
void *OMSetRenderTargetsAndUnorderedAccessViews;
void *OMSetBlendState;
void *OMSetDepthStencilState;
void *SOSetTargets;
void *DrawAuto;
void *DrawIndexedInstancedIndirect;
void *DrawInstancedIndirect;
void *Dispatch;
void *DispatchIndirect;
void *RSSetState;
void *RSSetViewports;
void *RSSetScissorRects;
void *CopySubresourceRegion;
void *CopyResource;
void *UpdateSubresource;
void *CopyStructureCount;
void *ClearRenderTargetView;
void *ClearUnorderedAccessViewUint;
void *ClearUnorderedAccessViewFloat;
void *ClearDepthStencilView;
void *GenerateMips;
void *SetResourceMinLOD;
void *GetResourceMinLOD;
void *ResolveSubresource;
void *ExecuteCommandList;
void *HSSetShaderResources;
void *HSSetShader;
void *HSSetSamplers;
void *HSSetConstantBuffers;
void *DSSetShaderResources;
void *DSSetShader;
void *DSSetSamplers;
void *DSSetConstantBuffers;
void *DSSetShaderResources;
void *CSSetUnorderedAccessViews;
void *CSSetShader;
void *CSSetSamplers;
void *CSSetConstantBuffers;
void *VSGetConstantBuffers;
void *PSGetShaderResources;
void *PSGetShader;
void *PSGetSamplers;
void *VSGetShader;
void *PSGetConstantBuffers;
void *IAGetInputLayout;
void *IAGetVertexBuffers;
void *IAGetIndexBuffer;
void *GSGetConstantBuffers;
void *GSGetShader;
void *IAGetPrimitiveTopology;
void *VSGetShaderResources;
void *VSGetSamplers;
void *GetPredication;
void *GSGetShaderResources;
void *GSGetSamplers;
void *OMGetRenderTargets;
void *OMGetRenderTargetsAndUnorderedAccessViews;
void *OMGetBlendState;
void *OMGetDepthStencilState;
void *SOGetTargets;
void *RSGetState;
void *RSGetViewports;
void *RSGetScissorRects;
void *HSGetShaderResources;
void *HSGetShader;
void *HSGetSamplers;
void *HSGetConstantBuffers;
void *DSGetShaderResources;
void *DSGetShader;
void *DSGetSamplers;
void *DSGetConstantBuffers;
void *CSGetShaderResources;
void *CSGetUnorderedAccessViews;
void *CSGetShader;
void *CSGetSamplers;
void *CSGetConstantBuffers;
void *ClearState;
void *Flush;
void *GetType;
void *GetContextFlags;
void *FinishCommandList;
};
struct d3ddevicecontext {
struct d3ddevicecontextVTBL** lpVtbl;
};First off, this is years apart but thank you so much because docs are not as thorough. I've come across the ID and always wondered whether this was a known literal (as @Mega64 suggests) or whatever so long as it's unique.
This post is mostly for anyone else stumbling into this issue, about creating a scene and a source (based on known types) and even though I wrote it in python, my observations should be transferable.
My use case is that I'm QAing games voluntarily for a small indie company and the tester pool doesn't use OBS or screen capture all the times, so I wanted to make sth to propose as standard practice. I wanted to write a script to automate setup of everything, including running the game, starting and stopping recording depending on game runtime and in case of game crash, stop the recording and assemble an archive based on selected saved games and the video clip of the session, which could be then emailed to the developer. I've figured out how to run the game and check for a crash. So I wanted to create a Scene and a Game Capture source, specifically for the game, set up with the game's window name so that the game window can be insta-recognized when source is active.
Creating a scene was simple:
Python:
def find_or_create_scene(scene_name):
scene_ref = find_scene(intended_scene_name)
if scene_ref is None:
print("Intended scene does not exist. Will create.")
scene_ref = obs.obs_scene_create(intended_scene_name)
return scene_refSo next step was to set up the source. At this point I have to say that this whole "a Scene is a source, but a Scene is also comprised of scene items which are also sources" had me confused and still does in some cases and I think this is mostly due to the python wrapper interface. At the end of the day, I make heads or tails just looking at the C api and what every function returns and try to go from there and whatever the error tracebacks tell me.
My first attempt to figure out how to create a source (because my first attempt at using the obs_source_create just crashed OBS) was to create the source myself on the GUI and then try to access all relevant info and print it so I can replicate the setup.
I first found the scene context. Then I enumerated the scene items from that context and for each source, enumerate its properties and their values in the settings.
But then I hit a wall with python not having an implementation for obs_property_next(**p). I found this:
I tried it, but it didn't yield results (or I just did it wrong) and honestly I didn't think it was worth the effort and time to set me back if there was a simpler way to move on.
Next step was to print everything for a property, provided I could figure out the property name by trial and error and that did show some promise and some results, but without knowing them all, I couldn't be sure I had all the necessary properties for a working setup. At this point, I understood how properties and settings worked together as I was trying to figure out how to draw the selected value out of a property.
This is what I had at the time. Sure looks enough, but...
Python:
def create_source(props, property):
scene_ref = find_scene(game_scene_name) #obs_scene_t*
settings = obs.obs_data_create()
obs.obs_data_set_string(settings, "capture_mode", "window")
obs.obs_data_set_string(settings, "window", game_capture_window_name)
new_source = obs.obs_source_create("<game_name>_capture", intended_scene_source_name, settings, None)
obs.obs_scene_add(scene_ref, new_source)
obs.obs_save_sources()It didn't crash mind you, but it also didn't look right and the new source was created, but I couldn't access the properties.
This corroborates what bfxdev was saying here, since if you notice my id above is "<gamename>_capture":
that the registration of ID won't fail if it's not one of the known ids. Which happened to me. I could create the source, but the source did not have a game capture icon next to it, likely because it was not properly recognized as one of the types of sources. I still thought just wasn't setting it up fully and I was missing some property/setting.
I figured since Settings are what keeps the actual selected values of the properties, I didn't need to enumerate properties but the settings instead, which led to obs_data_get_json_pretty_with_defaults.
JSON:
{
"window": "<game name>.exe",
"capture_mode": "window",
"priority": 2,
"sli_compatibility": false,
"capture_cursor": true,
"allow_transparency": false,
"premultiplied_alpha": false,
"limit_framerate": false,
"capture_overlays": false,
"anti_cheat_hook": true,
"hook_rate": 1,
"rgb10a2_space": "srgb"
}Most of those are what you see when you click on the Properties button. But even though that was a huge improvement, it still didn't help with having the GUI recognize the source as a game capture type.
So after reading Mega's post, I changed the ID from what I had to "game_capture", the source got the game capture icon, the Properties button was both visible and enabled and pressing it showed me the properties window and all of the properties had the right values.
I did think about why there are seemingly two ways to set up a new source, Mega's and mine's obs_source_create approach and @bfxdev's source info approach. I think the source info approach is for extending the source types to add new ones, mainly due to the fact that you have to reimplement a few funtions that you shouldn't have to just to add a source of a known type. Which makes sense that this was the go-to way to add a *new* type of filter to OBS. Otherwise, the docs have an api for applying a filter to a source (obs_source_filter_add).
Also, @Mega64, in your quoted post at the beginning, you said the only caveat is that new scenes don't persist between obs executions and that's because you have to explicitly call obs_save_sources. The only issue so far for me is that sometimes you a newly created scene or source won't be able to get the name you want, because you tried this same piece of code before. e.g. if I run OBS, delete my source and then trigger the python script to recreate the source, it will create <source-name>2.
Hello @npatch
Regarding implementation for obs_property_next(**p). I've checked your GitHub project where you've used cffi for this.
Personally I recommend to use first-party industry-standart : ctypes. I have had problems with cffi in the past and simple snippets would not compile on the virtual machine, while at the same time in ctypes - it 100% just works.
I am using SWIG trick, I've shared earlier. Here is conversion from LuaJIT ffi to ctypes interface. A link to the full source code.
Also why not using Lua for automating OBS Studio ?
Regarding implementation for obs_property_next(**p). I've checked your GitHub project where you've used cffi for this.
Personally I recommend to use first-party industry-standart : ctypes. I have had problems with cffi in the past and simple snippets would not compile on the virtual machine, while at the same time in ctypes - it 100% just works.
I am using SWIG trick, I've shared earlier. Here is conversion from LuaJIT ffi to ctypes interface. A link to the full source code.
Python:
def xyz():
source = obs_get_source_by_name("tmp")
if source:
fSource = obs_source_get_filter_by_name(source, "color")
if fSource:
props = obs_source_properties(fSource)
if props:
prop = obs_properties_first(props)
name = obs_property_name(prop)
if name:
_p = cast(c_void_p(int(prop)), POINTER(Property))
foundProp = G.obs_property_next(byref(_p))
prop = cast(_p, POINTER(Property))
while foundProp:
print(G.obs_property_name(prop))
_p = cast(prop, POINTER(Property))
foundProp = G.obs_property_next(_p)
prop = cast(_p, POINTER(Property))
obs_properties_destroy(props)
obs_source_release(fSource)
obs_source_release(source)Also why not using Lua for automating OBS Studio ?
thanks for that, I've also looked through your github. It's been a good resource for things, as well as this thread.
I ended up dropping cffi and obs_property_next (which was only really meant for printing to be honest). I just keep the import in case I need to dabble in it again against my will. My use-case is pretty simple. I only need recording automation with process tracking and some special handling if the app crashed.
As for lua vs python, I don't know lua, I know python from blender, so my choice was that. I didn't want to restart the whole thing, since I just sidestepped the issue of property traversal. Again, I need specific stuff. But I'm thinking of redoing this as a plugin instead, but after I have sth working in python. I figure a plugin might have better coverage of functionality than scripting. Currently I have a different issue, check my other thread, if possible.