RediJedi
New Member
Some background on what I'm trying to do:
I'm trying to build a music visualizer consisting of a series of custom sources/filters that use each have their own LUA scripts and share data. An Audio filter will extract and process the data, then other source/filter plugins will have a config option where you can pick which audio source you want to "listen" too(by audio filter name), and then the processed audio data will be made available to the LUA script also loaded into that source which will generate the data for rendering(generally a poly or line segments) by the plugin.
So say you want to make one of the kinda standard wave display that just shows the audio data as line segments. I would add the audio extractor filter to whichever audio source I wanted to react to. I'd then add a wave source. The filter would extract it's data and store it in a global array that the wave source would access, making that struct(or OBS_DATA object) available in the wave source plugin.
The wave source would have a LUA script. That LUA script would have a per_frame and a per_point function defined. I want to call those functions and pass them the audio data structure, and have them return the data for the render loop. I imagine I'll create some struct for that data as well.
The per_frame will be called for every frame, and the per_point called for every point in the line(one of the values you would set in per_frame is how many points in the line, along with other render options.. maybe textures too). So the plugin will call the per_frame function and then do a loop for however many points were requested and call the per_point function for each index to get the data for that point.
My immediate goal is bringing in features from Milkdrop 2 into OBS so I can satisfy my weird need for music reactive elements in my stream. But Some of these tools would be useful in general, like just being able to write a script to render some shape with lines based off lua equations could be super useful even without the music part. I've also considered trying to pull other data into the scripts, like notifications so you could have a shape/line change somehow when someone subs or something, all based on code instead of pre-rendered somehow.
What I've done so far:
I've got the audio extraction plugin that pulls the audio data, runs an fft, and sums it up into bass/mid/treb variables and stores it in a global array.
I have a wave plugin that allows you to select which audio source you want to listen too and will extract that data from the global array.
The wave plugin has a setting for a lua script path, and it includes the obs-scripting library.
I'm able to load the script from the path with this code:
Then I'm able to extract the functions from the script like this:
This all came fairly easy so I thought I had this figured out, but C had other ideas.
The problem comes with trying to actually call the functions, pass them custom data structs and get custom data structs back.
I can't seem to use the call_func macro from obs-scripting, or the underlying function the macro uses(I tried copying the define into my file), it complains about something not existing inside the call_func_ function, so the external code.. so clearly my linking isn't right.. or obs-scripting wasn't designed to be linked, and I'm hitting walls because of that. Reading the LUA docs I was able to just write my own code to call the functions, so now I just need to get the data in and out.
I found ls_push_libobs_obj that seemed like it would solve all my problems, as I'm fine with dealing in OBS_DATA_T objects, so I rewrote the audio capture plugin to also create an obs data object with the music data.
but I get runtime errors in the underlying function there too. SWIG related, and likey caused by the hacky way I faked swig support(I think that's what you'd call what I did??) to get this to compile. Either the obs-scripting project took no considering for another plugin trying to load scripts with it, or I suck a C, or both.
So I just need a way to pass libobjs or c structs into lua functions and get them in return(or have the function modify them) and I can move on to the actual render loop, but I can't seem to figure out how to go about this.
I've looked into FFI, but that seems to be more for creating c structs inside lua instead of passing them in from C.
It seems like userData is the basic LUA feature I would need for this, but there's a decent amount of boiler to do there, and it's not how ls_push_libobs_obj was implemented, so that gives me pause on if it'll actually work for this.
Does anyone know of any open source plugins that load LUA scripts or have any ideas on easy ways to pass data structures into and out of LUA scripts?
I'm trying to build a music visualizer consisting of a series of custom sources/filters that use each have their own LUA scripts and share data. An Audio filter will extract and process the data, then other source/filter plugins will have a config option where you can pick which audio source you want to "listen" too(by audio filter name), and then the processed audio data will be made available to the LUA script also loaded into that source which will generate the data for rendering(generally a poly or line segments) by the plugin.
So say you want to make one of the kinda standard wave display that just shows the audio data as line segments. I would add the audio extractor filter to whichever audio source I wanted to react to. I'd then add a wave source. The filter would extract it's data and store it in a global array that the wave source would access, making that struct(or OBS_DATA object) available in the wave source plugin.
The wave source would have a LUA script. That LUA script would have a per_frame and a per_point function defined. I want to call those functions and pass them the audio data structure, and have them return the data for the render loop. I imagine I'll create some struct for that data as well.
The per_frame will be called for every frame, and the per_point called for every point in the line(one of the values you would set in per_frame is how many points in the line, along with other render options.. maybe textures too). So the plugin will call the per_frame function and then do a loop for however many points were requested and call the per_point function for each index to get the data for that point.
My immediate goal is bringing in features from Milkdrop 2 into OBS so I can satisfy my weird need for music reactive elements in my stream. But Some of these tools would be useful in general, like just being able to write a script to render some shape with lines based off lua equations could be super useful even without the music part. I've also considered trying to pull other data into the scripts, like notifications so you could have a shape/line change somehow when someone subs or something, all based on code instead of pre-rendered somehow.
What I've done so far:
I've got the audio extraction plugin that pulls the audio data, runs an fft, and sums it up into bass/mid/treb variables and stores it in a global array.
I have a wave plugin that allows you to select which audio source you want to listen too and will extract that data from the global array.
The wave plugin has a setting for a lua script path, and it includes the obs-scripting library.
I'm able to load the script from the path with this code:
C:
#include <obs-scripting-internal.h>
#include <obs-scripting.h>
#include <obs-scripting-lua.h>
#include <lua.h>
.......
obs_data_t *settings = obs_source_get_settings(context->context);
const char *file_name =
obs_data_get_string(settings, "script_file_name");
struct obs_lua_script *script = obs_script_create(file_name, NULL);
lua_State *script_state = script->script;
context->script = script;Then I'm able to extract the functions from the script like this:
C:
if (context->script) {
pthread_mutex_lock(&script->mutex);
//luaL_openlibs(script);
//luaopen_ffi(script);
//extract per frame/point functions
lua_getglobal(script_state, "per_frame");
if (lua_isfunction(script_state, -1))
context->per_frame =
luaL_ref(script_state, LUA_REGISTRYINDEX);
else
context->per_frame = LUA_REFNIL;
lua_getglobal(script_state, "per_point");
if (lua_isfunction(script_state, -1))
context->per_point =
luaL_ref(script_state, LUA_REGISTRYINDEX);
else
context->per_point = LUA_REFNIL;
pthread_mutex_unlock(&script->mutex);
}This all came fairly easy so I thought I had this figured out, but C had other ideas.
The problem comes with trying to actually call the functions, pass them custom data structs and get custom data structs back.
I can't seem to use the call_func macro from obs-scripting, or the underlying function the macro uses(I tried copying the define into my file), it complains about something not existing inside the call_func_ function, so the external code.. so clearly my linking isn't right.. or obs-scripting wasn't designed to be linked, and I'm hitting walls because of that. Reading the LUA docs I was able to just write my own code to call the functions, so now I just need to get the data in and out.
I found ls_push_libobs_obj that seemed like it would solve all my problems, as I'm fine with dealing in OBS_DATA_T objects, so I rewrote the audio capture plugin to also create an obs data object with the music data.
but I get runtime errors in the underlying function there too. SWIG related, and likey caused by the hacky way I faked swig support(I think that's what you'd call what I did??) to get this to compile. Either the obs-scripting project took no considering for another plugin trying to load scripts with it, or I suck a C, or both.
So I just need a way to pass libobjs or c structs into lua functions and get them in return(or have the function modify them) and I can move on to the actual render loop, but I can't seem to figure out how to go about this.
I've looked into FFI, but that seems to be more for creating c structs inside lua instead of passing them in from C.
It seems like userData is the basic LUA feature I would need for this, but there's a decent amount of boiler to do there, and it's not how ls_push_libobs_obj was implemented, so that gives me pause on if it'll actually work for this.
Does anyone know of any open source plugins that load LUA scripts or have any ideas on easy ways to pass data structures into and out of LUA scripts?