Tips and tricks for Lua scripts

bfxdev

New Member
I start this thread to share my main advancements with Lua scripting. As of today (OBS v26.0.2 released October 6th 2020), the online documentation describes the OBS API in its original C form, and it is left as an exercise for the scripting enthusiast to figure out how data types are converted in the scripting environment. Sometimes it is not so trivial so here we are.

First of all: why Lua?

I started my OBS journey writing shaders in StreamFX (I like shaders). It is convenient for simple things but looks quickly cluttered with a large number of shader parameters. In addition it will not support dedicated pre-processing like extracting colors from an input picture. Other similar OBS plugins have the same limitations or are not working at all, because the last released version is not compatible anymore with the latest OBS version.

To implement new functionalities in OBS there are actually 3 options: plugins, Python scripts or Lua scripts.

Writing a plugin implies to setup a compilation environment, potentially for several target platforms (I don't have a mac), and to care about installation issues in various user environments. The hurdles linked to plugin development and maintenance make it no viable alternative for me. The only advantage of a plugin compared to a script is speed, but if shaders can be used for computing-intensive operations, then speed is no issue anymore.

Regarding the choice Python vs Lua, although I know Python very well and I'm convinced it is more powerful and compact than Lua, there are 2 main reasons to prefer Lua:
  • Lua is completely integrated into OBS, there is no need for an external scripting engine in a particular version (namely Python 3.6)
  • Lua supports adding OBS sources (input, filter or transition), typically using shaders, and apparently Python does not
In addition, Lua is the script language of the LÖVE2D game engine, Roblox and Wireshark, so learning it can be useful. The Lua syntax does not rely on indentation and is quite readable.

The main drawbacks of Lua are its poor standard library and the difficulty to add libraries. Actually I do not know how to add a non-pure-lua library to the Lua environment. But the OBS C API is full of platform-independent functions. The whole purpose of this thread is to share the tricks on how to use the OBS C API.
 

bfxdev

New Member
And to start with, this is a way to list directory entries with obslua, here to list all rst files in a directory stored in an OBS parameter called "obsdoc":

Lua:
-- Lists files in doc directory
local filenames = {}
local dir = obslua.os_opendir(obslua.obs_data_get_string(script_settings, "obsdoc"))
local entry
repeat
  entry = obslua.os_readdir(dir)
  if entry and not entry.directory and obslua.os_get_path_extension(entry.d_name)==".rst" then
    table.insert(filenames, obslua.obs_data_get_string(script_settings, "obsdoc") .. "/" .. entry.d_name)
  end
until not entry
obslua.os_closedir(dir)

The function obslua.os_opendir returns an object of type os_dir_t * treated as opaque userdata by Lua. Then, obslua.os_readdir returns the next entry in the directory (or nil if no more entry). This next entry is of type struct os_dirent or simply os_dirent in Lua.

There are getters and setters defined automatically in SWIG for struct data, including os_dirent. This structure is documented here: https://obsproject.com/docs/reference-libobs-util-platform.html?highlight=d_name#c.os_dirent
That is why after entry = obslua.os_readdir(dir) it is possible to use entry.d_name
 

bfxdev

New Member
I still have a small backlog of code to share so let's continue. Now a snippet to hide an existing property upon change of another property.

This is a common issue when you start adding different modes of operation for a script, e.g. with mode 1 you need to ask the user for a number, with mode 2 you need a color. You could just leave all properties displayed all the time, but it would be confusing for the user.

So let's say you have a property called mode, and you want to change some properties visibility when its value changes.
First, once your property is created, say in p, associate a callback to the property, here the callback function is named set_visibility:
obslua.obs_property_set_modified_callback(p, set_visibility)

The callback is defined in C as typedef bool (*obs_property_clicked_t)(obs_properties_t *props, obs_property_t *property, void *data);. In Lua, just define it as function set_visibility(props, property, settings).

In the callback, it is necessary to:
  • Retrieve the value of the selection: local mode = obslua.obs_data_get_int(settings, "mode")
  • Apply the visibility e.g. if mynumber should be displayed in mode 1: obslua.obs_property_set_visible(obslua.obs_properties_get(props, "mynumber"), mode==1)
  • Finally, and this is the most important here to remember (it took me literally hours to figure it out), return true from the callback to trigger the refresh of the properties!
Here we change the visibility but disabling/enabling the property can be done too with obs_property_set_enabled, or any other visible change.

This is the basis, but to get it work completely, i.e. to set the visibility at the very first display, it is necessary to call obs_properties_apply_settings using the settings caught e.g. in script_update

At the end it gives something like this:

visibility.gif


The full code (attached as well):
Lua:
-- This function is necessary to tell OBS it is a script
function script_description()
  return "Tips and tricks example code"
end


-- Called upon settings initialization and modification
my_settings = nil
function script_update(settings)
  -- Keep track of current settings
  my_settings = settings 
end

-- Displays a list of properties
function script_properties()

  local properties = obslua.obs_properties_create()

  -- Combo list filled with the options from MY_OPTIONS
  local p = obslua.obs_properties_add_list(properties, "mode", "My list",
              obslua.OBS_COMBO_TYPE_LIST, obslua.OBS_COMBO_FORMAT_INT)
  MY_OPTIONS = {"Mode 1", "Mode 2"}
  for i,v in ipairs(MY_OPTIONS) do
    obslua.obs_property_list_add_int(p, v, i)
  end

  -- Sets callback upon modification of the list
  obslua.obs_property_set_modified_callback(p, set_visibility)

  -- Integer option to be displayed in Mode 1
  obslua.obs_properties_add_int(properties, "mynumber", "My number in Mode 1", 1, 10, 1)

  -- Color option to be displayed in Mode 2
  obslua.obs_properties_add_color(properties, "mycolor", "My color in Mode 2")

  -- Calls the callback once to set-up current visibility
  obslua.obs_properties_apply_settings(properties, my_settings)
 
  return properties
end

-- Callback on list modification
function set_visibility(props, property, settings)

  -- Retrieves value selected in list
  local mode = obslua.obs_data_get_int(settings, "mode")
 
  -- Preset parameters
  obslua.obs_property_set_visible(obslua.obs_properties_get(props, "mynumber"), mode==1)
  obslua.obs_property_set_visible(obslua.obs_properties_get(props, "mycolor"), mode==2)

  -- IMPORTANT: returns true to trigger refresh of the properties
  return true
end
 

Attachments

  • tips-and-tricks.zip
    858 bytes · Views: 207

bfxdev

New Member
Another interesting hack I discovered: because OBS is based on QT, it is possible to format the script description with HTML.
Actually various text labels in the OBS properties can be pimped with fancy formatting. This would obviously work as well with Python or in a plugin (but where is the classical plugin description?).

The formatting is documented on the QT pages.

It looks like this for example:
Lua:
local alien="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABMAAAAVCAYAAACkCdXRAAAAAXNSR0IArs4c6QAAAARnQU1BAACxj"..
"wv8YQUAAAAJcEhZcwAADsIAAA7CARUoSoAAAADwSURBVDhPtZQxDsIwDEUDYoSBHWhHbsPC2tOxsnAbxhbYGWBEAn0rBid20lDBk1BS17z+hBT3"..
"S0Z+TFItq6efuu7cZfuTN1ky26/d9XCh2mR3pzElNYsQQSJhIYDUEqqCJWL6hGM/EjlRzKOZBvsJ3uZSkUwHZMIgWQnzzcLPNGTkVLftkYqMlTT"..
"uwXI5nUrWnlr6gPiLfC17JOYy61XtZx+BFMv7EiXjRuvJsmYJSYb14slyj6zmuCb3C9cq2TfnLCY4wSVnLfcWmD/AUIJkIJeu791UMmAJB/1rMB"..
"BihJRFkABLBJIyhqUgJfkDzr0Amw2KoGT2/LMAAAAASUVORK5CYII="

local description = [[
<center><h2>Tips and tricks for Lua scripts</h2></center>
<center><img width=38 height=42 src=']] .. alien .. [['/></center>
<center><a href="https://github.com/bfxdev/OBS">bfxdev</a> - 2020</center>
<p>Example code attached to the <a href=
"https://obsproject.com/forum/threads/tips-and-tricks-for-lua-scripts.132256/">
OBS Forum Thread "Tips and tricks for Lua scripts"</a>. You can format the description
with <strong>strong</strong>, <code>code</code>, <kbd>kbd</kbd> or
<a href="https://doc.qt.io/qt-5/richtext-html-subset.html">whatever is supported by QT</a>.</p>
<p>You can evan use an horizontal line to separate the description from the properties,
see below.<hr/></p>]]

-- Description displayed on the Tools->Scripts window
function script_description()
    return description
end

And the result:
1603393379046.png
 

Attachments

  • tips-and-tricks.zip
    1.6 KB · Views: 120

bfxdev

New Member
And another very cool one: the description of a property can as well contain QT-compliant HTML (see previous post). So it can contain a self-contained data URL or a link to a file, i.e. display the selected picture as description of an obslua.OBS_PATH_FILE property

First, define the property and associate a callback:

Lua:
  p = obslua.obs_properties_add_path(properties, "mypicture", "My picture", obslua.OBS_PATH_FILE,
    "Picture (*.png *.bmp)", nil)
  obslua.obs_property_set_modified_callback(p, set_picture_description)

Then write a callback that will change the description to contain an img tag with a URL starting with file::
Lua:
function set_picture_description(props, property, settings)
  local path = obslua.obs_data_get_string(settings, "mypicture")
  local desc = "<span valign=middle>My picture <img height=18 src='file:" .. path .. "'/></span>"
  obslua.obs_property_set_description(property, desc)
  return true
end

The result:
1603400972987.png


Unfortunately, it works only for BMP and PNG picture formats.

Happy Hacking!!
 

Attachments

  • tips-and-tricks.zip
    1.7 KB · Views: 97

bfxdev

New Member
Thanks to Warchamp7 we have now Lua code support in the forum. Lua listings will be more readable now. Time to look at some bigger code snippet: how to generate a gs_image_file object with content generated at run-time.

I searched a lot in the Lua bindings and type conversion in OBS to find a quick trick to access the pixel data of an image or a texture in Lua, but without success so far. So I followed a more general idea: generate a picture and read it. It can be done with temporary local file, or remote file, or even much better with a "data URL".

Using a data URL as a file name works somehow in OBS v26 on Windows, probably because OBS integrates ImageMagick. As an example, the following code uses a data URL as file name:
Lua:
local alien="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABMAAAAVCAYAAACkCdXRAAAAAXNSR0IArs4c6QAAAARnQU1BAACxj"..
"wv8YQUAAAAJcEhZcwAADsIAAA7CARUoSoAAAADwSURBVDhPtZQxDsIwDEUDYoSBHWhHbsPC2tOxsnAbxhbYGWBEAn0rBid20lDBk1BS17z+hBT3"..
"S0Z+TFItq6efuu7cZfuTN1ky26/d9XCh2mR3pzElNYsQQSJhIYDUEqqCJWL6hGM/EjlRzKOZBvsJ3uZSkUwHZMIgWQnzzcLPNGTkVLftkYqMlTT"..
"uwXI5nUrWnlr6gPiLfC17JOYy61XtZx+BFMv7EiXjRuvJsmYJSYb14slyj6zmuCb3C9cq2TfnLCY4wSVnLfcWmD/AUIJkIJeu791UMmAJB/1rMB"..
"BihJRFkABLBJIyhqUgJfkDzr0Amw2KoGT2/LMAAAAASUVORK5CYII="

local image = obslua.gs_image_file()
obslua.gs_image_file_init(image, alien)
print("Alien image: cx=" .. tostring(image.cx) .. "  cy=" .. tostring(image.cy))
obslua.gs_image_file_free(image)

It gives on the Script Log window: [tips-and-tricks.lua] Alien image: cx=19 cy=21

A data URL is simply a long text string containing the Base64 encoding of the content of a file, plus a text header. The difficulty is more to generate a picture file. BMP is a quite well documented format, which can support an alpha channel and does not require compression. Here is a good example with alpha.

Here is the code to generate an arbitrary picture as data URL:
Lua:
--- Returns a data URL representing a BMP RGBA picture of dimension `width` and `height`, and with bitmap `data` provided
--- as a one-dimensional array of 32-bits numbers (in order MSB to LSB Alpha-Red-Green-Blue) row-by-row starting on
--- the top-left corner.
--- @param width number
--- @param height number
--- @param data table
--- @return string
-- See https://docs.microsoft.com/en-us/windows/win32/gdi/bitmap-storage
-- See https://en.wikipedia.org/wiki/BMP_file_format#Example_2
function encode_bitmap_as_URL(width, height, data)
 
  -- Converts binary string to base64 from http://lua-users.org/wiki/BaseSixtyFour
  function encode_base64(data)
    local b='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
    return ((data:gsub('.', function(x)
      local r,b='',x:byte()
      for i=8,1,-1 do r=r..(b%2^i-b%2^(i-1)>0 and '1' or '0') end
      return r;
    end)..'0000'):gsub('%d%d%d?%d?%d?%d?', function(x)
      if (#x < 6) then return '' end
      local c=0
      for i=1,6 do c=c+(x:sub(i,i)=='1' and 2^(6-i) or 0) end
      return b:sub(c+1,c+1)
    end)..({ '', '==', '=' })[#data%3+1])
  end
 
  -- Packs 32-bits unsigned int into a string with little-endian encoding
  function pu32(v) return string.char(v%256, (v/256)%256, (v/65536)%256, (v/0x1000000)%256) end
 
  -- Prepared as table and then concatenated for performance
  local bmp = {}
 
  -- BITMAPFILEHEADER see https://docs.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapfileheader
  table.insert(bmp, "BM" .. pu32(width*height*4 + 122) .. pu32(0) .. pu32(122))
 
  -- BITMAPV4HEADER see https://docs.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapv4header
  table.insert(bmp, pu32(108) .. pu32(width) .. pu32(height) .. pu32(0x200001) .. pu32(3))
  table.insert(bmp, pu32(width*height*4) .. pu32(2835) .. pu32(2835) .. pu32(0) .. pu32(0))
  table.insert(bmp, pu32(0xFF0000) .. pu32(0xFF00) .. pu32(0xFF) .. pu32(0xFF000000) .. "Win ")
  for i = 1,12 do table.insert(bmp, pu32(0)) end
 
  -- Bitmap data (it starts with the lower left hand corner of the image)
  local offset
  for y = (height-1),0,-1 do
    offset = 1 + y*width
    for x = 0,(width-1) do
      table.insert(bmp, pu32(data[offset + x]))
    end
  end
 
  -- Finishes string
  bmp = table.concat(bmp, "")
 
  return "data:image/bmp;base64," .. encode_base64(bmp)
end

It can be used like this:
Lua:
local url = encode_bitmap_as_URL(3, 2, {0xFF0000FF, 0xFFFFFFFF, 0xFFFF0000, 0x7F0000FF, 0x7FFFFFFF, 0x7FFF0000})

local image = obslua.gs_image_file()
obslua.gs_image_file_init(image, url)
print("Data URL image: cx=" .. tostring(image.cx) .. "  cy=" .. tostring(image.cy))
obslua.gs_image_file_free(image)

It produces: [tips-and-tricks.lua] Data URL image: cx=3 cy=2

Of course it can be used for much larger images, but the time to generate might not be acceptable. I measured about 10 seconds for a 1000x1000 image on my AMD 3GHz. At some point there is also a size limitation, which remains to be exactly determined, but I observed that a LUT of 4096x4096 cannot be generated ("out of memory"). In case it is necessary to generate a very large image, then adapting the procedure to write into a temporary file is possible.
 

Attachments

  • tips-and-tricks.zip
    2.9 KB · Views: 93

upgradeQ

Member
Lua is also used in mpv (crossplatform media player), neovim (text editor), cheat engine (a memory editor/debugger)
Here is more detailed list https://en.wikipedia.org/wiki/List_of_applications_using_Lua

If you are using neovim, I suggest you to check out this plugin https://github.com/rafcamlet/nvim-luapad

Tips & tricks
Global sound
In OBS there is not yet crossplatform play sound API,
however you can use global sources with proper monitoring type to play sound in headphones.
In this example, on scene change "alert.mp3" will be triggered.

Lua:
local obs = obslua
mediaSource = nil -- Null pointer
outputIndex = 63 -- Last index

function play_sound()
  mediaSource = obs.obs_source_create_private("ffmpeg_source", "Global Media Source", nil)
  local s = obs.obs_data_create()
  obs.obs_data_set_string(s, "local_file",script_path() .. "alert.mp3")
  obs.obs_source_update(mediaSource,s)
  obs.obs_source_set_monitoring_type(mediaSource,obs.OBS_MONITORING_TYPE_MONITOR_AND_OUTPUT)
  obs.obs_data_release(s)

  obs.obs_set_output_source(outputIndex, mediaSource)
  return mediaSource
end

function obs_play_sound_release_source()
  r = play_sound()
  obs.obs_source_release(r)
end

function on_event(event)
  if event == obs.OBS_FRONTEND_EVENT_SCENE_CHANGED
    then obs_play_sound_release_source()
  end
end

function script_load(settings)
  obs.obs_frontend_add_event_callback(on_event)
end

function script_unload()
  obs.obs_set_output_source(outputIndex, nil)
end
This will create global source,load "alert.mp3" file from relative location of the script
and the most important one - will set monitoring type , so you can hear it in your headphones.
Lua:
  mediaSource = obs.obs_source_create_private("ffmpeg_source", "Global Media Source", nil)
  local s = obs.obs_data_create()
  obs.obs_data_set_string(s, "local_file",script_path() .. "alert.mp3")
  obs.obs_source_update(mediaSource,s)
  obs.obs_source_set_monitoring_type(mediaSource,obs.OBS_MONITORING_TYPE_MONITOR_AND_OUTPUT)
Based on this gist
from Palakis

Default hotkeys
It is possible to register hotkeys from json string, so any time script is loaded hotkeys will be predefined.
For example, this script will print true or false on toggle if 1 pressed.
Lua:
local obs = obslua

boolean = true

function toggle()
  print(tostring(boolean))
  boolean = not boolean
end

function htk_1_cb(pressed)
  if pressed then
    toggle()
  end
end

key_1 = '{"htk_1": [ { "key": "OBS_KEY_1" } ]}'
json_s = key_1
default_hotkeys = {
  {id='htk_1',des='Toggle something',callback=htk_1_cb},
}

function script_load(settings)

  s = obs.obs_data_create_from_json(json_s)
  for _,v in pairs(default_hotkeys) do
    local a = obs.obs_data_get_array(s,v.id)
    h = obs.obs_hotkey_register_frontend(v.id,v.des,v.callback)
    obs.obs_hotkey_load(h,a)
    obs.obs_data_array_release(a)
  end
  obs.obs_data_release(s)
end
 

bfxdev

New Member
Thanks for sharing! I heard about Luapad but never used it myself. Indeed the list of projects that rely on Lua is impressive (OBS should be listed there, World of Warcraft too), big surprise for me to see that even RPM embeds Lua scripting.

Now time to go into the details of the integration of Lua and type conversions in OBS, starting with a disclaimer: the information below may not be 100% correct as I have no idea about the details of the development or the rationale behind some technical choices. I just understood a couple of things through reading various pieces of documentation, looking at the code and re-compiling OBS.

OBS Scripting Documentation

The starting point is obviously the OBS documentation on scripting, where we can learn:
  • Lua in OBS is implemented based on LuaJIT 2.
  • We should take care about memory leaks (i.e. always release allocated memory)
  • The main interface between OBS and the script are several global functions called by OBS at different stages like script_load, script_defaults, script_update, etc. Nevertheless on this page there is an important non-documented function: script_description, which returns the description as a string and can be formatted as HTML, see my previous posts. script_description is the function that tells OBS that it is a valid script.
  • Again, new sources can be created with Lua (and not Python, unclear to me why). There is a short example of a new source.
  • Some functions of the API were re-written to work in the scripting environment, most of them to be able to cope with callbacks, and two of them are available for Lua only.
After reading this, the best way to get started is to look at such a page about starting to write a plugin with Lua or find an example script similar to what you want and modify it, which is enough for most features. For more complicated stuff, we need to get into the details.

Standard and LuaJIT-specific libraries

An important thing to look at are the extensions of LuaJIT and the Lua standard libraries. These libraries are pre-imported, so no require statement is necessary (but the LuaJIT documentation for bit says that writing the require is a good practice):
  • LuaJIT-specific bit library for bitwise-operations
  • LuaJIT-specific ffi or FFI library for "calling external C functions and using C data structures from pure Lua code" (more about this one later on)
  • LuaJIT-specific jit library for managing the Just-In-Time compiler, if needed
  • Standard string library for string manipulation including searching for Lua-flavoured patterns (not regexp)
  • Standard table library for table manipulation
  • Standard math library for mathematical functions
  • Standard io library for input/output facilities
  • Standard os library for Operating System relevant functions
  • Standard debug library for access to the stack, the metatable or to activate the interactive debug mode
  • Standard coroutine library for multi-threading
  • Last but not least of course the obslua library
The list of global variables and functions can be printed to the Script Log window with the following code:
Lua:
for key, value in pairs(_G) do
  print("Global " .. type(value) .. ": " .. key .. " = " .. tostring(value))
end

Apart from the mentioned libraries, which appear with the type "table", and the classical global functions available in Lua (print, assert, etc) some interesting global functions can be observed in this list:
Code:
[...]
[tips-and-tricks.lua] Global function: swig_equals = function: 0x22c4b4e0
[tips-and-tricks.lua] Global function: swig_type = function: 0x22c4b490
[tips-and-tricks.lua] Global function: script_path = function: 0x22c91130
[tips-and-tricks.lua] Global function: newproxy = function: builtin#28
[...]

For script_path it is clear, it is a documented OBS function. newproxy is a deprecated, non-documented Lua function. More interesting are swig_type and swig_equals, they show that Lua is integrated in OBS using SWIG.

SWIG - Simplified Wrapper and Interface Generator

"The primary purpose of SWIG is to simplify the task of integrating C/C++ with other programming languages" says the documentation. In the frame of OBS, SWIG generates the function bindings in Lua and Python for all C functions in OBS (better said the exported ones) during the compilation.

As far as I understand, it works like this:
  • The C functions expected to be available in Lua as bindings are defined through their header .h files in the obslua.i file (functions with a script-specific implementation are listed in the file two, with the prefix %ignore)
  • From this file, SWIG generates a large C file at obs-studio/build/deps/obs-scripting/obslua/CMakeFiles/obslua.dir/obsluaLUA_wrap.c with the function binding definitions. This file is compiled after generation.
  • Functions with a script-specific implementation are defined in the file obs-scripting-lua.c, in a similar way as SWIG generates the function bindings.
The obsluaLUA_wrap.c file shows how SWIG interprets the C code and converts types between C and Lua. For the cases where his conversion is not trivial, SWIG has to make a choice. That is why this file is fundamental to know the expected types for a given function (the file is attached).

Let's take the wrapper of the C function void obs_data_set_string(obs_data_t *data, const char *name, const char *val):
C:
static int _wrap_obs_data_set_string(lua_State* L) {
  int SWIG_arg = 0;
  obs_data_t *arg1 = (obs_data_t *) 0 ;
  char *arg2 = (char *) 0 ;
  char *arg3 = (char *) 0 ;
 
  SWIG_check_num_args("obs_data_set_string",3,3)
  if(!SWIG_isptrtype(L,1)) SWIG_fail_arg("obs_data_set_string",1,"obs_data_t *");
  if(!SWIG_lua_isnilstring(L,2)) SWIG_fail_arg("obs_data_set_string",2,"char const *");
  if(!SWIG_lua_isnilstring(L,3)) SWIG_fail_arg("obs_data_set_string",3,"char const *");
 
  if (!SWIG_IsOK(SWIG_ConvertPtr(L,1,(void**)&arg1,SWIGTYPE_p_obs_data,0))){
    SWIG_fail_ptr("obs_data_set_string",1,SWIGTYPE_p_obs_data);
  }
 
  arg2 = (char *)lua_tostring(L, 2);
  arg3 = (char *)lua_tostring(L, 3);
  obs_data_set_string(arg1,(char const *)arg2,(char const *)arg3);
 
  return SWIG_arg;
 
  if(0) SWIG_fail;
 
fail:
  lua_error(L);
  return SWIG_arg;
}

It shows us that the argument 1 of type obs_data_t * is kept as a pointer ((SWIG_ConvertPtr(L,1,(void**)&arg1,SWIGTYPE_p_obs_data,0)). Lua will present it as userdata, i.e. it is an allocated area of the memory at the address whose value is contained in the pointer. In the SWIG C code, the variable has the special userdata "type" SWIGTYPE_p_obs_data.

According to the Lua documentation: A userdata value represents a block of raw memory. There are two kinds of userdata: full userdata, which is an object with a block of memory managed by Lua, and light userdata, which is simply a C pointer value. It is not clear to me if the obs_data is light or full userdata. I see no allocation managed by Lua here, so I bet it is light userdata, so we have to free it when necessary (but most of the time OBS takes care of it... I hope at least).

The argument 2 and 3 of type const char * are retrieved from strings (e.g. arg2 = (char *)lua_tostring(L, 2);).

Again, this is my understanding, and I'm far from following all details of this wrapper function, but this superficial look at the code is sufficient to follow the types.

Let's take another example with the function int os_get_config_path(char *dst, size_t size, const char *name), (don't ask me what the function is useful for, I don't know), where SWIG expects a string as argument dst, to be filled with the new data. Normally there is no concept of passing a string "by reference" in Lua, because Lua strings are immutable (it means that if you want to add a single character to an existing string, then a new memory block needs to be allocated, the content of the source string needs to be copied there etc).

The best is to give it a try by pre-filling dst with spaces:
Lua:
-- Tries to use os_get_config_path
local dst = "                                                                                                       "
print("Before call to os_get_config_path: dst=" .. dst)
obslua.os_get_config_path(dst, #dst, "OBS\\sceneCollection\\")
print("After call to os_get_config_path: dst=" .. dst)

It works! The log shows After call to os_get_config_path: dst=C:\Users\bfxdev\AppData\Roaming\OBS\sceneCollection\. I'm not sure if this behavior is very idiomatic for Lua. I can imagine a number of corner cases leading to buffer overflows. In the case of int os_get_config_path(char *dst, size_t size, const char *name), if the buffer is too short (according to the given size), then the destination string is left untouched.

In the wrapper of the function, we can see that the buffer of the destination string is provided using lua_tostring like other input strings. There is not conversion back so the content of the string is modified in place (code reduced for brevity):
C:
static int _wrap_os_get_config_path(lua_State* L) {
[..]
  if(!SWIG_lua_isnilstring(L,1)) SWIG_fail_arg("os_get_config_path",1,"char *");
[..]
  arg1 = (char *)lua_tostring(L, 1);
[..]
}


Setters and getters

By exploring the C wrapper file, I noticed functions like:
C:
static int _wrap_new_gs_image_file(lua_State* L) {
[..]
  SWIG_check_num_args("gs_image_file::gs_image_file",0,0)
[..]
}

static int _wrap_gs_image_file_cx_set(lua_State* L) {
[..]
  SWIG_check_num_args("gs_image_file::cx",2,2)
[..]
}

These functions correspond to the struct gs_image_file, which has no detailed documentation. The full definition can be found in the source code of OBS, here an extract:
C:
struct gs_image_file {
    gs_texture_t *texture;
    enum gs_color_format format;
    uint32_t cx;
    uint32_t cy;
    bool is_animated_gif;
    bool frame_updated;
    bool loaded;
[..]
};

Here again, SWIG is smart enough to create new/get/set bindings to some of the struct objects (not all). Such functions correspond to this kind of Lua code:
Lua:
local image = obslua.gs_image_file()
obslua.gs_image_file_init(image, url)
print("Data URL image: cx=" .. tostring(image.cx) .. "  cy=" .. tostring(image.cy))
obslua.gs_image_file_free(image)

As a comparison, the struct obs_data is defined without any member: struct obs_data; . This is probably the reason why SWIG does not create bindings.

Conclusion

That's all for now. There is still much to say on the bindings created by SWIG but for the common cases remember that:
  • The exact Lua types for C functions can be found in the files obs-scripting-lua.c and obsluaLUA_wrap.c (attached)
  • Explicitly defined struct objects like gs_image_file need to be created first (local image = obslua.gs_image_file()), and the members can bet accessed
  • Other struct objects, called "reference-counted objects", are created through OBS functions (see e.g. obs_data_create) and the members cannot be accessed directly
  • The struct objects are passed by reference transparently, that is why they are so easy to use in Lua
  • Strings are converted wherever necessary by SWIG (from/to char * or const char *), and unlike immutable pure-Lua strings, their content may be modified in place by the C function
  • Enums are just defined as constants of the obslua module, e.g. obslua.OBS_SOURCE_VIDEO is valid
  • In certain cases, a function may not be usable at all due to the bindings. This will be the purpose of another post.
Happy hacking!
 

Attachments

  • obsluaLUA_wrap.zip
    137.3 KB · Views: 93

upgradeQ

Member
Hotkeys addendum
Hotkey press/release
With common on hotkey do something / on hotkey toggle something using scripting we can extend it to :
  • Do something while holding hotkey
  • Do something after holding hotkey for some period of time
Lua:
-- hotkey_hold.lua
local obs = obslua
holding = true
one_time = true
timer = 0

function htk_1_cb(pressed)
  if pressed then
    holding = true
  else
    holding = false
  end
end

function check_timer()
  if timer > 1.3 and one_time then
    print('activated after 1.3 sec')
    one_time = false
  end
end

function script_tick(seconds)
  if holding then
    timer = timer + seconds
    check_timer()
  else -- reset
    timer = 0
    one_time = true
  end
end

key_1 = '{"htk_1": [ { "key": "OBS_KEY_1" } ]}'
json_s = key_1
default_hotkeys = {
  {id='htk_1',des='Hold 1.3 sec',callback=htk_1_cb},
}

function script_load(settings)
  s = obs.obs_data_create_from_json(json_s)
  for _,v in pairs(default_hotkeys) do
    local a = obs.obs_data_get_array(s,v.id)
    h = obs.obs_hotkey_register_frontend(v.id,v.des,v.callback)
    obs.obs_hotkey_load(h,a)
    obs.obs_data_array_release(a)
  end
  obs.obs_data_release(s)
end
In the above example, after holding hotkey 1.3 seconds, script will print to the console.

Send hotkey to obs

This one is more on how to use OBS C API. There is a project obs-websocket which does that, reading it's protocol I've noticed that there is a two API regarding hotkeys:
TriggerHotkeyByName and TriggerHotkeyBySequence, the first one uses special syntax for hotkey indexing and there is no way to replicate it in Lua, however for the second one it is possible to adapt the code to Lua. I suggest you to read about original implementation and decisions made in pull by LorenaGdL

Lua:
-- send_hotkey.lua
local obs = obslua
local bit = require('bit')

function send_hotkey(hotkey_id_name,key_modifiers)
  shift = key_modifiers.shift or false
  control = key_modifiers.control or false
  alt = key_modifiers.alt or false
  command = key_modifiers.command or false
  modifiers = 0

  if shift then modifiers = bit.bor(modifiers,obs.INTERACT_SHIFT_KEY ) end
  if control then modifiers = bit.bor(modifiers,obs.INTERACT_CONTROL_KEY ) end
  if alt then modifiers = bit.bor(modifiers,obs.INTERACT_ALT_KEY ) end
  if command then modifiers = bit.bor(modifiers,obs.INTERACT_COMMAND_KEY ) end

  combo = obs.obs_key_combination()
  combo.modifiers = modifiers
  combo.key = obs.obs_key_from_name(hotkey_id_name)

  if not modifiers and
    (combo.key == obs.OBS_KEY_NONE or combo.key >= obs.OBS_KEY_LAST_VALUE) then
    return error('invalid key-modifier combination')
  end

  obs.obs_hotkey_inject_event(combo,false)
  obs.obs_hotkey_inject_event(combo,true)
  obs.obs_hotkey_inject_event(combo,false)
end

function start(props,p)
  send_hotkey("OBS_KEY_3",{shift=true})
end

function stop(props,p)
  send_hotkey("OBS_KEY_2",{shift=true})
end

function script_properties()
  props = obs.obs_properties_create()
  obs.obs_properties_add_button(props, "button1", "Start", start)
  obs.obs_properties_add_button(props, "button2", "Stop", stop)
  return props
end

In the above example, after pressing Start or Stop it will send shift + 3 , shift + 2 accordingly (you can set those hotkeys for show/hide source to check )
You can get those hotkey codes by reading this file
Alternatively you can set hotkey combination by yourself, exit from OBS, open either ~/basic/profiles/your_profile.ini or ~/basic/profiles/scenes/your_scene.json search for "hotkeys" and get those codes.
Note about TriggerHotkeyByName: in obslua obs_hotkey_trigger_routed_callback(hotkey_id,bool) needs to be called like in example above, false/true/false , but it's pretty much useless since you can't get hotkey id, though it will execute that routed callback properly , try using hotkey_id in range 0 till 30
 

bfxdev

New Member
Today a post about the difficulties with some pointer-related types in the current implementation of OBS Lua scripting. No trick, no tip.

As mentioned in my previous post (where I attached the C wrapper file), scripting in OBS is based on SWIG. It makes a very good job at building bindings based on the C code, but some C constructs just cannot be converted properly due to the inherent ambiguity of C pointers. i.e. because of pointer/array equivalence or arguments passed by reference. In some cases, SWIG would provide mechanisms to cope with that.

Let's look at some examples and try to categorize by data type.

Type char *

The type char * is commonly used to manipulate character strings like in strcpy (please note that the standard C does not bother too much with "const"). The type char * has several interpretations when passed as argument to a function:
  • Memory area where the function can write some character-typed data
  • Memory area containing a zero-ended sequence of characters (a classical C character string) to be read by the function. Normally it should be typed const char * or char const * (both are equivalent).
  • Pointer to a single character that can be changed by the function, or not (again a const would be expected if read-only)
  • Array of characters, where a zero has no special meaning (in this case the type should be char[], or const char[] if it is read-only)
Obviously, depending on this interpretation, the C code would be different and the type used in Lua should reflect it, but SWIG interprets any char * or const char * as a Lua string. It works smoothly in most cases.

In the function seen previously int os_get_config_path(char *dst, size_t size, const char *name) (used internally by OBS to retrieve e.g. Roaming AppData on Windows, thanks WizardCM), the destination dst is a pointer on a memory block where the function will copy the resulting string content. In other words it is a character string passed by reference, which is not supported in Lua.

This violates the immutability of Lua strings: the data of the string is replaced at its current memory location, i.e. without new allocation. The size of the allocated memory area can be passed in the size argument, but obviously the string passed to the function needs to contain already a number of characters at least equal to size, such that enough space is allocated to prevent a buffer overflow.

What could be a better binding for this function? There is no simple answer because there is just no concept of passing arguments by reference in Lua. Actually, SWIG foresees a smart Lua-style (or Python-style) solution: redefining the arguments as OUTPUT, i.e. the variables passed by reference are not modified in place but multiple values are returned.

Another example is the function char *os_generate_formatted_filename(const char *extension, bool space, const char *format). It returns a new bmalloc-allocated filename generated from specific formatting.

In the corresponding wrapper code, the memory is allocated by the function, then the content is copied in a new Lua-controlled memory block via lua_pushstring, but at the end the allocated memory block is systematically leaked:
C:
static int _wrap_os_generate_formatted_filename(lua_State* L) {
[...]
  result = (char *)os_generate_formatted_filename((char const *)arg1,arg2,(char const *)arg3);
  lua_pushstring(L,(const char *)result); SWIG_arg++;
  return SWIG_arg;
[...]

Actually SWIG provides a feature called "%newobject" to cope with memory allocation within a function. Slowly I'm thinking again about a fix for OBS scripting (but I tried a bit and I know it is not so easy to put in place given the number of functions in OBS).


Types uint8_t * and uint32_t *

The situation is different with pointers on data buffers. They can be found in a number of functions and getters/setters (no occurrence of uint16_t * so far).

First example, for the undocumented member texture_data of gs_image_file, SWIG creates a "set" function:
C:
static int _wrap_gs_image_file_texture_data_set(lua_State* L) {
[...]
  if(!SWIG_isptrtype(L,2)) SWIG_fail_arg("gs_image_file::texture_data",2,"uint8_t *");
[...]
  if (!SWIG_IsOK(SWIG_ConvertPtr(L,2,(void**)&arg2,SWIGTYPE_p_unsigned_char,SWIG_POINTER_DISOWN))){
    SWIG_fail_ptr("gs_image_file_texture_data_set",2,SWIGTYPE_p_unsigned_char);
  }

  if (arg1) (arg1)->texture_data = arg2;
 [...]

And a "get" function:
C:
static int _wrap_gs_image_file_texture_data_get(lua_State* L) {
[...]
  uint8_t *result = 0 ;
[...]
  result = (uint8_t *) ((arg1)->texture_data);
  SWIG_NewPointerObj(L,result,SWIGTYPE_p_unsigned_char,0); SWIG_arg++;
  return SWIG_arg;
[...]

SWIG manages the data internally as SWIGTYPE_p_unsigned_char (not as a Lua string) and uses various pointer conversion functions . The big question is now: how to transfer data between Lua and this buffer? I have no answer so far.

One promising alternative is the very low-level ffi library. It can allocate a buffer of particular type, i.e. like in the following code:
Lua:
ffi = require("ffi")
obslua.obs_enter_graphics()
local image = obslua.gs_image_file()
image.cx = 1
image.cy = 1
image.texture_data = ffi.new("uint8_t[?]", 4)
Unfortunately this code does not work. The allocated buffer is a "cdata" type, not userdata.
The execution stops with this error: Error running file: Error in gs_image_file::texture_data (arg 2), expected 'uint8_t *' got 'cdata'

It seems that there is no foreseen way to convert cdata into userdata. Now crafting userdata from Lua seems to be possible with the crazy pure-Lua "luastate" library. It is able to re-create the complete binary object used in Lua bindings. This would be the ultimate hack for all problems! But way too complex to use I think.

Reading data from a buffer seems to be supported by ffi.string(ptr [,len] ). The following code can be executed without error (hex_dump is a function to display bytes from a string):
Lua:
ffi = require("ffi")
local url = encode_bitmap_as_URL(3, 2, {0xFF0000FF, 0xFFFFFFFF, 0xFFFF0000, 0x7F0000FF, 0x7FFFFFFF, 0x7FFF0000})
obslua.obs_enter_graphics()
local image = obslua.gs_image_file()
obslua.gs_image_file_init(image, url)
local str = ffi.string(image.texture_data, image.cx*image.cy*4)
if str then print("Image data as string: " .. hex_dump(str)) end
obslua.obs_leave_graphics()
But again, unfortunately, this is no solution. For a reason I cannot follow, the returned string does not contain the expected data: Image data as string: 08 f2 d1 54 fc 7f 00 00 00 00 00 00 00 00 00 00 00 5a 8d 95 99 02 00 00
There is maybe still something to be converted (e.g. getting a pointer on the buffer referenced by the userdata), I don't see how.

Another example with the function void gs_get_size(uint32_t *cx, uint32_t *cy) and this wrapper:
C:
static int _wrap_gs_get_size(lua_State* L) {
[...]
  if (!SWIG_IsOK(SWIG_ConvertPtr(L,1,(void**)&arg1,SWIGTYPE_p_unsigned_int,0))){
    SWIG_fail_ptr("gs_get_size",1,SWIGTYPE_p_unsigned_int);
  }
  if (!SWIG_IsOK(SWIG_ConvertPtr(L,2,(void**)&arg2,SWIGTYPE_p_unsigned_int,0))){
    SWIG_fail_ptr("gs_get_size",2,SWIGTYPE_p_unsigned_int);
  }
   gs_get_size(arg1,arg2);
   return SWIG_arg;
[...]
}
No need to try, the pointers passed as reference are interpreted as inputs, and the result of the function is lost.

Pointer-pointer types

This was my main concern as I started to explore Lua bindings, and the reason I wrote an issue on GitHub.

With the function bool gs_texture_map(gs_texture_t *tex, uint8_t **ptr, uint32_t *linesize), as we saw previously, there is no direct way to create a userdata with type uint8_t **. Even if some variable or pointer could be passed, the result would be lost.

The function gs_effect_t *gs_effect_create(const char *effect_string, const char *filename, char **error_string) is a bit more interesting as it deals with a character string. The wrapper includes:
C:
  if (!SWIG_IsOK(SWIG_ConvertPtr(L,2,(void**)&arg2,SWIGTYPE_p_p_char,0))){
    SWIG_fail_ptr("gs_effect_create_from_file",2,SWIGTYPE_p_p_char);
  }
The pointer-pointer is not converted into a string so it is interpreted as input only.

Conclusion

This post was frustrating to write! This is a sequence of dead-ends.

Fortunately, most of the functions in the OBS API are usable and with some workarounds (e.g. the BMP loading) I did not identify real blocking points so far. But someone may want to read a picture file and retrieve its RGBA data or get the errors from an effect file compilation. In my opinion there is no possibility to use the related functions in the OBS API for that.

I see two approaches still to explore now:
  • Adapt the SWIG binding rules, risking to break existing Lua plugins
  • Explore the possibilities of the FFI library to call directly the C functions without going through the SWIG bindings. That could work.
That's all for today.
 

bfxdev

New Member
Still trying to get something useful working with FFI (now I experience just just many crashes).
UpgradeQ pointed out an interesting use of FFI to enumerate properties in another forum thread by MacTartan.

I copy it here with better source code highlighting.

From the gist created by UgradeQ: create a source with name "tmp" , then add a Color Correction filter named "color"
It will print to the console properties names after loading of OBS has been finished.

Lua:
local obs = obslua
local ffi = require("ffi")
local obsffi

ffi.cdef[[

struct obs_source;
struct obs_properties;
struct obs_property;
typedef struct obs_source obs_source_t;
typedef struct obs_properties obs_properties_t;
typedef struct obs_property obs_property_t;

obs_source_t *obs_get_source_by_name(const char *name);
obs_source_t *obs_source_get_filter_by_name(obs_source_t *source, const char *name);
obs_properties_t *obs_source_properties(const obs_source_t *source);
obs_property_t *obs_properties_first(obs_properties_t *props);
bool obs_property_next(obs_property_t **p);
const char *obs_property_name(obs_property_t *p);
void obs_properties_destroy(obs_properties_t *props);
void obs_source_release(obs_source_t *source);

]]

if ffi.os == "OSX" then
    obsffi = ffi.load("obs.0.dylib")
else
    obsffi = ffi.load("obs")
end

local function filterTest()
    local source = obsffi.obs_get_source_by_name("tmp")
    if source then
        local fSource = obsffi.obs_source_get_filter_by_name(source, "color")
        if fSource then
            local props = obsffi.obs_source_properties(fSource)
            if props then
                local prop = obsffi.obs_properties_first(props)
                local name = obsffi.obs_property_name(prop)
                if name then
                    local propCount = 1
                    obs.script_log(obs.LOG_INFO, string.format("Property 1 = %s", ffi.string(name)))
                    local _p = ffi.new("obs_property_t *[1]", prop)
                    local foundProp = obsffi.obs_property_next(_p)
                    prop = ffi.new("obs_property_t *", _p[0])
                    while foundProp do
                        propCount = propCount + 1
                        name = obsffi.obs_property_name(prop)
                        obs.script_log(obs.LOG_INFO, string.format("Property %d = %s", propCount, ffi.string(name)))
                        _p = ffi.new("obs_property_t *[1]", prop)
                        foundProp = obsffi.obs_property_next(_p)
                        prop = ffi.new("obs_property_t *", _p[0])
                    end
                end
                obsffi.obs_properties_destroy(props)
            end
            obsffi.obs_source_release(fSource)
        end
        obsffi.obs_source_release(source)
    end
end

After starting OBS it produces:
Code:
[tips-and-tricks.lua] Property 1 = gamma
[tips-and-tricks.lua] Property 2 = contrast
[tips-and-tricks.lua] Property 3 = brightness
[tips-and-tricks.lua] Property 4 = saturation
[tips-and-tricks.lua] Property 5 = hue_shift
[tips-and-tricks.lua] Property 6 = opacity
[tips-and-tricks.lua] Property 7 = color

Thanks MacTartan for sharing it!
 

bfxdev

New Member
I found different behaviors of the string.char function depending on the OBS executable.
This function takes an integer and transform it into a single char string. Now what happens if you pass a non-integer?

In OBS 64 bits, a "floor" is applied to the input of string.char such that string.byte(string.char(255.99)) gives 255

In OBS 32 bits, a "round" is applied to the input of string.char such that string.byte(string.char(255.99)) gives bad argument #1 to 'char' (invalid value) because it is trying to convert 256 to a single char.

I noticed this by opportunity while checking if the same behavior can be seen with FFI on 32-bits and 64-bits versions (and yes it crashes similarly! I'm still doing something wrong). In turn, I have to correct the function I published earlier that converts a Lua table in a BMP Data URL (or better said Data URI as on the English Wikipedia page).

This is the corrected code working with 32-bits and 64-bits OBS versions, with URI naming:

Lua:
--- Returns a data URI representing a BMP RGBA picture of dimension `width` and `height`, and with bitmap `data` provided
--- as a one-dimensional array of 32-bits numbers (in order MSB to LSB Alpha-Red-Green-Blue) row-by-row starting on
--- the top-left corner.
--- @param width number
--- @param height number
--- @param data table
--- @return string
-- See https://docs.microsoft.com/en-us/windows/win32/gdi/bitmap-storage
-- See https://en.wikipedia.org/wiki/BMP_file_format#Example_2
function encode_bitmap_as_URI(width, height, data)
 
  -- Converts binary string to base64 from http://lua-users.org/wiki/BaseSixtyFour
  function encode_base64(data)
    local b='ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
    return ((data:gsub('.', function(x)
      local r,b='',x:byte()
      for i=8,1,-1 do r=r..(b%2^i-b%2^(i-1)>0 and '1' or '0') end
      return r;
    end)..'0000'):gsub('%d%d%d?%d?%d?%d?', function(x)
      if (#x < 6) then return '' end
      local c=0
      for i=1,6 do c=c+(x:sub(i,i)=='1' and 2^(6-i) or 0) end
      return b:sub(c+1,c+1)
    end)..({ '', '==', '=' })[#data%3+1])
  end
 
  -- Packs 32-bits unsigned int into a string with little-endian encoding
  function pu32(v)
    return string.char(v%256, math.floor(v/256)%256, math.floor(v/65536)%256, math.floor(v/0x1000000)%256)
  end
 
  -- Prepared as table and then concatenated for performance
  local bmp = {}
 
  -- BITMAPFILEHEADER see https://docs.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapfileheader
  table.insert(bmp, "BM" .. pu32(width*height*4 + 122) .. pu32(0) .. pu32(122))
 
  -- BITMAPV4HEADER see https://docs.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-bitmapv4header
  table.insert(bmp, pu32(108) .. pu32(width) .. pu32(height) .. pu32(0x200001) .. pu32(3))
  table.insert(bmp, pu32(width*height*4) .. pu32(2835) .. pu32(2835) .. pu32(0) .. pu32(0))
  table.insert(bmp, pu32(0xFF0000) .. pu32(0xFF00) .. pu32(0xFF) .. pu32(0xFF000000) .. "Win ")
  for i = 1,12 do table.insert(bmp, pu32(0)) end
 
  -- Bitmap data (it starts with the lower left hand corner of the image)
  local offset
  for y = (height-1),0,-1 do
    offset = 1 + y*width
    for x = 0,(width-1) do
      table.insert(bmp, pu32(data[offset + x]))
    end
  end
 
  -- Finishes string
  bmp = table.concat(bmp, "")
 
  return "data:image/bmp;base64," .. encode_base64(bmp)
end

That's all for now.
 

Mega64

New Member
Hey, thanks a LOT for this thread. I've been getting into OBS scripting myself in the last couple days but the API documentation leaves some stuff to be desired haha, this is VERY helping.

One thing I just don't understand is creating new sources. You CAN create sources in Python judging by what I've been trying with the obs_source_create function; what you cannot do is make them show up in the OBS GUI along with the other sources. I figured maybe I had to register the source instead with the Lua-exclusive function, but this has not worked either. Soooo yeah, creating new sources has left me quite blocked lmao.
 

bfxdev

New Member
Thanks for feedback @Mega64 (and I'm sure @upgradeQ will appreciate too). The thread feels a bit like a blog but yes, this is exactly its purpose, help to understand the API bindings. BTW I often look at the Discord forum #scripting. It is always a good place for questions and interactions. Interesting to know that it is actually possible to create new sources in Python! There is probably a hack to let OBS display Python sources. But well, I favor Lua for scripting, I never tried Python in OBS.

I'm almost done with the backlog of things I wanted to share (I still want to write a big post about my development environment with auto-completion) and still exploring FFI, today with working code showing the Retrieval of bitmap data from an FFI-created gs_image_file structure.

We start with the import of the the FFI library and the loading of OBS code (original code thanks to @MacTartan).
Lua:
local ffi = require("ffi")
local obsffi
if ffi.os == "OSX" then
    obsffi = ffi.load("obs.0.dylib")
else
    obsffi = ffi.load("obs")
end

The C definition code is veeery long, even after simplification. We need to define the gs_image_file_init and gs_image_file_free functions, plus the members of struct gs_image_file, which in turn contains lots of animated GIF stuff that we do not use. At the end, FFI needs to know this definition just to determine the data size and finally the offsets in memory of the members of the structure:
Lua:
ffi.cdef([[
  enum gs_color_format {GS_UNKNOWN};
  typedef enum {GIF_OK = 0} gif_result;

  typedef void* (*gif_bitmap_cb_create)(int width, int height);
  typedef void (*gif_bitmap_cb_destroy)(void *bitmap);
  typedef unsigned char* (*gif_bitmap_cb_get_buffer)(void *bitmap);
  typedef void (*gif_bitmap_cb_set_opaque)(void *bitmap, bool opaque);
  typedef bool (*gif_bitmap_cb_test_opaque)(void *bitmap);
  typedef void (*gif_bitmap_cb_modified)(void *bitmap);
  typedef struct gif_bitmap_callback_vt
  {
    gif_bitmap_cb_create bitmap_create;
    gif_bitmap_cb_destroy bitmap_destroy;
    gif_bitmap_cb_get_buffer bitmap_get_buffer;
    gif_bitmap_cb_set_opaque bitmap_set_opaque;
    gif_bitmap_cb_test_opaque bitmap_test_opaque;
    gif_bitmap_cb_modified bitmap_modified;
  } gif_bitmap_callback_vt;
 
  typedef struct gif_frame gif_frame;

  typedef struct gif_animation
  {
    gif_bitmap_callback_vt bitmap_callbacks;
    unsigned char *gif_data;
    unsigned int width;
    unsigned int height;
    unsigned int frame_count;
    unsigned int frame_count_partial;
    gif_frame *frames;
    int decoded_frame;
    void *frame_image;
    int loop_count;
    gif_result current_error;

    unsigned int buffer_position;
    unsigned int buffer_size;
    unsigned int frame_holders;
    unsigned int background_index;
    unsigned int aspect_ratio;
    unsigned int colour_table_size;
    bool global_colours;
    unsigned int *global_colour_table;
    unsigned int *local_colour_table;
 
    unsigned char buf[4];
    unsigned char *direct;

    int table[2][(1 << 12)];
    unsigned char stack[(1 << 12) * 2];
    unsigned char *stack_pointer;
    int code_size, set_code_size;
    int max_code, max_code_size;
    int clear_code, end_code;
    int curbit, lastbit, last_byte;
    int firstcode, oldcode;
    bool zero_data_block;
    bool get_done;
    bool clear_image;

  } gif_animation;

typedef struct gs_texture gs_texture_t;

typedef struct gs_image_file
{
  gs_texture_t *texture;
  enum gs_color_format format;
  uint32_t cx;
  uint32_t cy;
  bool is_animated_gif;
  bool frame_updated;
  bool loaded;

  gif_animation gif;
  uint8_t *gif_data;
  uint8_t **animation_frame_cache;
  uint8_t *animation_frame_data;
  uint64_t cur_time;
  int cur_frame;
  int cur_loop;
  int last_decoded_frame;

  uint8_t *texture_data;
  gif_bitmap_callback_vt bitmap_callbacks;
} gs_image_file_t;

void gs_image_file_init(gs_image_file_t *image, const char *file);

void gs_image_file_free(gs_image_file_t *image);

]])
Actually the C definition block does not need simplification (like for the enumerations at the beginning of the block), this is just cosmetic. It is primarily a copy of blocks from various C header files in the OBS repository.

Nevertheless, there is no C preprocessor in FFI so macros (#define etc) are not supported and need to be adapted from the code. As an example, in the line int table[2][(1 << 12)]; the "12" was originally a macro definition.

Now let's go to the interesting part:
Lua:
local uri = encode_bitmap_as_URI(3, 2, {0xFF0000FF, 0xFFFFFFFF, 0xFFFF0000, 0x7F0000FF, 0x7FFFFFFF, 0x7FFF0000})
obslua.obs_enter_graphics()
local image = ffi.new("gs_image_file_t[1]")
obsffi.gs_image_file_init(image, uri)
print("FFI image size " .. tostring(image[0].cx) .. " x " .. tostring(image[0].cy))
print(string.format("First 2 bytes of the buffer: %02x %02x", image[0].texture_data[0], image[0].texture_data[1]))
local str = ffi.string(image[0].texture_data, image[0].cx*image[0].cy*4)
print("Image data as string: " .. hex_dump(str))
obsffi.gs_image_file_free(image)
obslua.obs_leave_graphics()

It produces:
[tips-and-tricks.lua] FFI image size 3 x 2
[tips-and-tricks.lua] First 2 bytes of the buffer: ff 00

[tips-and-tricks.lua] Image data as string: ff 00 00 ff ff ff ff ff 00 00 ff ff ff 00 00 7f ff ff ff 7f 00 00 ff 7f

So finally it works! The trick was to define the data type as gs_image_file_t[1] (no asterisk) and add [0] to all references to the object.

Some remarks:
1605167552694.png
1605170444638.png

  • At least two possibilities to access the bitmap buffer: directly by using it as a byte array with image[0].texture_data[i], or copy the data into a Lua string with ffi.string. These dereferencing rules are documented too (in my opinion according to these rules, we should be able to translate image->texture_data as image.texture_data in Lua but it does not work in my examples):
1605171120876.png
  • The order of the bitmap data is a bit unexpected for me (BGRA)
To conclude: it works but the code is quite long for what it does.. Using FFI is highly dependent on the C code and does not use the obslua bindings at all. For instance, adding a single member in the animated GIF stuff would probably break everything here, while obslua bindings would be recompiled and correctly point to the correct offsets.

The long-term maintainability of such code is a concern.

That's all for today!
 

Mega64

New Member
For the sake of completing the documentation this thread provides, and relating to my previous post here, I can confirm that creating sources is indeed possible in Python, as well as adding them to the OBS GUI. It's just that doing these two very fundamental actions for a script is a bit counter-intuitive, and the API documentation is not very clear about this (to the extent I've studied it so far), so for anyone who was just as lost as me, I will indicate here how to create sources and add them to the OBS GUI so that you can further manipulate them.

@bfxdev , maybe you'd like to edit the messages in this thread which say that sources can't be added with Python, to increase the readability of the thread and not mislead possible new readers.

The example code will be in Python but is perfectly doable in Lua as well.

ADDING A SOURCE

On paper, it's fairly simple; you just have to use the obs_source_create function to create it. And that's right, this alone is enough to create the source as shown by the obs_enum_sources function, which, when iterated upon, also displays the newly added source. The thing is, in order for it to actually be displayed as a source that can be added to a scene, you must correctly define the id field of the parameters. But how?

When you choose to add a source in OBS GUI, this menu is shown(yeah, it's in Spanish, sorry lol but you get the idea by the icons :P):

rec.png


This shows all the types of sources OBS has. The id parameter in obs_create_sources indicates which of these types your new source will be, which means that every type of source in OBS has a distinct id.

For example, let's say you want to create a new multimedia source that plays a .mp3 file or whatever. The code to create this new source would be as follows.

Python:
settings = obs.obs_data_create()
obs.obs_data_set_string(settings, "local_file", song_path)
source = obs.obs_source_create("[B]ffmpeg_source[/B]", file, settings, None)

In the specific case of this source, since we assume it's being played from a local file (song_path), we have to also add to the source's settings an obs_data_t object with a string called "local_file" as key and the song path as value.

As you can see, in the specific case of multimedia files, the id you must enter into the id parameter is called "ffmpeg_source". This right here is the key in order for it to show up in OBS. If you don't put it (or make it up), the source WILL be created, but it won't show up anywhere in the GUI, and hence won't be able to be accessed by the user, which makes manipulating it later basically hell. If you do put the id properly to fit the id of the type of source you want to add, the source will be added and will be shown among the other existing sources. Yay!

Of course, if you now want to add it to a scene, you have to make use of the obs_scene_t and obs_sceneitem_t structures and functions, but that is a whole other can of worms (I can explain it if somebody wants me to though).

The id thing is very poorly explained in the documentation, and in fact, the only way I've found to learn what the id for each type of source is, is creating sources of each type from the GUI, then list them with obs_enum_sources, and THEN get their id with obs_source_get_id and print it. It's an essential part of adding sources, and I haven't seen it explained anywhere else, and it's a very dumb and simple thing that should be obvious. I dunno, perhaps I am stupid and that's why it took me so long to figure this out :P

The only caveat to this method is that the sources don't seem to persist between OBS executions. Sometimes. Sometimes they do persist. I don't know, it's very weird and I haven't found a clear pattern for when this happens, so if anyone can shed some light into this, it would be very appreciated!
 

bfxdev

New Member
Thanks for sharing @Mega64. Indeed it sounds like a possible way to create sources in OBS in Python, applicable as well to Lua. Now I'm not sure that this is the intended use of the obs_source_create function. I wrote myself a script that registers a filter (one of the types of "sources") and did not use this function at all. It shows up in the list of filters, and is persistent across OBS executions.

The documentation is not very explicit about what the function does: Creates a source of the specified type with the specified settings. The “source” context is used for anything related to presenting or modifying video/audio. Use obs_source_release to release it. My understanding is that you use the function to create an instance of an existing source type (no idea how to get the registered id values!), and then you add it to a scene. Now if the "id" is not registered, the function will not fail and will keep an own id field. I can imagine that this id is registered somehow in the list of source types later on, that is why it appears in the list of source types.

I would suggest to start a dedicated thread on the topic if it you want to go further into this method.

I could correct a bit my previous posts if I could find a way to do it (except I missed some button, it does not seem to be possible to edit older posts) because I improperly formulated what I saw in the documentation: to be very exact, it seems that in Python it is not possible to "register a source". I never tried myself Python in OBS and I still wonder why it is supposed not to be possible.

As a reminder, this is what the documentation on scripting in OBS says about sources:

1605469508272.png

Then you have this example code (just a copy of the documentation, no original code):
Lua:
local info = {}
info.id = "my_source_id"
info.type = obslua.OBS_SOURCE_TYPE_INPUT
info.output_flags = obslua.OBS_SOURCE_VIDEO

info.get_name = function()
        return "My Source"
end

info.create = function(settings, source)
        -- typically source data would be stored as a table
        local my_source_data = {}
        [...]
        return my_source_data
end

info.video_render = function(my_source_data, effect)
        [...]
end

info.get_width = function(my_source_data)
        [...]
        -- assuming the source data contains a 'width' key
        return my_source_data.width
end

info.get_height = function(my_source_data)
        [...]
        -- assuming the source data contains a 'height' key
        return my_source_data.height
end

-- register the source
obs_register_source(info)

So basically, the method foreseen to create new source types by script in OBS is to fill an obs_source_info structure with lots of information (including the constant "id") and callback functions, and then to call obs_register_source.

Then, when the user selects a source, or filter to process a source, a copy of this obs_source_info structure is created and a unique set of persistent data is setup for the source instance. Note that the "id" is registered at the time of script loading (i.e. at each OBS startup). If the script is removed from OBS, then the id does not show up in the list, and all related filters or sources are simply removed.

In general, I tried myself in Lua the "source info" method and it works like a charm! Of course, without more documentation, it took me a while to interpret the different arguments in the callback functions, and how to setup properly the global and source-specific parameters.

That's all for today.
 

upgradeQ

Member
Coroutines - the major mechanism for control in Lua
@bfxdev , In this post description of coroutine from standard library is not correct, Lua does not create any threads see docs :
Unlike threads in multithread systems, however, a coroutine only suspends its execution by explicitly calling a yield function.
I will show later how to write concurrent code using coroutines in one thread.

A note about script menu:
There is possibility to run same script in another thread,
you just need to copy&paste script file, rename it, add it to obs and adjust settings.
For example setup different countdown timers, one version will be attached to specific scene, and have that specific scene parameters, while the other will operate on another one with different parameters.
Using coroutines to schedule, pause, delay, streamline execution of scripts in obslua
I’ve adapted scheduler code from this blogpost: How to implement action sequences and cutscenes
See also PIL chapter.
Lua:
local Timer = {}
function Timer:init(o)
  o = o or {}
  setmetatable(o,self)
  self.__index = self
  return o
end
function Timer:update(dt)
  self.current_time = self.current_time + dt
  if self.current_time >= self.delay then
    self.finished = true
  end
end
function Timer:enter()
  self.finished = false
  self.current_time = 0
end
function Timer:exit()
  --print('on exit')
end
function Timer:launch()
  self:enter()
  while not self.finished do
    local dt = coroutine.yield()
    self:update(dt)
  end
  self:exit()
end
function time_sleep(s)
  local action = Timer:init{delay=s}
  action:launch()
end
Usage
time_sleep is coroutine, later it will be continuously resumed in main loop (script_tick).
Lua:
function infinity_scene()
  while true do
    n = math.random(1,3)
    print('random sleep [' .. n .. '] seconds')
    time_sleep(n)
  end
end
local coro = coroutine.create(infinity_scene)
function script_tick(dt)
  coroutine.resume(coro,dt)
end
Cancelling and locking
Consider this function, it should print hello, then after 3 seconds world.
Lua:
function delayed_print_logic()
  print('hello')
  time_sleep(3)
  print('world')
end
In this example I'll be using hotkeys.
So if hotkey is pressed, logic will trigger and until it's done there is no way to launch it again. But if we decided to not to wait we can cancel it safely.
Hotkey callback, notice there is no booleans
Lua:
function htk_1_cb(pressed)
  if pressed then
    delayed_print()
  end
end
First let's create a coroutine:
local coro = coroutine.create(function() coroutine.yield() end)

Then in main loop (script_tick) it will be resumed right after OBS started, coroutine.status will be equal to dead and that will not propagate any error,script will continue to execute.
dealyed_print and script_tick
Lua:
function delayed_print()
  if coroutine.status(coro) == 'dead' then
    coro = coroutine.create(delayed_print_logic)
  end
end
function script_tick(dt)
  coroutine.resume(coro,dt)
end
This ensures that if and only if coroutine status is dead ,it is possible to launch that
code and it will be not interruptible by hotkey trigger . However it's also possible to
bind cancel callback to another hotkey
Lua:
function cancel_delayed_print()
  print('abort printing')
  coro = nil
  coro = coroutine.create(function() coroutine.yield() end)
end
Notice also that coro is global variable since here we are resetting it to initial state.
Easing
You might know that if you have used @Exeldro plugins. Here is Lua adaptation
Lua:
function easing_scene_item_movement(begin,end_val,duration)
  local begin = begin or 500
  local end_val = end_val or 0
  local duration = duration or 0.6
  local current = 0
  local interval = 0.02
  local change = end_val - begin
  repeat
    --local absolute_progress = inOutExpo(current,begin,change,duration)
    local absolute_progress = inOutQuart(current,begin,change,duration)
    move_scene_item(scene_item,absolute_progress)
    time_sleep(interval)
    current = current + interval
  until current >= duration
end
function easing_scene_loop()
  while true do
    easing_scene_item_movement(900,0,0.16)
    easing_scene_item_movement(0,300)
    time_sleep(0.1)
    easing_scene_item_movement(300,600,0.8)
    time_sleep(0.3)
    easing_scene_item_movement(600,900,0.23)
  end
end

This will move some source on scene in a loop smoothly.
Everything above runs concurrently, I’ve created gist with all those examples.
For easing_scene_item_movement create small rectangle color source with name c2c2,
then check Script log console output, default hotkeys are 1 and 2.
 

upgradeQ

Member
There is mistake in my previous post - obs.OBS_KEY_NONE does not exists in obslua (also in obspython) namespace, I guess it's value is 0 might be wrong though.
Turns out there is a way to TriggerHotkeyByName, using LuaJIT ffi:
Necessary C declarations
C:
typedef struct obs_hotkey obs_hotkey_t;
typedef size_t obs_hotkey_id;

const char *obs_hotkey_get_name(const obs_hotkey_t *key);
typedef bool (*obs_hotkey_enum_func)(void *data, obs_hotkey_id id, obs_hotkey_t *key);
void obs_enum_hotkeys(obs_hotkey_enum_func func, void *data);
Actual function
Lua:
function trigger()
  local target = 'OBSBasic.StartVirtualCam'
  local htk_id
  function callback_htk(data,id,key)
    local name = obsffi.obs_hotkey_get_name(key)
    if ffi.string(name) == target then -- (1)
      htk_id = tonumber(id) -- (2)
      print('found target hotkey id: ' .. htk_id)
      return false
    else
      return true
    end
  end
  local cb = ffi.cast("obs_hotkey_enum_func",callback_htk) -- (3)
  obsffi.obs_enum_hotkeys(cb,data)
  if htk_id then
    obs.obs_hotkey_trigger_routed_callback(htk_id,false)
    obs.obs_hotkey_trigger_routed_callback(htk_id,true) -- (4)
    obs.obs_hotkey_trigger_routed_callback(htk_id,false)
  end
end
  1. Converting cdata object to Lua string.
  2. tonumber is a special function, in LuaJIT it convert any number from C to Lua.
  3. Converting Lua function into C callback
  4. false/true/false because it does not works otherwise.
Six years ago, there was this commit - source_volume_level signal has been deleted.
Today there is no straightforward way to access audio level in db of any source. But attaching obs_volmeter_add_callback works,here is showcase for utilising audio data in Advaced Scene Swithcher , my ffi implementation is based on that.
Necessary C declarations
C:
typedef struct obs_source obs_source_t;
obs_source_t *obs_get_source_by_name(const char *name);
void obs_source_release(obs_source_t *source);

enum obs_fader_type {
    OBS_FADER_CUBIC,
    OBS_FADER_IEC,
    OBS_FADER_LOG
};

typedef struct obs_volmeter obs_volmeter_t;

bool obs_volmeter_attach_source(obs_volmeter_t *volmeter,
                       obs_source_t *source);

int MAX_AUDIO_CHANNELS;

obs_volmeter_t *obs_volmeter_create(enum obs_fader_type type);

typedef void (*obs_volmeter_updated_t)(
    void *param, const float magnitude[MAX_AUDIO_CHANNELS],
    const float peak[MAX_AUDIO_CHANNELS],
    const float input_peak[MAX_AUDIO_CHANNELS]);

void obs_volmeter_add_callback(obs_volmeter_t *volmeter,
                      obs_volmeter_updated_t callback,
                      void *param);
Actual function
Lua:
local lvl -- (1)
function callback_meter(data,mag,peak,input)
  lvl = 'Volume lvl is :' .. tostring(tonumber(peak[0]))
end
jit.off(callback_meter) -- (2)

function volume_lvl() -- (3)
  if lvl == nil then print('error lvl is nil') else print(lvl) end
end

function launch_callback() -- (4)
  local source = obsffi.obs_get_source_by_name("audio")
  local volmeter = obsffi.obs_volmeter_create(obsffi.OBS_FADER_LOG)
  -- https://github.com/WarmUpTill/SceneSwitcher/blob/214821b69f5ade803a4919dc9386f6351583faca/src/switch-audio.cpp#L194-L207
  local cb = ffi.cast("obs_volmeter_updated_t",callback_meter)
  obsffi.obs_volmeter_add_callback(volmeter,cb,data)
  obsffi.obs_volmeter_attach_source(volmeter,source)

  obsffi.obs_source_release(source)
end
  1. Setting level of audio in outer scope
  2. Turning of JIT features, because it might crash sometimes
  3. On demand access to audio volume level
  4. It is really unstable, so use this function with caution, e.g multiple instance might cause crash. Also I've tried to print volume level of audio source to Script log as audio data comes in eventually it's freezes with crash report
Hotkey enumeration via obs_enum_hotkeys gist , audio volume level gist
 

vaggoscc

New Member
Thank you very much for your Idea to help people like me to Start with Lua!

Can you Help me, how to Run the above sample code in Obs ?
I save as new file with Name Open_Dir.lua and after That i add it to Scripts in obs , but how run it to see the result of Files ?


Open_Dir.jpg
 

equilibrier

New Member
Hi, @vaggoscc , lua scripts are executed by default, it all deppends on how you configure the scripting defaults in lua, I mean, what function you give OBS to execute as "main" lua script function.
I'm a newbie in lua, but on a script I adapted, you have:

--- Loaded on startup
function script_load(settings)
print("Loading Next Scene script")
obs.timer_add(checkTransition, 500)
end


there are other default functions like
function script_save(settings)
function script_properties()
function script_update(settings)
function script_defaults(settings)
function script_description()


but these doesn't concern you right now, in script_load function I'm using an obs lib timer to trigger the execution of a custom function (this can be yours), in my case, named "checkTransition", at every 500 ms (so, a repeated execution). I think if you just want to process a single function, once, you can write something like this:

function myfunc(myargs)
--[[...todo...]]--
end
function script_load(settings)
print("Loading Next Scene script")
myfunc(dummy)

end

I hope this helps ! Cheers and a new year with good thoughts ! Health to all, thanks for this thread !

PS: Maybe this tutorial would also help: https://dev.to/hectorleiva/start-to-write-plugins-for-obs-with-lua-1172
I'm finding it useful for beginners like us.
 
Top