I would like to get UDP data from a port by using OBS-Lua. Is this possible and if so any examples on how to do this for a very novice programmer please! I have some Lua code which reads local files and updates OBS Sources dependent upon the file contents. These files are generated by an external Windows program from network UDP data. The file generation is outside my control so would like to be able to work with the UDP data directly as it has further information than that available in the text file. I know the port the data is sent to (58091) so have I think all the information to get at the data if I can get OBS-Lua to access it. I have tried using the Lua examples on the web but they all require to load a Socket module thus 'local socket = require("socket")' which fails as I'm guessing the module is not available to OBS-Lua. Thanks for any help or guidance.
Using UDP ports in OBS Lua
- Thread starter andy5211d
- Start date
John Hartman
Member
This sounds like a big lift for a "very novice programmer," so Google's your buddy.
An example of socket programming in OBS Lua may be found at
https://obsproject.com/forum/threads/control-visca-over-ip-based-cameras.136462/#post-502272
This includes the ljsocket library, but only uses TCP to send commands to PTZ cameras, where replies are expected almost immediately. So you would need to look through ljsocket to see what else is available.
Lua programs in OBS run mostly in the main UI thread. So if you code a simple "wait for input from UDP," the UI will lock up until the packet comes in or a timeout occurs. So to do the whole thing in Lua you would need to use non-blocking sockets. Google for "non-blocking socket in lua" gets a bunch of hits. But remember that ljsocket.lua may not have all the capabilities of some other Lua socket libraries.
Another option would be to put your UDP code in a separate program written in the language of your choice, and to have the program use Websocket to control OBS. Different set of complexities, but might give you more options (and more code examples) for the UDP side.
An example of socket programming in OBS Lua may be found at
https://obsproject.com/forum/threads/control-visca-over-ip-based-cameras.136462/#post-502272
This includes the ljsocket library, but only uses TCP to send commands to PTZ cameras, where replies are expected almost immediately. So you would need to look through ljsocket to see what else is available.
Lua programs in OBS run mostly in the main UI thread. So if you code a simple "wait for input from UDP," the UI will lock up until the packet comes in or a timeout occurs. So to do the whole thing in Lua you would need to use non-blocking sockets. Google for "non-blocking socket in lua" gets a bunch of hits. But remember that ljsocket.lua may not have all the capabilities of some other Lua socket libraries.
Another option would be to put your UDP code in a separate program written in the language of your choice, and to have the program use Websocket to control OBS. Different set of complexities, but might give you more options (and more code examples) for the UDP side.
Thank you very much for you reply. I have tried Googling but they all came up with video streaming so thought they were not relevant. I had thought of using a different program to handle the UDP comms but not that this could then talk to OBS via Websockets. Mostly I need to write text into Sources so not sure how simple that is via Websockets. But will now investigate further.
In my Googling I noticed that Lua UDP receive is usually blocking and found a few examples on how to deal with this but so far that is a bit beyond my understanding of Lua. Also found some UDP examples using VB from the thought of using an separate program but that code made no sense to me! At least I sort of understand Lua code. Thanks again.
Just for reference the OBS script I'm working on is there: https://github.com/andy5211d/DR2TVOverlay A bit of cobble together of ideas from other scripts, but it works. Many thanks to the OBS team.
In my Googling I noticed that Lua UDP receive is usually blocking and found a few examples on how to deal with this but so far that is a bit beyond my understanding of Lua. Also found some UDP examples using VB from the thought of using an separate program but that code made no sense to me! At least I sort of understand Lua code. Thanks again.
Just for reference the OBS script I'm working on is there: https://github.com/andy5211d/DR2TVOverlay A bit of cobble together of ideas from other scripts, but it works. Many thanks to the OBS team.
John Hartman
Member
I got bored/curious and put together a simple demo (attached) that listens on a specified UDP port (11111 as provided).
Once per second, a callback looks for input and prints it (assumes ASCII data). I used a 1-second poll to minimize debug log output. Actual rate would depend on your needs. If multiple datagrams are received between polls, each will be received separately.
(File has .txt extension, since the forum won't accept .lua)
Once per second, a callback looks for input and prints it (assumes ASCII data). I used a 1-second poll to minimize debug log output. Actual rate would depend on your needs. If multiple datagrams are received between polls, each will be received separately.
(File has .txt extension, since the forum won't accept .lua)
Thank you very much. Will have a closer look and test when home later. I guess being cheeky is there a chance you could provide an example for looking at four ports simultaneously. Need to grab what is on port 58091, 58092, 58093 and 58094! But 58091 is the most important for this version of DR2TVOverlay so not really a problem if multiple ports are too complicated for a simple example.
Did try the example in the ljsocket repo here https://github.com/CapsAdmin/luajitsocket/blob/master/examples/udp_client_server.lua but for some reason it took exception to this line
not liking the 'address' component (is Nil not an Int). Not a problem as I did not spend more than a few mins at it and did not really try to figure it out the problem.
If its ok with you I'd love some idea how the timers work in OB. I need several, but as best I can work out they all seem to have the same name and thus callback name. Is there some resource that's gives a beginners description of how to use them? All the ones I've found seem to assume that the user understand Callback and its implications. Is it as simple as:
with the callback function called:
and the next being
and the associated callback function being:
Can timers be nested and if so how does the callback work then? Sorry, can't get my head around this!
Anyway, thanks again for your help.
Did try the example in the ljsocket repo here https://github.com/CapsAdmin/luajitsocket/blob/master/examples/udp_client_server.lua but for some reason it took exception to this line
Code:
assert(client:send_to(address, "hello from client " .. os.clock()))If its ok with you I'd love some idea how the timers work in OB. I need several, but as best I can work out they all seem to have the same name and thus callback name. Is there some resource that's gives a beginners description of how to use them? All the ones I've found seem to assume that the user understand Callback and its implications. Is it as simple as:
Code:
obs.timer_add(timer_callback, 1000)
Code:
function timer_callback()
Code:
obs.timmer_add(timer_anothercallback)
Code:
function timer_anothercallback()Can timers be nested and if so how does the callback work then? Sorry, can't get my head around this!
Anyway, thanks again for your help.
John Hartman
Member
You CAN have multiple timers, each with a unique callback function. But a single timer would be sufficient to handle input from multiple UDP ports: Call socket.create/set_option/set_blocking/bind for each port you want to monitor. Then in a timer callback, do the repeat/receive_from/until for each created socket.
Since the script calls set_blocking(false), receive_from always returns without waiting: it returns a datagram if one has been received, else "timeout" (even though no "time" has elapsed), or an error. Since multiple datagrams may have been received since the the last time the callback was executed, I wrapped receive_from to deal with any queued data.
If you were doing this in an environment that supported multiple threads, it would be more efficient to use the "select" function to block a receive thread until at least one socket has input, then process that socket, rather than polling all the sockets each time. But Lua in OBS doesn't allow multiple threads.
You could handle each UDP port in its own timer callback, but single timer will have less overhead if the message rate on all the ports is similar.
This method should be OK if the message rate is low - a few per second total. Above that, you may start to see frame loss. If you see that, you may need to move to websocket.
Since the script calls set_blocking(false), receive_from always returns without waiting: it returns a datagram if one has been received, else "timeout" (even though no "time" has elapsed), or an error. Since multiple datagrams may have been received since the the last time the callback was executed, I wrapped receive_from to deal with any queued data.
If you were doing this in an environment that supported multiple threads, it would be more efficient to use the "select" function to block a receive thread until at least one socket has input, then process that socket, rather than polling all the sockets each time. But Lua in OBS doesn't allow multiple threads.
You could handle each UDP port in its own timer callback, but single timer will have less overhead if the message rate on all the ports is similar.
This method should be OK if the message rate is low - a few per second total. Above that, you may start to see frame loss. If you see that, you may need to move to websocket.
John Hartman
Member
From an admin command prompt, "netstat -ab" will show which programs are using which TCP and UDP ports. So with your script running, it should show OBS.exe waiting on UDP port 58091. Annoyingly, netstat dumps the program on a line AFTER the address:port, so you can't just grep the output, and there is a lot to wade through. Redirect to a text file and search for it that way.
Last edited:
So running the DR2Video program which I know listens to 58091 and at the same time running OBS does not introduce an exception which is odd as they should be listening to the same port and I know that if you run two instances of DR2Video the second instance gets upset! The relevant parts of netstat output are thus:
All I know from this is that obs is using the local host IP address and DR2Video is not. Do I need to define an IP of 0.0.0.0 for the client somewhere in the script?
Code:
UDP 0.0.0.0:58091 *:*
[DR2Video.exe]
UDP 127.0.0.1:58091 *:*
[obs64.exe]
UDP [::]:58091 *:*
[DR2Video.exe]All I know from this is that obs is using the local host IP address and DR2Video is not. Do I need to define an IP of 0.0.0.0 for the client somewhere in the script?
If I try to define the address as 0.0.0.0 in the following part of the script
then it return an error of
'... UDP-test.lua:40: An attempt was made to access a socket in a way forbidden by its access permissions'.
Code:
-- Bind our_port on all local interfaces
assert(our_server:bind('0.0.0.0', our_port))'... UDP-test.lua:40: An attempt was made to access a socket in a way forbidden by its access permissions'.
However if I enter the IP address of the machine sending the packets to 58091 or the IP address of the machine running OBS and this script, in the above line then your code works :-) Fantastic, thanks for your help with this.
Interesting to see that the following code
introduced in the timer_callback loop after the printing of the received data does produce something but not what I'd expect. I'm guessing that the returned data is not directly printable as it looks sort of correct but is wrong. This is what I received:
whereas I would have expected: 192.168.1.166 and 58091.
And finally the script does get (and thus print) 10 copies of the data received by the port, not obvious to me why, yet! I'm guessing that I need to 'clear' the receive buffer of something in the timer call_back loop.
Interesting to see that the following code
Code:
print('Get IP: ' .. status:get_ip() .. ', Get port: ' .. status:get_port())
Code:
Get IP: 200.190.192.168, Get port: 48840And finally the script does get (and thus print) 10 copies of the data received by the port, not obvious to me why, yet! I'm guessing that I need to 'clear' the receive buffer of something in the timer call_back loop.
John Hartman
Member
Are you sure you are getting 10 copies, versus 10 distinct messages that happen to contain the same data? The first cut of my demo script only did a single receive_from on each callback. But then if my sender sent more than one message in a timer interval, the extra messages got queued up on your bound socket, and the Lua receive_froms got further behind on each callback. So I added the loop to deal with any backlog.
If your packet source sends redundant messages, you might process only the last one received in the loop, or compare the contents for change, etc.
If your packet source sends redundant messages, you might process only the last one received in the loop, or compare the contents for change, etc.
John Hartman
Member
I haven't written any IP-related code since I retired four years ago, so I have forgotten a lot of the fine points. The address and port stuff is at least partially due to byte order: on the wire, IP sends addresses and ports most significant byte first. On a PC, integers are stored least significant byte first. The sockets API (which ljsockets more or less duplicates) dates back to the dawn of IP, when computers were slow, so the API tries to minimize byte order conversion. So in the C API there are/were platform-specific functions "ntohl()" (network to host long), "htonl" (host to network long) etc. that you had to use to turn integers into IP addresses and ports, and vice versa.
I would have hoped that a Lua sockets library would have hidden all that, but looking at ljsocket.lua, you can see that it just wraps the C API, and I don't know the details of ffi to see exactly what is happening.
If I call get_port() on the address returned along with data by receive_from, the value I get is the byte-flipped version of the UDP source port from which the message came (not the UDP destination port that I am bound to and listening on). So if the source port is 4097 (0x1001), get_port() returns 272 (0x0110). get_ip() returns a string that is mangled even more than byte order - I haven't figured it out yet, but the first two bytes seem to be the port rather than part of the address.
BUT: I can USE the address as returned by receive_from as the address for send_to(), and my datagram will be sent there.
I also note that if I do such a send, and nobody is listening on that address and port, my socket is closed and subsequent receive_from calls fail. There is probably a flag or option to prevent that (something like the "reuseaddr" in the example), but I don't recall it and Google isn't helping. Or you could create a second socket just to send the datagram, and then close it.
I would have hoped that a Lua sockets library would have hidden all that, but looking at ljsocket.lua, you can see that it just wraps the C API, and I don't know the details of ffi to see exactly what is happening.
If I call get_port() on the address returned along with data by receive_from, the value I get is the byte-flipped version of the UDP source port from which the message came (not the UDP destination port that I am bound to and listening on). So if the source port is 4097 (0x1001), get_port() returns 272 (0x0110). get_ip() returns a string that is mangled even more than byte order - I haven't figured it out yet, but the first two bytes seem to be the port rather than part of the address.
BUT: I can USE the address as returned by receive_from as the address for send_to(), and my datagram will be sent there.
I also note that if I do such a send, and nobody is listening on that address and port, my socket is closed and subsequent receive_from calls fail. There is probably a flag or option to prevent that (something like the "reuseaddr" in the example), but I don't recall it and Google isn't helping. Or you could create a second socket just to send the datagram, and then close it.
Thanks for the above re address and port bytes etc. I was thinking along similar lines but have not tried to work out what the 'conversion' is yet as not sure of the sender details. Will try 'netstat -ab' at some point to try to discover the senders details.
I 'think' the remote application is sending 10 messages because if I trigger another function on that machine I get 5! Although I'm not sure on what is being sent it does this consistently switching between functions on the remote machine. I'll check with the author.
.
I 'think' the remote application is sending 10 messages because if I trigger another function on that machine I get 5! Although I'm not sure on what is being sent it does this consistently switching between functions on the remote machine. I'll check with the author.
.
As best I can tell at this point I could use the returned IP address to grab a file off the remote machine as one of the messages I get (the one with five repeats) gives me a file name and remote machine name. I think by FTP but that is not something I've attempted or know anything about, yet!
Just for clarity and to help others I have modified your original code by adding this line as a config item:
and then changing your 'assert our_server:bind' line to:
to set the IP address.
Alternatively the IP address for your machine can be inserted in the 'assert our_server:bind' line. I'm not finding a Lua function to get the IP address on the machine the script is running on but actual address data format works, eg '192.168.1.100' in place of Address1. I'm sure there must be a way to get the Ip address in Lua but guess looking at the wrong pages found by google.
I'll post my working test script when I have confirmed a few things and tided up the inline comments.
Code:
Address1 = socket.find_first_address("*", our_port1)
Code:
assert(our_server1:bind(Address1, our_port1))Alternatively the IP address for your machine can be inserted in the 'assert our_server:bind' line. I'm not finding a Lua function to get the IP address on the machine the script is running on but actual address data format works, eg '192.168.1.100' in place of Address1. I'm sure there must be a way to get the Ip address in Lua but guess looking at the wrong pages found by google.
I'll post my working test script when I have confirmed a few things and tided up the inline comments.
John Hartman
Member
Do you need to know your PC's IP address? The original demo bound to "*", which includes every interface on the computer including localhost. You need to know the address If your computer has multiple interfaces and you want to EXCLUDE one or more, but that is less common.
I hate to crank up the firehose even further, but it is well worth your time to download and learn at least the basics of Wireshark, so you can see what is being sent and received. It makes network programming much more understandable. Initially, just a few filters such as "udp", or "udp.port == 58091" will let you focus on the stuff you are interested in and ignore everything else.
Regarding FTP: is the file something that will change during your OBS session, so that you will need to refresh it during the session? If you need to read files during a session I would be inclined to move at least the FTP part out of your script an into a separate executable for several reasons:
I hate to crank up the firehose even further, but it is well worth your time to download and learn at least the basics of Wireshark, so you can see what is being sent and received. It makes network programming much more understandable. Initially, just a few filters such as "udp", or "udp.port == 58091" will let you focus on the stuff you are interested in and ignore everything else.
Regarding FTP: is the file something that will change during your OBS session, so that you will need to refresh it during the session? If you need to read files during a session I would be inclined to move at least the FTP part out of your script an into a separate executable for several reasons:
- Complexity. There are lots of implementation and libraries available in C and other languages that you could, rather than trying to port to Lua
- Danger of frame dropping. The simple polled scheme we have been looking at should be OK for a few messages per second, but at some level of traffic your script is going to eat enough time to impact OBS UI, and maybe drop frames.
- Classic FTP is insecure, passing unencrypted data over the network. That may be fine if the server is on your local network, but to be avoided in most cases. But be aware that some references to "FTP" may actually mean one of the secure/encrypted versions (which you definitely don't want to implement in Lua)
Your original demo did not work for me, as I said above in #7 and then what I did to make it work in #12. But no I don't need to know my or the remote machines address just as long as I can get the data from the four UDP ports. The script is now working for one port as you know, not tried all four at present. And unlikely to see anything on the other three ports until we run an event (Thursday to Sunday this week and live stream on YouTube) when all the machines are running and interacting with each other. For this competition I'll not be using the UDP communications, but will have a 'development' laptop with OBS and the UDP software running to see what happens and logs.
Thanks for reminding me of Wireshark. I did have it on one of my laptops but that was years ago. I will try using it again, thanks.
I think the FTP file data or the contents of that file change fairly regularly. Don't really know, yet. As I can see the complexity of all this growing exponentially I'm guessing that an external program interfacing with OBS via Websocket will be the way to go. Guess I have a lot more learning to do particularly on how to use Websocket. Still I am retired now so have a bit more time; when I'm not told to do the gardening...
Again, thanks very much for you help. I would not have been able to fathom this out by myself, even with a compliant mar google!
Thanks for reminding me of Wireshark. I did have it on one of my laptops but that was years ago. I will try using it again, thanks.
I think the FTP file data or the contents of that file change fairly regularly. Don't really know, yet. As I can see the complexity of all this growing exponentially I'm guessing that an external program interfacing with OBS via Websocket will be the way to go. Guess I have a lot more learning to do particularly on how to use Websocket. Still I am retired now so have a bit more time; when I'm not told to do the gardening...
Again, thanks very much for you help. I would not have been able to fathom this out by myself, even with a compliant mar google!