Plugin Idea. Looking for a stream watchdog that will e-mail me if a stream goes down.

[John Zapf]

Plugin Idea. Looking for a stream watchdog that will e-mail me if a stream goes down.

Not sure if there is anything like that. I have my own SMTP server I just need a plug in that we'll watch my live streams 24/7 365 and shoot me an e-mail if one goes down. I believe they call something like that a watchdog?

 

[AaronD]

Just get a command-line utility that can send an e-mail - that's all it has to do - and call it from here:

Advanced Scene Switcher

This plugin will allow you to automate various tasks using "Macros". Macros consist of a list of conditions under which a list of actions will be performed. Examples and guides can be found in the wiki. Feel free to contribute! If you run...

obsproject.com

If you ask in Adv. SS's discussion forum, you might even get an E-mail action, just for you, that eventually becomes part of the official plugin. Open Sound Control got there because of *my* request.

Advanced Scene Switcher

Warmuptill submitted a new resource: Automatic Scene Switching - Switches to specified scenes depending on which window is in focus A remake of the "Simple Scene Switcher"(https://obsproject.com/forum/resources/simple-scene-switcher.29/) for obs-studio. Installation 1. Extract the folder. 2...

obsproject.com

 

[Lawrence_SoCal]

An important technical consideration is if you desire/require that monitoring to be from a different system than the one streaming.
Aaron's suggestion is spot on, and there are some features in AdvSS due to my specific requests, as well.
However, without a specific email feature, you could still have (i think) AdvSS monitor for connection status (as that is native to OBS Studio) and if not connected, execute/trigger/call a script or program that could be your email message. Obviously you'll need to consider and code some conditions for those times when you intentionally stop stream and don't want to get flooded with emails

However, there can be times when OBS Studio still has a connection, but stream stops for some reason. OBS Studio has no way to know a CDN (like YouTube or similar) has had an issue (glitch in your stream source, or glitch on CDN side) and stream has stopped processing on their end, or audio stopped, or whatever. Another scenario would be if your OBS Studio computer/LAN loses Internet connectivity... no way to send email..
Monitoring from elsewhere addresses these scenarios. But now you have to decide where to run that monitoring from, how that will access your SMTP server, etc. And how reliable is the ISP connection at this alternate location? And do you want to get really sophisticated and monitor if issues with specific ISPs? These questions are familiar with anyone who does IT monitoring of public facing websites.
Like so many things, simple up/down monitoring is usually pretty easy (therefore cheap). But once in place, expectations sometimes go up, and then things can get complicated (expensive) where budget constraints tend to be the limiting factor (vs technology).

I'd certainly start at monitoring/notification from the OBS Studio computer until such time that expectation and cost/benefit justify getting more sophisticated.

 

[John Zapf]

An important technical consideration is if you desire/require that monitoring to be from a different system than the one streaming.
Aaron's suggestion is spot on, and there are some features in AdvSS due to my specific requests, as well.
However, without a specific email feature, you could still have (i think) AdvSS monitor for connection status (as that is native to OBS Studio) and if not connected, execute/trigger/call a script or program that could be your email message. Obviously you'll need to consider and code some conditions for those times when you intentionally stop stream and don't want to get flooded with emails

However, there can be times when OBS Studio still has a connection, but stream stops for some reason. OBS Studio has no way to know a CDN (like YouTube or similar) has had an issue (glitch in your stream source, or glitch on CDN side) and stream has stopped processing on their end, or audio stopped, or whatever. Another scenario would be if your OBS Studio computer/LAN loses Internet connectivity... no way to send email..
Monitoring from elsewhere addresses these scenarios. But now you have to decide where to run that monitoring from, how that will access your SMTP server, etc. And how reliable is the ISP connection at this alternate location? And do you want to get really sophisticated and monitor if issues with specific ISPs? These questions are familiar with anyone who does IT monitoring of public facing websites.
Like so many things, simple up/down monitoring is usually pretty easy (therefore cheap). But once in place, expectations sometimes go up, and then things can get complicated (expensive) where budget constraints tend to be the limiting factor (vs technology).

I'd certainly start at monitoring/notification from the OBS Studio computer until such time that expectation and cost/benefit justify getting more sophisticated.

 

[John Zapf]

Well the long and short of this is the OBS streams are all running on the same box that's dedicated just to OBS. And I don't care if it notifies me when I stop the streams that's very rare. Like I said they stream 24/7 365. As far as SMTP server that's local right here in house. So there won't be an issue there. I just need to figure out a way to have it tell me there is an issue with a strem on OBS by sending me an e-mail, a stream has had an issue or has stopped or whatever. Like the other day the stream was still running but it wasn't streaming the light was green but the bites going out were 0, so I don't know what happened there, there was no crash log. But I didn't know about it until somebody said something, and that's not cool I think it was down for several hours.

 

[AaronD]

the stream was still running but it wasn't streaming the light was green but the bites going out were 0

You probably want this then:

I think streaming at 0 bits/sec is still "streaming", as far as anything here is concerned.

 

[Lawrence_SoCal]

And I don't care if it notifies me when I stop the streams that's very rare.

You need to care. Think about it... is stream down? yes = send alert... home many times a second?
A carelessly created alert/monitor could easily become the equivalent of a denial of service event (self-inflicted in this case). and crash/overload SMTP server, potentially LAN link impact (preventing remote access), etc. And if that SMTP relay going to Internet located SMTP relay/mailbox, you risk being flagged as a SPAMMER (by your ISP, SMTP relays, etc)

You could easily do the figurative 'shoot yourself in the foot' if you aren't careful.

Like the other day the stream was still running but it wasn't streaming the light was green but the bites going out were 0, so I don't know what happened there, there was no crash log. But I didn't know about it until somebody said something, and that's not cool I think it was down for several hours.

I can easily envision a scenario (reported in these forums in recent years) where OBS Studio is sending a normal stream (not 0 bitrate) but there is STILL an issue with client viewability of the stream. I can't imagine anything in OBS Studio itself that will be able to report on such a status. Largely because, I'm not aware of the free service CDNs self-monitoring and reporting back to end-users or having an API to check such status. A lower then expected bitrate, for an extended period of time is easy to check for from the sending unit as Aaron listed above (but there are other failure scenarios that won't be).

Regardless, you are using free software not designed for 24/7 operation (that I'm aware of), on Windows desktop OS (not designed for 24/7), and a free service... so be thoughtful of reasonable expectations. There are plenty of folks using OBS in this manner, but it strikes me as wrong tool for the job, and one that at least for now, and I'd suspect for a long time if not forever, comes with serious caveats for 24/7 usage.. The tool is designed for a user-interactive environment. And Windows desktop OS, has benefitted consistently by regular logging and reboots (sometimes due to OS itself, but usually driver and/or application software) since the beginning.

And to clarify expectations, It appears your expectation of monitoring include actual stream output from the CDN.
And you want to know, presumably even when you aren't home, meaning your monitoring and notification system has to work even if your ISP link goes down. right?
Do you have multiple ISP connections on divergent physical paths? For residential, most wires follow same path (poles, underground trench, etc), so realistic divergent-path connectivity will include something wireless, with care taken to avoid common incumbent local telco switching station (no help if cellular carrier has fiber connection to same common station as coax, DSL, and/or fiber wired termination points).

 

[John Zapf]

You need to care. Think about it... is stream down? yes = send alert... home many times a second?
A carelessly created alert/monitor could easily become the equivalent of a denial of service event (self-inflicted in this case). and crash/overload SMTP server, potentially LAN link impact (preventing remote access), etc. And if that SMTP relay going to Internet located SMTP relay/mailbox, you risk being flagged as a SPAMMER (by your ISP, SMTP relays, etc)

You could easily do the figurative 'shoot yourself in the foot' if you aren't careful.


I can easily envision a scenario (reported in these forums in recent years) where OBS Studio is sending a normal stream (not 0 bitrate) but there is STILL an issue with client viewability of the stream. I can't imagine anything in OBS Studio itself that will be able to report on such a status. Largely because, I'm not aware of the free service CDNs self-monitoring and reporting back to end-users or having an API to check such status. A lower then expected bitrate, for an extended period of time is easy to check for from the sending unit as Aaron listed above (but there are other failure scenarios that won't be).

Regardless, you are using free software not designed for 24/7 operation (that I'm aware of), on Windows desktop OS (not designed for 24/7), and a free service... so be thoughtful of reasonable expectations. There are plenty of folks using OBS in this manner, but it strikes me as wrong tool for the job, and one that at least for now, and I'd suspect for a long time if not forever, comes with serious caveats for 24/7 usage.. The tool is designed for a user-interactive environment. And Windows desktop OS, has benefitted consistently by regular logging and reboots (sometimes due to OS itself, but usually driver and/or application software) since the beginning.

And to clarify expectations, It appears your expectation of monitoring include actual stream output from the CDN.
And you want to know, presumably even when you aren't home, meaning your monitoring and notification system has to work even if your ISP link goes down. right?
Do you have multiple ISP connections on divergent physical paths? For residential, most wires follow same path (poles, underground trench, etc), so realistic divergent-path connectivity will include something wireless, with care taken to avoid common incumbent local telco switching station (no help if cellular carrier has fiber connection to same common station as coax, DSL, and/or fiber wired termination points).

ok, thanks, power and ISP has only gone down a couple times in 22 years so that is usualy not an issue. And as far as the 24/7 365 streaming I have other ways of doing that directly from the camera to YouTube and never goes down, But the quality is not as good and I can't inject info data whether all kinds of things like I can on OSB so that's why I started using it, I have been doing this for a long time, I use great cameras, keep track of my streams and compression so that's as good as I can get it. You can see the live streams here. https://www.youtube.com/@artisticimages

During nesting season in the spring we have up to 1800 people watching at one time during the spring.

 

[John Zapf]

And also I could run this in Linux if you think that would be better period but the testing I did apples to apples on the streams I'm doing windows and Linux the video card performance was better in windows than it was Linux.

 

[Lawrence_SoCal]

I was curious about the use case.
I get why using OBS for dynamic overlays that aren't practical to include in most cameras that can stream directly

in your scenario, I'd be inclined to use AdvSS as Aaron mentioned, and see if that is good enough. And I'd look into automating regular OS reboots, with auto-starting OBS Studio... but that is only as I've seen too many OS issues over the decades. At a minimum, I'd disable auto OS updates, and do such updates manually (presumably during dark hours). And I'd really trim the OS down in terms of background running processes (primarily preventing auto-start of many Services) that aren't needed in your use case.

However, I'd be inclined to think something else would be a nicer, lower footprint, setup. But, I'm not a command-line/scripting ninja, so I totally get the GUI ease of OBS Studio. I've seen prior comments about using FFmpeg (i think it was) directly in your type of scenario, but that is completely outside my area of expertise (so I'll stop here). I'd think with the right assistance and right GPU, you'd be able to get same video performance in Lunix and Windows.... unless you have a Ryzen APU/GPU, and trying to use its awful H.264 (older) encoding SDK [not sure if more recent AMD SDK update really helped matters sufficiently. Historically, AMD software has been 'weak' (at best) and intentionally under-invested effort in optimizing/improving H.264 encoding. I love the competition AMD is giving intel in the CPU space... but we run computers, not stand-alone CPUs, so system software, drivers, etc need to be competitive as well... /rant off]. *IF* your GPU was AMD, and that was only reason to avoid Linux, I'd seriously consider an older NVENC capable GPU (for lowest cost), otherwise if YouTube today, maybe consider an Intel ARC card, and trying AV1 encoding? or at least, something to consider when its time for next compute upgrade

 

[AaronD]

And also I could run this in Linux if you think that would be better period but the testing I did apples to apples on the streams I'm doing windows and Linux the video card performance was better in windows than it was Linux.

It's more of an advanced thing as I understand it, but at least *some* Linux distributions (maybe all? I really don't know) have a way to apply updates on a live, running system, *including the kernel!* Apps keep running and communicating to the outside world throughout the entire process, uninterrupted. So you really could update the entire OS mid-stream, and no one would notice.

But like I said, I don't know much more than that. I've never had a reason to look into it myself. Eventually though, you'd have to update the apps too, in which case they'd probably have to restart. Or reboot a memory-leaking app just because of that. But if everything works, you *might* be able to disguise an OBS-only reboot as just a hiccup in the same continuing stream. Maybe?

 

[John Zapf]

It's more of an advanced thing as I understand it, but at least *some* Linux distributions (maybe all? I really don't know) have a way to apply updates on a live, running system, *including the kernel!* Apps keep running and communicating to the outside world throughout the entire process, uninterrupted. So you really could update the entire OS mid-stream, and no one would notice.

But like I said, I don't know much more than that. I've never had a reason to look into it myself. Eventually though, you'd have to update the apps too, in which case they'd probably have to restart. Or reboot a memory-leaking app just because of that. But if everything works, you *might* be able to disguise an OBS-only reboot as just a hiccup in the same continuing stream. Maybe?

Everything seems really stable now. I paid for an Internet upgrade and got the most I can get on my COX business connection is 30 megs up so that gives me a little headroom there. Processing power is not a problem I have it on a temp computer now and it's running fine and I have a Dell power edge 740 in the garage with 40 cores and uh really good video card so that's not going to be an issue. And like I say it's running good now on this computer that's a quarter of what I'm going to put in so I think we're going to be fine.

I don't think I'm going to plan any rebooting. Streams stream all night also with the IRS on don't have many viewers right now this time of year but during nesting season we'll still have hundreds on during the night.

I think I probably need to make a new post for this but I have a question about what you think is better. I can tailor my streams using the camera (axis cameras are amazing you can do anything with them) I can use the zip stream and compression 2 get the data coming from the camera from 50 megs down to say 10 megs going into obs and then OS can handle it from there right now that's what I have the streams doing I tailored them first so they're sending about the same amount of data to obs that obs is going to be putting to YouTube. So with that said do you think that's a good way of doing it or do you think I should send the raw data from the camera two obs and just let obs compress it to the rate I set? and I do use H.265 from the camera to Yutube with an HLS stream.

 

[AaronD]

If your transports can handle uncompressed, then of course it's best to do that. The processing must be uncompressed anyway, so if it receives a compressed stream, it needs to uncompress first and *then* work with it. But that's a LOT of data to push around a network!

 

[John Zapf]

If your transports can handle uncompressed, then of course it's best to do that. The processing must be uncompressed anyway, so if it receives a compressed stream, it needs to uncompress first and *then* work with it. But that's a LOT of data to push around a network!

the cameras don't compress it in that way so OBS doesn't have to uncompress anything. and that's just a fraction of the data going. The cameras are running three profiles each all three record in full HD to the server and then all three stream and then two of them stream again in picture and picture so there's five streams total and then three streams being recorded in full HD all at the same time. but, again the cameras are handling all that. The camera send the data read to the hard drive and manage the other five streams not whatever data output I cap it at.