Number of dropped frames due to insufficient bandwidth/connection stalls

[daveairel]

I'm a a relatively new (and slightly technical) OBS user. I have a 2020 MacBook Pro M1. My configuration is 2 iphone (11ProMax & 12Pro) as cameras using obs.camera and a Zoom H2n mic. Everything connects and records fine at 720p / 60fps and my bitrate is set to 4000 kbps, but I get lots of dropped frames on the stream. This afternoon testing yielded the message (log file attached):

13:40:24.986: Output 'simple_stream': Total drawn frames: 35376 (35385
attempted)
13:40:24.986: Output 'simple_stream': Number of lagged frames due to
rendering lag/stalls: 9 (0.0%)
13:40:24.986: Output 'simple_stream': Number of dropped frames due to
insufficient bandwidth/connection stalls: 12554 (35.6%)

If I'm reading this correctly, the M1 is able to render the images fine (only 9 dropped frames) but the stream is hitting issues with 35% dropped frames. I assume this points at my internet connection, which is a wired ethernet connection to a 400Mbps up / 40Mbps down connection. I've run speedtest.net a pile of times and I will occasionally see upload speeds fall below 10Mbps, but not for long. Is there a way to tell for sure whether the issue is my internet connection or the M1 getting overloaded trying to stream?

I have iStat Menus running and its showing slower uploads, but I can't tell the cause from there. Any guidance (remember I'm not a programmer) would be appreciated.

 

[Lawrence_SoCal]

Kinda odd
the 2 iPhones are WiFi connected, right? You definitely have potential traffic contention issues with wifi to M1 Mac. But Mac is using wired Ethernet, which is good, and that error is from Mac to stream provider. Are you using a singular access point/router [most common home setup]? is that device getting overloaded?

Have you tested using Record (not stream) and how does it look? If no dropped frames during recording, then issue would certainly seem to be in the network uplink (test just to confirm)

Though your situation isn't helped by the fact that OBS is running in a translated mode vs native (which makes sense based on waiting for underlying dependencies to get ported by others [details in another thread in this forum], then re-doing OBS software from x86 is no small task, especially for something as complex and powerful as OBS

What are you doing to monitor your LAN/WAN? Might something else be using that upstream bandwidth? Note: I'm specifically NOT asking what you 'think' might be using Internet bandwidth. I'm asking what your network traffic actually looks like. And you can't monitor your computer to determine that (unless there is no WiFi and single computer was only device powered on and connected to LAN).
If you aren't monitoring, you are only guessing. This includes background tasks/apps on your computer which many people tend to forget they have running. Classic example is having a background file sync process trying to upload files while trying to stream. Or streaming via OBS and also streaming via Zoom/Teams, etc.

 

[BCFischer]

Kinda odd
the 2 iPhones are WiFi connected, right? You definitely have potential traffic contention issues with wifi to M1 Mac. But Mac is using wired Ethernet, which is good, and that error is from Mac to stream provider. Are you using a singular access point/router [most common home setup]? is that device getting overloaded?

Have you tested using Record (not stream) and how does it look? If no dropped frames during recording, then issue would certainly seem to be in the network uplink (test just to confirm)

Though your situation isn't helped by the fact that OBS is running in a translated mode vs native (which makes sense based on waiting for underlying dependencies to get ported by others [details in another thread in this forum], then re-doing OBS software from x86 is no small task, especially for something as complex and powerful as OBS

What are you doing to monitor your LAN/WAN? Might something else be using that upstream bandwidth? Note: I'm specifically NOT asking what you 'think' might be using Internet bandwidth. I'm asking what your network traffic actually looks like. And you can't monitor your computer to determine that (unless there is no WiFi and single computer was only device powered on and connected to LAN).
If you aren't monitoring, you are only guessing. This includes background tasks/apps on your computer which many people tend to forget they have running. Classic example is having a background file sync process trying to upload files while trying to stream. Or streaming via OBS and also streaming via Zoom/Teams, etc.

Lawrence:

I'm having similar concerns as Dave. What do you recommend as far as monitoring other potential bandwidth bandits, including those that are running in the background of the PC also running OBS?

Thanks, take care, and stay safe,

Brian

 

[Lawrence_SoCal]

Unfortunately, I'm not aware of a big red Easy (and cheap) button type network throughput monitoring solution/approach. I know a bunch of really expensive Enterprise class solutions, but this is the wrong forum to discuss such.

The short answer is RTFM and research your network devices (starting at modem and working your way back). But that doesn't really help does it?
Buying a dedicated device (between router and modem) just to make monitoring easy, seems silly (to me at hundreds of dollars). But for some folks that may be worth it. In the open-source world, and a Raspberry Pi, one could roll-your-own for cheap [with a lot of knowledge]. There are actually a number of open source network monitoring tools, but the real issue is support for your specific network device (home router/access point for many folks) and most tools interface with protocols standards on business class network devices, which many consumer devices don't support. ugh
Each router/switch/modem varies in what is exposed to a user interface. So for some, you can log into a web interface and see real-time stats. For more powerful units (often small business and higher focused), a user can get details per connected device. it really does depend on exactly what you have. For example, a Netgear R7800 router (stupid Netgear and their terrible consumer device code quality, the only thing worse is all the other consumer device mfgs, but I digress). Turns out there is a 3rd party is maintaining the firmware (long after Netgear left it a mess) replaces a lot of the out-of-date open-source code in the firmware, and there is then an add-on that enables detailed real-time monitoring. This is but one example, there are plenty of others. Then there are items like SonicWall, etc which have fairly thorough monitoring capabilities. It really does depend. And then it depends on how technical you are and patient. For example, some routers don't really have real-time monitoring, but have some basics. With patience one could isolate and test individual devices, etc. [assuming family, etc doesn't have a fit]. Or you could, if devices support it, set up Quality of Service to prioritize your stream over other traffic [but that has limits and isn't a cure-all, even if it sounds like it should be].

First thing, I'd do a long upload test
- make sure everything else is powered off/unplugged ('smart' devices unplugged from power), WiFi disabled at router/access point
- Then, with monitoring setup as possible with your environment, plus monitoring on your computer.
- Make sure all possible background apps on computer turned off (no messaging, conferencing, status update, sync, etc apps running)
- do a large file upload to a site with a known ingest rate above your upload threshold, (so a multi GB file or larger?) and see what sustained upload speeds are as indicated on your PC. And does that # match your ISP connection rate (per monitoring)
That will establish a baseline rate (at that time of day, some ISPs are more impacted by neighborhood bandwidth usage than others). If need be, test at multiple times of the day (like after kids get home from school). Seriously, I was in Switch Internap datacenter in Las Vegas (yes, escorted by armed guards with assault rifles) and they talked about being almost able to set your watch by when the fans across many racks (dozens upon dozens) of Everquest servers would start spin up each weekday afternoon

Then recognize SpeedTest are highly optimistic (eliminating lower results, which happen to be the ones you need to know about). Now that file upload test I mentioned test can be deceiving (single target, etc) but can be a good starting point. Feel free to use an alternate test, just look for a sustained test that shows low upload results.

 

[Harold]

https://obsproject.com/wiki/Dropped-Frames-and-General-Connection-Issues covers all the connection issue scenarios

 

[Lawrence_SoCal]

What do you recommend as far as monitoring other potential bandwidth bandits, including those that are running in the background of the PC also running OBS?

Unless dealing with some REALLY sophisticated malware, your own PCs monitoring tools should have relatively accurate network interface card (NIC) monitoring which I know for sure can get down to app/process level on Windows.
makeuseof.com has a decent high-level article for Windows PCs posted in Jul '21 "What's using my bandwidth?" Seems like a decent article.

Also appears via quick Google search that there are a couple of hardware devices in the $100-200 range

 

[daveairel]

Kinda odd
the 2 iPhones are WiFi connected, right? You definitely have potential traffic contention issues with wifi to M1 Mac. But Mac is using wired Ethernet, which is good, and that error is from Mac to stream provider. Are you using a singular access point/router [most common home setup]? is that device getting overloaded?

Have you tested using Record (not stream) and how does it look? If no dropped frames during recording, then issue would certainly seem to be in the network uplink (test just to confirm)

Though your situation isn't helped by the fact that OBS is running in a translated mode vs native (which makes sense based on waiting for underlying dependencies to get ported by others [details in another thread in this forum], then re-doing OBS software from x86 is no small task, especially for something as complex and powerful as OBS

What are you doing to monitor your LAN/WAN? Might something else be using that upstream bandwidth? Note: I'm specifically NOT asking what you 'think' might be using Internet bandwidth. I'm asking what your network traffic actually looks like. And you can't monitor your computer to determine that (unless there is no WiFi and single computer was only device powered on and connected to LAN).
If you aren't monitoring, you are only guessing. This includes background tasks/apps on your computer which many people tend to forget they have running. Classic example is having a background file sync process trying to upload files while trying to stream. Or streaming via OBS and also streaming via Zoom/Teams, etc.

HI, Lawrence - Thanks for the response. Couple things I should have included earlier:

1. iphones are connected via USB (thru a hub).
2. The router I have is actually a gateway ... Arris Docsis3.0 (don't remember the model number but it's about 2 year old)
3. I have been recording while streaming and the recordings are clean. No visible dropped frames.
4. I'm using iStat Menu to monitor the internet connection. I don't know what it's telling me, that's why I'm needing help. There are other devices plugged in (my Solar City bridge which uses almost no bandwidth), my Sonos speaker (which was off). And my desktop iMac was on wifi, but only one browser window open monitoring the YouTube feed (I have 400 Mbps download); I wasn't uploading anything during the testing.
5. iStat Menu also has a CPU monitor, and it shows the CPU running at no more than 50%. I have no idea how accurate that is, BTW. Usage goes up when streaming, down when not, further down when the iphone feed is not active ...
6. I was careful to close all other apps on my MacBook before I did these tests. DropBox was open but not moving any files. Not sure about background services, though. But CPU utilization indicated it wasn't much.

I'd be happy to try better monitoring if it helps me figure this out but I haven't found anything useful. Happy to take suggestions ...

 

[daveairel]

I did some more testing this morning - to the extent possible, I made sure nothing else was on line (desktop Mac powered down, etc). After reading more on the link sent by @Harold , I tried enabling Variable Bit Rate. I then streamed/recorded for 50+ minutes. Same behavior as yesterday but the VBR made the resulting YT video much more watchable (still not great but pixellation is better than dropped frames), and many of the drops in bit rate were subtle and not that noticeable. So I'm pretty convinced the issue is with my ISP, so I'll be contacting them today. Any other suggestions here would be appreciated.

Two files attached - today's OBS log and the network performance graph from iStats. (note the graph is in kB/s so math is required)

 

[Lawrence_SoCal]

The issue may well be your ISP, but the onus is on you to monitor your LAN (not uncommon for an ISP to say they don't see an issue on their end, and then ball back in your court so to speak). I'm not trying to be discouraging, just don't want to see you get your hopes raised then dashed as to how much the ISP will do.
There are a number of Arris monitoring options. So, I'd recommend figuring out your specific gateway, and the monitoring options available for that model (could be a web interface, or there appear to be Android and iOS apps)

 

[daveairel]

The bad news is that I haven't solved this yet.

The good news is that I've proven to myself, and convinced our ISP, that the problem is on their side. Have had two tech calls so far, one escalation and I'm hopeful that I have their attention and that they'll get this resolved for me. I hope. More later.

 

[Lawrence_SoCal]

Good luck

 

[daveairel]

Just an update on this, no reply needed. I did get my ISP to fix the problem. Weirdly, they found a live, unterminated feed cable buried nearby that was radiating like crazy. Once they cut the signal to that cable, my issue disappeared, along with about 6 neighbors that I found out were having similar disconnect/upload speed issues. The next day, I was pushing 8 GB video files to YouTube at 40Mbps, and was able to stream 720/60 from OBS at 5000kbps for 45 minutes with zero frame dropouts. So, problem solved ... thanks for the feedback here.

 

[Lawrence_SoCal]

Thanks for the update. odd circumstance for sure.

 

[airxsteel]

Just an update on this, no reply needed. I did get my ISP to fix the problem. Weirdly, they found a live, unterminated feed cable buried nearby that was radiating like crazy. Once they cut the signal to that cable, my issue disappeared, along with about 6 neighbors that I found out were having similar disconnect/upload speed issues. The next day, I was pushing 8 GB video files to YouTube at 40Mbps, and was able to stream 720/60 from OBS at 5000kbps for 45 minutes with zero frame dropouts. So, problem solved ... thanks for the feedback here.

That is wild.