Got a new modem, hooked it up, nothing, messed around with a cable splice, nothing. Randomly was doing some pingtests on pingtest.net and eventually everything started working like magic. I have no idea what happened. I think there was a down router at some place in the network that was detected and it was turned off or something.
Weird thing was I took the same modem over to my friends across town and it worked fine over there, just didn't at home for some reason.
Also some feedback for R1CH if someone could point him at this thread as I did a bit of poking around while my internet was messed up.
As of recently I've had some issues with my internet connection and I found his really helpful bandwidth tester on teamliquid.net. I'm not exactly sure why it's not on the OBS forums and no one talks about it here. It's extremely handy for diagnosing network issues with twitch.
Suggestion for this tiny little program is maybe testing multiple TCP window sizes until it arrives at the one with the least amount of jitter and best RTT. I don't know if that's already what it does, but after playing around with the manual sizes a bit I ended up finding a better one then it was apparently using. Telling someone how to manually set this in OBS would be a good idea too as it seems to make a decent amount of difference when you do do it.
http://www.teamliquid.net/forum/tech-support/478845-twitchtest-twitch-bandwidth-tester
Also, found on his website he has a custom bit for dealing with streaming and latency via QoS, although it requires a piece of hardware to do it. Although not extremly expensive, it's just a router with a nice piece of firmware attached it. Something similar probably could be pretty easily done with DD-WRT, which should be common place for anyone that streams as it's basically the equivalent of the firmware on $300 routers.
DD-WRT is extremely robust and supports quite a bit of that functionality with a 'normal' install. I haven't actually attempted to do what he was talking about in DD-WRT yet as it seems to work pretty good anyway.
http://r1ch.net/11-video-streaming/4-reduce-gaming-and-streaming-lag-with-a-mikrotik-routerboard