no, you should be fine.
With anywhere from 3 to 5 Megabits being used as your stream bandwidth, that still leaves you with 5 - 7 megabits for other things on your network, including the broadcasting computer, to make use of.
Minimize Network Impact is mainly a Quality of Service option. You wont be maxing out your main internet bandwidth with a broadcast, so you'll be fine.
pro-tip: you can adjust the amount of bandwidth your stream is using WHILE broadcasting to twitch. start at 3, open the stream in another window or on a laptop, phone, whatever, and see how it looks. then pop it up to 4, then to 5, and see the quality difference
keep in mind, however, that just because you can send at 5 doesnt mean people who watch your stream want to download 5. I broadcast at 2 Megabits on average, and I have a lot of viewers that say they have a lot of buffering issues. this is a problem WITH Twitch specifically, not the viewer or the broadcaster