From your point of view, VBR is fine. But from the streaming service's point of view and the viewer's point of view, it's not good. It's due to the way the internet works, especially packet forwarding in the backbone routers. If there is a certain amount of constant data flow detected between some routers, it is classified as some kind of data streaming with realtime requirement to avoid lags and buffering on the clients. This is in contrast to short requests from small downloads or http browser traffic.To achieve no-buffering for the clients, the backbone routers reserve the detected bandwidth for the streaming data.
But with VBR the bandwidth is variable. The backbone routers reserve something in the middle between the lowest and the highest detected data rate. Now, if your stream suddenly sends high data rate due to high motion, it may be that it congests the reserved bandwidth, and the viewers get buffering. You as sender doesn't see this, and even if you watch your own stream realtime, you will not always see this. It's not in your void's as well.
It gets worse if you have a large amount of viewers. In this case, the outbound network connection of the streaming service might congest if your VBR goes up to the upper limit. This results in lags and buffering on the clients. The streaming service puts as many users on a server as it is possible without congesting its network connection. If you use VBR, you suddenly need about 20% more bandwidth, and this multiplies with the amount of viewers. To not congest the outbound, the streaming service has to reserve this 20% more bandwidth in the first place, but then there is no point in using VBR at all - you can just use CBR with the same bandwidth as your upper VBR limit.
And this is what you as streamer should in fact do: don't use VBR. Instead, use CBR with the same bitrate you set as upper VBR limit. It might look as inefficient use of bandwidth, but it grants your users a smoother and no-buffer stream.
