That's just the nature of digital live production. Audio does that too, but the difference is the sample rate.
For audio, the minimum "full quality" sample rate is 44.1kHz, or 44100 samples per second. (a sample is a single number for each channel) A 32-sample buffer, for example, allows that many samples to be sent at once across a network or radio link (record the buffer, send it, and then play it out), and represents a bit less than 1ms of added delay. (44 samples would be 1ms at 44kHz) 1ms of audio is about 1ft or 30cm in air. Not a problem.
For video, we typically run about 25-60fps, which can be thought of as a sample rate of 25-60Hz, treating each color of each pixel as a separate channel. That same 32-sample buffer now represents a whole second! Video is much more sensitive to buffering, and some operations *require* a buffer of at least one complete frame. So it's quite difficult to get the latency down to where you don't notice it, especially when you're using a tool that is designed to process it in arbitrary ways.
What are you trying to do?
If all you want is a single audience camera to feed a backstage monitor, then you should look at hardwiring that connection directly. No computer at all, only a converter to a long-distance format, if it isn't already...but understand that the converters may also have their own frame buffers.
If you want to live-produce something good with minimal latency, then you might look at a physical video mixer. It's designed specifically and only for that one job, so its own buffers can be kept small and less noticeable, compared to a multitasking general-purpose PC that might eventually get around to processing the previous 3 frames or so.
