It's true that VCE produces a far worse image at the same bitrates so you should try to only use it to record high quality footage at very high bitrates. I stream 720p w x264 and record at 1080p with VCE.
Also, 720p60 should look pretty acceptable at x264/3500kbps/veryfast. Can you provide us a sample so we can judge how much pixelation we're talking about here?