VAC (or one of the freeware alternatives) really is about the only way, due to how the Windows audio subsystem works.
I'd incidentally expect your viewers to be curious, hearing only half of a conversation. Or seeing you talking without hearing anything. Or hearing a vast amount of dead-air while you talk with friends, if you don't use a face-cam.