Remove the Desktop Audio from OBS, if you haven't already. (Settings -> Audio) Then you won't stream anything that comes out of your computer. Keep the mic active, and connected directly to the actual mic, not default.
Default technically works, but it relies on the operating system to make the connection instead. If the OS decides to change it, like if you plug in something else and it tries to be "smart" or "helpful" by switching to it, then OBS's default will follow that switch.