We have observed that once in a while a Windows Update will re-enumerate the audio devices and give them different Device IDs, which can cause OBS to lose them, since OBS stores Device IDs. (This is why we pause Windows Updates when we have a livestream planned)
In OBS, open Settings and select the Audio page.
"Global Audio Devices" shows your audio sources.
"Advanced", "Monitoring Device" shows the monitoring device, which is your issue. This may be a specific device, or "Default" to use whatever device Windows uses by default.
When OBS can no longer find a device, it may show "Device not connected or not available" in one or more of these locations.
If you have "Default" selected as your monitoring device, you can either change it to be your headphone device (maybe a USB audio interface?), OR you can tell Windows to change its default output device in Windows Sound Settings. Changing the Windows default would also change where Windows warning sounds and suchlike play.