You may try "Source Clone" or "Asynchronous Source Duplication" plugin with combination of the Gain filters or just Audio Mixer volume sliders (this depends how you adjusted the volume in the first place):
Add source to OBS that lets you clone sources to allow different filters than the original
obsproject.com
Introduction This plugin provides a duplicated source of an asynchronous source. Asynchronous filters cannot be applied to a scene or a group. That means you cannot have one source as two scene-items with and without the asynchronous filter...
obsproject.com
The newly added Source Clone or Asynchronous Source Duplicator source/filter can be excluded from all output tracks but still can be monitored. In some cases these sources make clones of the video image too, so be careful to not blend video twice into output when you need only audio (zero sized source also an option).
