Btw, for anyone wondering, Osu! is a game that Nvenc excels at encoding-wise.
I would look at trying to limit your in-game framerate. You're letting it run wild with GPU usage, which is forcing OBS to have very little resources for frame building... especially with a game like Osu, which uses the exact same 2d scaling pipelines that OBS does. Unfortunately I'm not familiar with the graphics settings in Osu, and I'm not about to tell you to turn on vsync since that's probably going to be unplayable.
I would look into
frame limiting methods @BK-Morpheus recommended recently. I'm not sure if the Nvidia profile inspector will pick up Osu! to have its own profile, but it's absolutely worth a shot.