Realize that immediately after starting OBS (or the PC) the computer is busy doing various background things and takes a little time to "settle down". Make sure your latency measurements are after many minutes (at least 5ish?? on app restart, 5->10 minutes for OS restart... just guessing, each person's PC is slightly different)
Audio being faster is the norm, hence OBS having built in audio latency adjustment option. Varying video latency over USB capture device.. ugh.. not my area, but also, not surprised ... as noted above, I'd certainly be looking into related driver changes (maybe an update required? or an update messed things up?)
Not all USB3 are the same, so make sure using fastest of the USB3 ports for the video capture? or maybe try the other USB3 port?
I'm just guessing, but with you mention of varying video latency, I'm inclined to suspect
- capture card driver issue
- specific sequence (and timing) of restarting laptop, OBS, and capture device leading to variability (ideally shouldn't.. but...)
Do you know Windows OS well enough to know how to restart a service/process?
I'd be inclined to test will Ending Task on USB Capture driver and restarting, or going into Services and starting/stopping [depends on whether running app, or running as a service]. And see if you can get consistent video capture latency
Have you tested seen a difference in video latency whether video feed is started before or after computer/OBS start? My inclination (which may NOT be relevant at all), would be to start PC with USB capture device NOT plugged in. After 5 minutes or so, plug in USB HDMI capture adapter. Once CPU/disk activity settles down (per Performance Monitor), then turn on OBS, again waiting for CPU/Disk activity to settle down. Only then turn on camcorder and start video feed. [this assumes that the capture card being plugged in, but not receiving anything, will show up in OBS correctly (ie enumerated/listed/device available)].