I too am having this exact problem, and it's doing my swede in! I have a MacBook Pro (mid 2012) connected via USB to a DJM-850 with inbuilt Traktor capability, and when I'm mixing with timecode vinyl (switches at the tops of channels 1 and 4 set to USB) OBS picks up the whiny tone from the timecode aswell as the track that's being played. This is when I've chosen the mixer as the source for the audio input capture. When I'm mixing normal vinyl (switches at the tops of the channels set to Phono) it comes through completely fine, and I imagine it would do with CDJs aswell.
I've now got my old Audio 8 DJ out from before I got this mixer, connected it to my laptop via USB and then connected the master phono out on the mixer to the line input on Channel C (5/6) and there is no light from the levels on it unless I turn it up quite loud (I think I may have to tinker with the phono gain on it but I can't open the damn settings tool!), and OBS still doesn't register anything when the Audio 8 is chosen on the audio input capture.
All I have seen in my internet browsing is that it's highly likely that we might have to download a rather expensive app called Loopback which can route the audio from all sorts of things to/through eachother on your computer. It also seems like this is a uniquely Apple problem, which is another kick in the teeth after they removed the line in capability from MacBooks aswell. Dicks.
I have downloaded the free trial of Loopback to see if I can get it to work without paying for it first, and will keep testing and looking and see if there are any cheaper options, and hope that someone a bit more knowledgable chips in here aswell!
