The original Red Faction? I know OBS is really more geared toward DX9 and later games... that one's almost 20 years old if memory serves, may still be DX8 (or even 7), so unlikely that one will get working.
SR3 though really should meet the specs needed, and BF3 definitely should (unless their anti-cheat stuff is killing the ability to 'hook' the game... I personally am boycotting EA due to Origin so can't help there). On the up side, I'm told that Window Capture mode (with Aero on for the hardware acceleration) is only very slightly slower than Game Cap mode. The big one to avoid using if at all possible is Monitor Capture.
Are you running into speed problems? If window capture works OK, I'm trying to understand the motivation to get Game Cap working. If it's performance issues (in-game or with the stream) do post a log from a live streaming session (from %appdata%/OBS/logs/ ) and we can look through that and see what else might be causing them. :)