Question / Help Game Capture problem/bug (?)

[jimmy213]

Hello.
I usually play games in borderless windowed or windowed mode. But the Game Capture in OBS Studio doesn't work (it seems it's only for fullscreen games).
I don't want to use Monitor Capture or Window Capture as they capture other overlays too (Steam overlay, Mumble overlay, etc).

Is there a plan to fix/add the ability of Game Capture to detect the games despite the mode we use?

 

[jimmy213]

I tried that but I get a black screen with sound.

 

[jimmy213]

Unfortunately it doesn't. The output file shows nothing and there's only sound.

 

[jimmy213]

Here's the log.

 

[jimmy213]

Yes, it's checked by default. I'll try disabling it, but I don't think it matters.
Also, I opened everything (OBS Studio, Steam, Game) with admin rights but nothing changed.

 

[RytoEX]

I've captured games in windowed mode just fine. I pretty much exclusively use OBS Studio to game capture windowed games and not fullscreen games. Has this setup ever worked for you before with anything else? Can you record anything else successfully? Your log shows a few interesting points:

1. 19:54:38.161: CPU Name: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
   19:54:38.161: CPU Speed: 2400MHz
2. 19:54:38.162: Physical Memory: 2047MB Total, 2047MB Free (NOTE: 2 or 4 gigs max is normal for 32bit programs)
3. 19:54:38.191: Adapter 1: AMD Radeon R7 200 Series
   19:54:38.191: Dedicated VRAM: 1055023104
   19:54:38.191: Shared VRAM: 3239878656
4. 19:54:40.331: source 'Desktop' (scene) created
   19:54:40.333: source 'Display Capture' (monitor_capture) created
   19:54:40.334: source 'Rocket League' (scene) created
   19:54:40.334: source 'Game Capture' (game_capture) created
5. 19:58:54.536: [game-capture: 'Game Capture'] attempting to hook process: RocketLeague.exe
   19:58:54.537: [game-capture: 'Game Capture'] using helper (compatibility hook)
   19:58:55.253: [game-capture: 'Game Capture'] Hooked to process: RocketLeague.exe
   19:58:55.253: [game-capture: 'Game Capture'] (half life scientist) everything.. seems to be in order
6. 19:58:55.266: [game-capture: 'Game Capture'] Hooked DXGI
   19:58:55.266: [game-capture: 'Game Capture'] D3D9 values invalid, manually obtaining
   19:58:55.467: [game-capture: 'Game Capture'] Hooked D3D9
   19:58:58.745: [game-capture: 'Game Capture'] attempting to hook process: RocketLeague.exe
7. 19:59:19.168: [Media Foundation encoder]: Activating encoder: AMD VCE
8. 20:00:17.504: [game-capture: 'Game Capture'] capture window no longer exists, terminating capture
9. 20:00:30.397: Output 'adv_file_output': Number of skipped frames due to encoding lag: 1797 (84.4%)
   20:00:30.397: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 414 (19.5%)

1. This is a very old CPU (early 2007).
2. OBS is only accessing 2GB of memory and seems to be the 32-bit version. Are you running a 32-bit version or 64-bit version of Windows 10 (the log doesn't say)?
3. Your graphics card seems okay, but let's get more info. What exact card in the R7 200 series is it?
4. Does the "Desktop" scene work with your "Display Capture" source? Is the "Game Capture" source the only thing in the "Rocket League" scene?
5. Here, OBS Studio seems to get RocketLeague.exe just fine. However...
6. Here, OBS Studio seems to get the DXGI and D3D9 hooks, though it doesn't say whether it actually hooked Rocket League. I know Rocket League runs in DirectX 9, but I'm not sure what a successful DX9 game hook looks like in an OBS Studio log. Maybe someone else can chime in on this point?
7. I'm pretty sure that Jim and others have said that the Media Foundation encoders are terrible, but someone else is free to correct me on this. Given the age of your CPU, x264 might be difficult, but have you tried it?
8. At this point, the log notes that the game window no longer exists. Did you close it manually at this point, or did something else close it?
9. Here's the real business - your computer skipped/lagged way too many frames. Have you successfully recorded any footage with lower skip/lag rates?

 

[jimmy213]

1. I know but now I usually capture with AMD's VCE. Although, even at its age, it can handle game capture (at least in the original OBS). I used to capture Path of Exile without issues and in my native resolution (in original OBS).
2. I have a 64bit installation of Windows 10. I forgot to test with the 64bit version of OBS Studio. I'll give it a try.
3. It's the HD 7770 1GB.
4. Yes and yes.
7. I'll also try to capture with my CPU and get back with the results.
8. I didn't do anything from my side...
9. Well, I recorded Rocket League a few days ago with the Window Capture (with AMD) and the output file wasn't nice. It was too laggy (many skipped frames I guess). Although I wasn't having any issues in-game. But if I locked the fps to 30, the output was fine.

I know that it's not a gaming/capturing PC, but it tries :)
Also I record in a separate drive. Maybe worth mentioning, with an other capture software (D3DGear) I use the AMD codec without issues.

 

[RytoEX]

I know the feeling of having an old computer that tries its best to keep up.

There was a hotfix just released for OBS Studio (0.15.2) that specifically addresses D3D9 games on Windows 10's latest update. Maybe give the 64-bit version of that a try?

 

[jimmy213]

Well, I tried the hotfix (both 32bit and 64bit) and nothing changed. I also tried capturing with my CPU and nothing changed too. Window Capture and Display Capture both work but no luck with Game Capture.

I don't know what else to try. Is it a known issue or it's just me?

 

[RytoEX]

Can you post new logs for recording attempts with at least the 64-bit version? That way we can see if something has changed.

Although, even at its age, it can handle game capture (at least in the original OBS). I used to capture Path of Exile without issues and in my native resolution (in original OBS).

Does OBS Classic still work for you without issue? If so, perhaps also post logs from that to compare.

9. Well, I recorded Rocket League a few days ago with the Window Capture (with AMD) and the output file wasn't nice. It was too laggy (many skipped frames I guess). Although I wasn't having any issues in-game. But if I locked the fps to 30, the output was fine.

If locking output to 30 FPS produces working results, maybe that's what you'll have to do.

Also I record in a separate drive. Maybe worth mentioning, with an other capture software (D3DGear) I use the AMD codec without issues.

The method used by other software may not be comparable to what OBS Studio uses, but it is worth mentioning in case someone else more knowledgeable can answer why one performs differently than the other.


Since you're using AMD to encode, it might be relevant to point you towards jackun's AMD VCE fork of OBS Classic.
https://obsproject.com/forum/thread...x1440p-at-60-fps-w-amd-vce.50536/#post-224725
https://obsproject.com/forum/threads/amd-stuck-with-media-foundation.47117/
https://obsproject.com/forum/threads/obs-studio-amd-vce-bitrate-problem.45454/
https://obsproject.com/forum/threads/obs-branch-with-amd-vce-support.13996/ (the first post has download links; pages 140-143 have some interesting tidbits about AMD VCE support in OBS Studio)


It does seem that there's recently been some movement towards getting proper AMD VCE support in OBS Studio, but that'll likely still be a while before it's done.

 

[jimmy213]

1. No, OBS Classic outputs a black screen as well in Game Capture (it used to work on Windows 7 if I remember correctly). But it's in 32-bits. Is there a 64-bit version? (although I don't think that it matters for the issue I have)

2. Yes, but as I said in my first post, I want to use Game Capture and not Window Capture or Display Capture as I don't want the Steam overlay and Mumble overlay to be recorded.

3. I tried that version but the black screen remains in Game Capture.

I'll also make a test in a laptop that I have and see if it's really a problem of OBS Studio or it's my desktop's problem.

EDIT
1. I captured with OBS Classic 64-bit too, but black again.
2. Running OBS Classic 64-bit on the laptop, the Game Capture worked fine. In OBS Studio I get a black screen again. In both versions of OBS I captured with the CPU. It also runs Windows 10 x64.

 

[Osiris]

Completely uninstall d3dgear.

 

[jimmy213]

We were trying to find a complex solution here and you ruined it @Osiris. Just with 3 words.

Just kidding. It actually worked. Now I can capture with Game Capture source. Is there a conflict with these programs or in general between other capturing software and OBS?

Anyway, thanks @Osiris and thanks a lot @RytoEX for your help.

 

[Osiris]

d3dgear hooks into every directx or opengl game you run, and that conflicts with OBS' game capture hook.

 

[jimmy213]

I didn't know that. Thanks for the info :)

 

[RytoEX]

Completely uninstall d3dgear.

d3dgear hooks into every directx or opengl game you run, and that conflicts with OBS' game capture hook.

This is exactly the kind of expert knowledge I was hoping someone would deliver. Thanks Osiris!