OBS branch with AMD VCE support.

[Kobata]

So can this do 1080p@60fps game capture from a windowed game running quality settings? My efforts have been met with "High encoding cpu" error while attempting 5000, 10000, 20000, 30000 kbps bitrates.

edit: the video also freezes as a result.

Most AMD card's encoders simply cannot keep up with 1080p60 on quality preset in general. The newer ones with VCE3 can do that, but with AMD's constant rebranding it can be a bit hard to figure out.

The current list is R9 285/380 and Fury cards. Also some specific newer mobile APU models (8000 series).

 

[ufster]

Most AMD card's encoders simply cannot keep up with 1080p60 on quality preset in general. The newer ones with VCE3 can do that, but with AMD's constant rebranding it can be a bit hard to figure out.

The current list is R9 285/380 and Fury cards. Also some specific newer mobile APU models (8000 series).

Oh, ok. Thanks for the clarification. It's the GCN 1.1 - 1.2 thing I presume. Yeah, I have an R9 290 which doesn't help. In your experience with VCE is it best if I record using quality with 720p@60fps settings or balanced with 1080p@60fps? On a slighty different note, this might be a good excuse to buy a Fury X or Fury.

 

[dping]

So can this do 1080p@60fps game capture from a windowed game running quality settings? My efforts have been met with "High encoding cpu" error while attempting 5000, 10000, 20000, 30000 kbps bitrates.

edit: the video also freezes as a result.

when doing 1080@60, change the preset at the top of VCE settings to balanced or speed. as long as you have more bitrate you shouldn't see much of a difference

 

[cl0ver]

@dping i change the configuration but the obs crash again

 

[ufster]

when doing 1080@60, change the preset at the top of VCE settings to balanced or speed. as long as you have more bitrate you shouldn't see much of a difference

Thanks for the response although I already understood that much (regarding 1080p@60 limitations) from Kobata's reply. My concern is now whether 720p quality would be a better choice than 1080p balanced or not, I assume the second part of your response was related to that but I couldn't be sure. If so, it seems like 720p@60 quality settings would be better overall for file size and image quality. I have done some tests with Fez but that game doesn't have many high res textures etc. so both videos turned out quite acceptable. I'll try once more with BF4 tonight and try to figure this out for certain.

 

[Kobata]

My concern is now whether 720p quality would be a better choice than 1080p balanced or not

This is going to depend mostly on what bitrate you have set -- 1080p has 2.25x more pixels to deal with than 720p, so to get the same quality with the same settings you would need to change the bitrate by the same factor, slightly more to deal with the drop in settings from quality->balanced. Once you get high enough bitrate though, the differences will start to get fairly unnoticeable.

 

[dping]

Thanks for the response although I already understood that much (regarding 1080p@60 limitations) from Kobata's reply. My concern is now whether 720p quality would be a better choice than 1080p balanced or not, I assume the second part of your response was related to that but I couldn't be sure. If so, it seems like 720p@60 quality settings would be better overall for file size and image quality. I have done some tests with Fez but that game doesn't have many high res textures etc. so both videos turned out quite acceptable. I'll try once more with BF4 tonight and try to figure this out for certain.

Yes. higher quality preset means more time is spend encoding each pixel, hence, more compression and lower bitrate needed for the same qual. I prefer 720@60 unless doing a youtube video, then use speed preset for local recording.

 

[ufster]

This is going to depend mostly on what bitrate you have set -- 1080p has 2.25x more pixels to deal with than 720p, so to get the same quality with the same settings you would need to change the bitrate by the same factor, slightly more to deal with the drop in settings from quality->balanced. Once you get high enough bitrate though, the differences will start to get fairly unnoticeable.

Ok, for some reason BF4 windowed mode is wrong size for the resolution but I'll set up another game with detailed textures and models to compare different settings at different bitrates and determine at what bitrate I am satisfied with the quality. But if I'm right the AVC codec is also dependent on the type of movement in the game etc. because it essentially analyzes frames as it converts them. This means different kind of games might respond differently to the settings, am I accurate in assuming this?

Yes. higher quality preset means more time is spend encoding each pixel, hence, more compression and lower bitrate needed for the same qual. I prefer 720@60 unless doing a youtube video, then use speed preset for local recording.

I tried balanced preset with BF4 but that was full screen downscaled by a factor of 2 (3440x1440 -> 1720x720, 10Mbps) and for some reason it turned out muddy and blocky. OTOH, 720p@60 with 5000 kbps and Quality setting with Fez (the only game I tried so far) gave me some promising results. I can't really compare the two in terms of quality unless I get BF4 to run at 1920x1080 windowed with proper sizing (the window for some reason is always smaller and can't be resized without crashing the game) so I decided to try another game like Metro 2033 Redux or Shadow of Mordor when I get the time for a more reasonable comparison.

update edit: tried Metro 2033 Redux Windowed 1080p recording with 720@60 quality and 1080@60 balanced, both 10 Mbps. The 720 image feels a bit sharper, will try with higher bitrate for 1080p (15Mbps).

 

[Kobata]

But if I'm right the AVC codec is also dependent on the type of movement in the game etc. because it essentially analyzes frames as it converts them. This means different kind of games might respond differently to the settings, am I accurate in assuming this?

Yes, the exact bitrate will vary depending on the source. If bitrate isn't an issue at all, it can make sense to just use CQP mode -- it probably will end up looking slightly worse than VBR with the same average bitrate, but it tries to target a specific amount of 'quality' (in a rather basic way) and will use whatever it needs to achieve that. The settings for that mode are the I/P/B QP, and they should be somewhere in the 20s usually, with a lower number being higher quality. (Ignore B-delta, that one's not important for this, and B-frames are generally better off anyway) It's also useful to get a feeling for the bitrate you'd need for a specific source -- record a representative sample in CQP and watch the bitrate display to see what it's using in different situations

 

[sneaky4oe]

Is there a working way to stream Witcher 3 with graphics maxed out and good clear grass in movement? I get about 40FPS with all maxed out on my single R9 390x, and can stream it 720p with my CPU, but when I switch to VCE balanced 8mbit/s, stream stucks when I go back to the game.
GOP = 60
IDR = 0

Same issue about DayZ.

 

[dping]

Is there a working way to stream Witcher 3 with graphics maxed out and good clear grass in movement? I get about 40FPS with all maxed out on my single R9 390x, and can stream it 720p with my CPU, but when I switch to VCE balanced 8mbit/s, stream stucks when I go back to the game.
GOP = 60
IDR = 0

Same issue about DayZ.

Stream sucks? I dont understand what you mean by sucks?

From what I know about Witcher, it is very hard on the GPU so VCE might not be a plausible solution for that game. That and streaming with anything below quality preset will not have great quality and nor will 8000 bitrate for viewers.

 

[sneaky4oe]

Stream sucks? I dont understand what you mean by sucks?

From what I know about Witcher, it is very hard on the GPU so VCE might not be a plausible solution for that game. That and streaming with anything below quality preset will not have great quality and nor will 8000 bitrate for viewers.

*stucks
I'm streaming to youtube, it encodes high bitrate on source to lower on 1080p without any effort on my side, and my viewers are kinda fine with resulting quality.

Kk, gotta use CPU for this one than.

 

[ufster]

Yes, the exact bitrate will vary depending on the source. If bitrate isn't an issue at all, it can make sense to just use CQP mode -- it probably will end up looking slightly worse than VBR with the same average bitrate, but it tries to target a specific amount of 'quality' (in a rather basic way) and will use whatever it needs to achieve that. The settings for that mode are the I/P/B QP, and they should be somewhere in the 20s usually, with a lower number being higher quality. (Ignore B-delta, that one's not important for this, and B-frames are generally better off anyway) It's also useful to get a feeling for the bitrate you'd need for a specific source -- record a representative sample in CQP and watch the bitrate display to see what it's using in different situations

Ok, I tried recording some samples with 25-25-25-4 settings that were default on my OBS settings screen. The videos stutter and OBS complains of high cpu usage during recording, the bitrate is around 3.5-4Mbps. Also the Metro benchmark slows down massively in conjunction with the OBS high cpu usage warnings. I guess the GPU can't handle both playing the game and recording. Videos are also not as sharp as the ones recorded with CBR settings. What I want to know is apparently increasing these numbers lower image quality, if so how am I supposed to increase image quality and make vce not stutter without increasing bitrate. As far as I understand higher bitrate with same settings will result in higher utilization of the video encoding engine in the GPU.

 

[Kobata]

Ok, I tried recording some samples with 25-25-25-4 settings that were default on my OBS settings screen. The videos stutter and OBS complains of high cpu usage during recording, the bitrate is around 3.5-4Mbps.

That's rather strange, it should if anything be faster since it's actually more directly controlling the encoder. There's always strange issues with random things that really shouldn't be doing much making it excessively slow though. Bitrate usually shouldn't have a large effect on performance -- if anything, higher bitrate is easier on the computational section as it needs to do less compression -- although performance would fall off a cliff if you hit amounts excessively high enough that VRAM/PCI-e/System RAM bandwidth is saturated.

 

[ufster]

That's rather strange, it should if anything be faster since it's actually more directly controlling the encoder. There's always strange issues with random things that really shouldn't be doing much making it excessively slow though. Bitrate usually shouldn't have a large effect on performance -- if anything, higher bitrate is easier on the computational section as it needs to do less compression -- although performance would fall off a cliff if you hit amounts excessively high enough that VRAM/PCI-e/System RAM bandwidth is saturated.

Is there a way to change the cqp settings so that it uses higher bitrate without lowering image quality (25-25-25-4) to see if it helps with the image quality and GPU video encoder utilization?

 

[dping]

Is there a way to change the cqp settings so that it uses higher bitrate without lowering image quality (25-25-25-4) to see if it helps with the image quality and GPU video encoder utilization?

those numbers you are controlling are the QP of each type of frame. lower is more perfect. QP of 0 is pretty much lossless but those are target numbers, so I dont know if they enforce 0 of set to 0

 

[ufster]

those numbers you are controlling are the QP of each type of frame. lower is more perfect. QP of 0 is pretty much lossless but those are target numbers, so I dont know if they enforce 0 of set to 0

I understand. My issue currently is that 25-25-25-4 gives me choppy playback video and stuttering gameplay with 3.5-4.0 Mbps videos that are of a lesser quality than ones recorded using CBR with 720p@60, 10Mbps using Quality preset. Kobata stated that higher bitrate will likely be easier on the GPU encoder so I wanted to know if I could keep these quality settings and increase the bitrate for better image quality and less of a performance hit. I don't need such low bitrates as these videos are local files that will be edited in Sony Movie Studio and re-encoded before getting uploaded to Youtube so quality of files are a bigger concern than size.

edit: Constant QP Settings and resulting files. GPU is R9 290.

Running Metro Benchmark 1080p Windowed, Quality Very High, 16x AF, Normal Motion Blur, Tesselation Very High, Vsync Off, Advanced PhysX off. Turning off SSAA stops GPU choking and choppy video midway through benchmark.

Results
No recording: 53 fps.

18-18-18-4: 49 fps, resulting bitrate 12.9 Mbps. Some macroblocking if I really look for it. Image somewhat noticeably sharper than 22-22-22-4.

22-22-22-4: 50 fps, resulting bitrate 7.7 Mbps. Macroblocking but not so pronounced. Image quality not discernible from 23-23-23-4.

23-23-23-4: 50 fps, resulting bitrate 6.5 Mbps. Macroblocking but not so pronounced, Image quality a bit better than 25-25-25-4.

 

[Kobata]

I understand. My issue currently is that 25-25-25-4 gives me choppy playback video and stuttering gameplay with 3.5-4.0 Mbps videos that are of a lesser quality than ones recorded using CBR with 720p@60, 10Mbps using Quality preset. Kobata stated that higher bitrate will likely be easier on the GPU encoder so I wanted to know if I could keep these quality settings and increase the bitrate for better image quality and less of a performance hit.

CQP mode is controlled for quality*, not bitrate -- you can't directly change the bitrate. However, changing the quality-speed preset will cause the bitrate to change as the faster presets will need more bitrate to meet the parameter given.

* It's not really directly quality, but the way the encoding works a lower QP will be closer to the original image.

Edit: A difference from 53 to 50 fps is probably about expected with any setting though, as the capture itself tends to cause some slowdown. The encoder is dedicated hardware and doesn't have much effect on the rest of the card on its own, beyond being another thing competing for bandwidth -- which is probably part of why SSAA makes it hurt, SSAA is extremely bandwidth intensive.

 

[dping]

I understand. My issue currently is that 25-25-25-4 gives me choppy playback video and stuttering gameplay with 3.5-4.0 Mbps videos that are of a lesser quality than ones recorded using CBR with 720p@60, 10Mbps using Quality preset. Kobata stated that higher bitrate will likely be easier on the GPU encoder so I wanted to know if I could keep these quality settings and increase the bitrate for better image quality and less of a performance hit. I don't need such low bitrates as these videos are local files that will be edited in Sony Movie Studio and re-encoded before getting uploaded to Youtube so quality of files are a bigger concern than size.

edit: Constant QP Settings and resulting files. GPU is R9 290.

Running Metro Benchmark 1080p Windowed, Quality Very High, 16x AF, Normal Motion Blur, Tesselation Very High, Vsync Off, Advanced PhysX off. Turning off SSAA stops GPU choking and choppy video midway through benchmark.

Results
No recording: 53 fps.

18-18-18-4: 49 fps, resulting bitrate 12.9 Mbps. Some macroblocking if I really look for it. Image somewhat noticeably sharper than 22-22-22-4.

22-22-22-4: 50 fps, resulting bitrate 7.7 Mbps. Macroblocking but not so pronounced. Image quality not discernible from 23-23-23-4.

23-23-23-4: 50 fps, resulting bitrate 6.5 Mbps. Macroblocking but not so pronounced, Image quality a bit better than 25-25-25-4.

Instead of modifying those settings, I would actually just adust your min and max QP instead. bring down the max for less macro blocking (I found max QP 41 rids the IDR from blurring at lower bitrates) but shouldnt affect bitrate to much unless you are doing some serious scene changes or movement.

 

[ufster]

Instead of modifying those settings, I would actually just adust your min and max QP instead. bring down the max for less macro blocking (I found max QP 41 rids the IDR from blurring at lower bitrates) but shouldnt affect bitrate to much unless you are doing some serious scene changes or movement.

You said before that QP 0 is uncompressed. Is it possible to use an uncompressed video of a benchmark to compare to compressed runs with different settings using freely available video analysis tools like Moscow State University's Video Quality Measurement Tool? I can't tell from looking at two sources unless one is clearly inferior, if the video quality is same. What is the best (resource wise) way to record uncompressed video using OBS?

The problem is if I record the same exact benchmark with different settings, each run will be slightly different (in terms of fps dips, frame times etc.) than the previous one which might make the comparison invalid as the frames won't be identical. Now, my question is can OBS record from an uncompressed video file being played on a media player for exact comparison purposes? If so are there any caveats I should be looking for?

CQP mode is controlled for quality*, not bitrate -- you can't directly change the bitrate. However, changing the quality-speed preset will cause the bitrate to change as the faster presets will need more bitrate to meet the parameter given.

* It's not really directly quality, but the way the encoding works a lower QP will be closer to the original image.

Edit: A difference from 53 to 50 fps is probably about expected with any setting though, as the capture itself tends to cause some slowdown. The encoder is dedicated hardware and doesn't have much effect on the rest of the card on its own, beyond being another thing competing for bandwidth -- which is probably part of why SSAA makes it hurt, SSAA is extremely bandwidth intensive.

Ok, I'll make sure to avoid it in the future. BTW, can you also take a look at the question I directed at dping? I am trying to achieve some sort of benchmarking method that is not dependent on my poor eyes.