Question / Help Video Coding Engine (VCE) doesn't work with OBS

JokerTheJoke

JokerTheJoke

New Member
I can't record with OBS and Video Encoding Engine. When I click on "Start Recording" nothing happens when I try to record in QHD (2560x1440p, native). When I check the "Rescale Output" box and set it to 1920x1080 or lower I am able to click in record but it actualy doesn't record and when I try to stop the recording it just says "Stopping Recording..." and nothing happens anymore and I have to kill OBS' process tree in the Process Explorer. The preview in OBS' window works btw...

It shows all the time, in each resolution, "Encoding overloaded! [...]" while I try to record with VCE but the CPU is used to around 1% and the GPU isn't in use at all, same with the Memory.


No matter which settings I use in the "Recording" tab. It works with the x264 encoder but this is very demanding to the performance...


I attached two logfiles of OBS, one is about the recoding in QHD, my native resolution, and the other is about the downscaled version...


My setup:

Sapphire R9 290 Tri-X OC (1 Ghz)

Intel Xeon i3-1231 v3 (4x 3,4 Ghz with Hyper Threading)

Windows 10 x64

OBS 0.15.2 (newest) x64

Radeon Crimson Edition 16.7.2 Hotfix (newest)



I'm going to also post this in AMD's and Reddit forums, I'll attach a link to it when it's done! ;D
AMD: https://community.amd.com/message/2738557
Reddit: https://www.reddit.com/r/obs/comments/4uvujf/video_coding_engine_vce_doesnt_work_with_obs/



I hope you can help me!
 

Attachments

  • QHD.txt
    10.2 KB · Views: 15
  • downscaled.txt
    13.9 KB · Views: 14
  • QHD.txt
    10.2 KB · Views: 4
Last edited:
Suslik V

Suslik V

Active Member
Resolutions over 1920x1080 for AMD VCE not supported by now. To avoid 'overloaded' message, try to up bitrate x20 times.

This errors:

SetCodecProperty(codecApi, CODECAPI_AVEncVideoEncodeQP, UINT64(qp.Pack(true))) failed, Nicht implementiert (0x80004001)
SetCodecProperty(codecApi, CODECAPI_AVEncVideoEncodeFrameTypeQP, UINT64(qp.Pack(false))) failed, Nicht implementiert (0x80004001)

related to encoder. Can you create new profile in OBS Studio and make new tests? And what settings do you push into the
  • profile
  • Keyframe interval
  • rate control
  • bitrate
  • use custom buffer
  • advanced:
    • use custom max bitrate
    • low latency (disable frame re-ordering)
    • consecutive b-frame count
    • min QP
    • max QP
fields of the AMD VCE?
 
JokerTheJoke

JokerTheJoke

New Member
Suslik V said:
Resolutions over 1920x1080 for AMD VCE not supported by now. To avoid 'overloaded' message, try to up bitrate x20 times.

This errors:

SetCodecProperty(codecApi, CODECAPI_AVEncVideoEncodeQP, UINT64(qp.Pack(true))) failed, Nicht implementiert (0x80004001)
SetCodecProperty(codecApi, CODECAPI_AVEncVideoEncodeFrameTypeQP, UINT64(qp.Pack(false))) failed, Nicht implementiert (0x80004001)

related to encoder. Can you create new profile in OBS Studio and make new tests? And what settings do you push into the
  • profile
  • Keyframe interval
  • rate control
  • bitrate
  • use custom buffer
  • advanced:
    • use custom max bitrate
    • low latency (disable frame re-ordering)
    • consecutive b-frame count
    • min QP
    • max QP
fields of the AMD VCE?
Click to expand...

Here is a screanshot of my settings: https://drive.google.com/file/d/0BzkEKR-RhZVYdDdFeG1OQlJpUk0/view?usp=sharing

I basicly only changed the profile to HIGH and switched the rate control to CQB, all the values has been set by OBS automaticly.

Thanks for the info, I didn't read so far that VCE doesn't support such resolutions... May it will support it someday with a driver update? Don't think so... :/


When I put the variable bitrate to 50k (should be enough) the same happens as with donwnscaling and CQB: it seems to record, warns me still and can't stop recording. At the end of the day it still generated no footage in the specified folder, as always with VCE.
 
Suslik V

Suslik V

Active Member
Then try to put VBR to rate control (or make one more test with 32bit exe of OBS Studio).

50Mbit or so is a right value. Last log says that you had only 2.5Mbit (20 times lower).
 
O

onison13

New Member
basically lots of data, little pipe to go through the rate control controls how much data is allowed to be pushed through the encoder.

setting your rate control that high might create bigger files but should be able to handle the higher res.
 
Suslik V

Suslik V

Active Member
Then create new Profile and new Scene Collection and try to record still image in simple recording mode.
 
JokerTheJoke

JokerTheJoke

New Member
Suslik V said:
Then create new Profile and new Scene Collection and try to record still image in simple recording mode.
Click to expand...
If you mean with unmodified settings:
Yes, this works but it downscales the image to 1280x720.
The x264 encoder works for QHD aswell but as I read in my first post, it uses a lot of power...
 
Suslik V

Suslik V

Active Member
Encoder option? Can you set only Encoder to AMD VCE then click OK? Click recording button. Check result and post current log file.
 
You must log in or register to reply here.
Top