Bug Report Videos made in OBS Studio cannot be used with Windows.Media.Editing APIs due to incorrect encoding

J

johnmsft

New Member
Hey guys,

I used to record videos in OBS Classic and edit them using the built in Windows.Media.Editing APIs. However, once I switched to Studio, all my OBS Studio videos couldn't be processed correctly due to a bug in the audio encoding. I happen to work at Microsoft and got them to look at it and here was their quick and dirty explanation to me of what went wrong.

"In the AudioSpecificConfig in the bitstream, there are 5 bits to signal the audio object type. In this case it is AAC_LC. There is a way to signal the syncExtension (11 bits) type further on. If the syncExtension == 0x2b7, the first 5 bits are used to decipher the extensionAudioObjectType. OBS is setting this to AAC_LC. We expect this to be SBR or PS. Further more, setting the extensionAudioObject type to null, allows for the bitstream to be decoded so perhaps it was a bug to signal extensionAudioObjectType =AAC_LC."

To be more specific, I was using NVENC at the time and recording to .mp4.

On the windows side, they have a workaround for this to help the Windows.Media.Editing APIs function on these videos, but I wouldn't count on that fix being available any time soon. Additionally, this issue may pop up with other types of editing software so it's probably worth looking at. If you like, I can send you the program I was using to do the editing as well as one of the videos from OBS Classic that edited fine and one from OBS Studio that had a problem.

Thanks,
John
 
Suslik V

Suslik V

Active Member
Can you post log-file with recording attempt (nvenc)?

It would be interesting - which codec is used to encode AAC sound on your system. Container and muxing options if any.
 
NalaNosivad

NalaNosivad

Member
johnmsft said:
https://gist.github.com/612695c53e195eff0939e0fb388119de

I don't think I changed any settings at all around audio from the defaults. Let me know if there's anything I can try/do to help with this (or if i misunderstood your previous request).
Click to expand...
Well, it's not using the CoreAudio AAC encoder. Ty following this, and seeing if the AAC tracks produced by the CoreAudio encoder have the same problem.

It'll be interesting to see if it does.

In fact, before you do that, try encoding with the audio set to 96, 128, 160, or 192Kbps to see if the Media Foundation has the same problem, too.

Edit: In fact, the default bitrate seems to be 160Kbps. Meaning, if you haven't changed it from that, that it's Microsoft's Media Foundation AAC encoder that's producing the bad files.
 
J

johnmsft

New Member
I have filed a bug on the Media Foundation AAC encoder team so they can get this worked out. If they come back to me and suggest that there is no bug I'll try a few of your suggestions to test different encoders and bitrates.
 
J

johnmsft

New Member
The Windows devs finished their investigation into this and they found that it looks like Media Foundation is outputting valid information but that something else is writing the information to the file incorrectly. Here's the information I got from the dev. If you have any questions I would be happy to forward them back.

Looking into this bug, it is using our encoder, which puts the audio specific info(that includes AAC extension object type, in our case we don't support an extensions for AAC encoding) on the output media type as MT_USER_DATA, but it is using the FFPMPEG muxer to write out the file.

Some component other than our component is finally writing the audio specific config out to MP4 file. When our MP4 source retrieves it, it has the invalid value set.​
 
J

johnmsft

New Member
The devs I spoke to did not know exactly why we didn't support extensions but they said it was likely that we only had a license to use the core codec and not any extensions.

Regardless, it looks like when OBS writes out the file, in the audio specific config info it lists the base as AAC_LC and the extension also as AAC_LC. Since no extension was used, it really should not be writing that AAC_LC is the extension.
 
J

johnmsft

New Member
Is it really sad? I know next to nothing about codecs and such, but is there some awesome benefit that could be had if only Microsoft supported some extension? I could bring it up at some of our meetings if there was some worthwhile benefit for users.
 
R

RytoEX

Forum Admin
Forum Moderator
Developer
johnmsft said:
On the windows side, they have a workaround for this to help the Windows.Media.Editing APIs function on these videos, but I wouldn't count on that fix being available any time soon.
Click to expand...

Any word on this workaround?


johnmsft said:
I used to record videos in OBS Classic and edit them using the built in Windows.Media.Editing APIs. However, once I switched to Studio, all my OBS Studio videos couldn't be processed correctly due to a bug in the audio encoding. I happen to work at Microsoft and got them to look at it and here was their quick and dirty explanation to me of what went wrong.

"In the AudioSpecificConfig in the bitstream, there are 5 bits to signal the audio object type. In this case it is AAC_LC. There is a way to signal the syncExtension (11 bits) type further on. If the syncExtension == 0x2b7, the first 5 bits are used to decipher the extensionAudioObjectType. OBS is setting this to AAC_LC. We expect this to be SBR or PS. Further more, setting the extensionAudioObject type to null, allows for the bitstream to be decoded so perhaps it was a bug to signal extensionAudioObjectType =AAC_LC."
Click to expand...

johnmsft said:
The Windows devs finished their investigation into this and they found that it looks like Media Foundation is outputting valid information but that something else is writing the information to the file incorrectly. Here's the information I got from the dev. If you have any questions I would be happy to forward them back.

Looking into this bug, it is using our encoder, which puts the audio specific info(that includes AAC extension object type, in our case we don't support an extensions for AAC encoding) on the output media type as MT_USER_DATA, but it is using the FFPMPEG muxer to write out the file.

Some component other than our component is finally writing the audio specific config out to MP4 file. When our MP4 source retrieves it, it has the invalid value set.​
Click to expand...

Emphasis mine. If I'm reading that correctly, the devs are blaming FFmpeg, OBS Studio, or whatever component actually writes the MP4 file. Am I understanding that correctly? Or is there something more to them mentioning that they "don't support an extensions for AAC encoding"?
 
Suslik V

Suslik V

Active Member
Yes, you are. Microsoft follows standards and also defines standard. If ffmpeg influence beats the microsoft, then support of the Windows.Media.Editing APIs software would be stopped and you'll see new software product for video editing from microsoft. If not - nothing changes.
 
R

RytoEX

Forum Admin
Forum Moderator
Developer
Suslik V said:
Yes, you are. Microsoft follows standards and also defines standard. If ffmpeg influence beats the microsoft, then support of the Windows.Media.Editing APIs software would be stopped and you'll see new software product for video editing from microsoft. If not - nothing changes.
Click to expand...

Sorry, I don't understand what you're trying to say. Could you try again?

I am hoping to get a response from @johnmsft or the developers he referenced since it was his post and the words of the Microsoft developers that I'm curious about.
 
Last edited:
You must log in or register to reply here.
Top