Question / Help Hardware encoder results in weird saturation issues

A

abdulapaydin

New Member
Hello community,

Whenever I record videos in simple mode with any recording quality preset with the AMD hardware encoder my videos are very saturated and look absolutely weird. My drivers are up-to-date so I've got no idea what could make this happen. When I record with the CPU encoder there is no such problem. Also, in advanced mode I can record with the hardware encoder with no problems. Does anybody know what's going on and how to fix it?

Thanks in advance!

Here is a screenshot of the footage: https://prnt.sc/g4qm4d

EDIT:
Now I've reseted my OBS, and I get a pop-up when recording in Advanced mode (which was fine previously) that says: "Starting the output failed. Please check the log for details. Note: If you are using the NVENC or AMD encoders, make sure your video drivers are up to date."

I AM using the AMD encoder and my drivers are up to date as well, something weird is going on. Could it be that OBS isn't detecting my GPU? And how could I fix this?

New log file: https://gist.github.com/d27838e8a87dd575b508d93bfaf3456a
20:26:02.179: Warning: OBS is already running! (I closed the 2nd instance of OBS and same issue)

EDIT 2:
Suslik V said:
Take a look at this thread too: https://obsproject.com/forum/resour...-framework-encoder-plugin-for-obs-studio.427/
Click to expand...

Okay, so this definitly fixed the output issue! I can record in simple mode now without any issues, HOWEVER now my OBS crashes when I record in advanced mode with the h265 codec! What the heck is going on..

Log: https://gist.github.com/02e631e169877f26a24ec6a806b98c2c
 
Last edited:
K

Kurtalicious

Member
Bare with me my AMF is a little fuzzy but I have used it quite a bit and I have moved to Nvenc encoding.

I'm not 100% on a solution but I had something similar happen when I played with the color format. Settings > Advanced > Video > color format. I believe you are using the preferred color format NV12. Also, play with color space and color range settings. I've never had an issue with NV12, 709, partial but you may want to try other color formats. I would set color space/range to default 709/partial and try different color formats.

The other thing that sticks out to me is "Usage: Transcoding". If I remember correctly this field enables a list of presets depending on what option you have selected. I would select something more appropriate like Recording (I believe recording is an option). I think when I used AMF for recording I actually left this option blank so that I could select my own custom settings.

I hope this helps. I'm not an expert like some of the great mods/admins around here but believe the color format is more likely to solve the problem.
 
A

abdulapaydin

New Member
Kurtalicious said:
Bare with me my AMF is a little fuzzy but I have used it quite a bit and I have moved to Nvenc encoding.

I'm not 100% on a solution but I had something similar happen when I played with the color format. Settings > Advanced > Video > color format. I believe you are using the preferred color format NV12. Also, play with color space and color range settings. I've never had an issue with NV12, 709, partial but you may want to try other color formats. I would set color space/range to default 709/partial and try different color formats.

The other thing that sticks out to me is "Usage: Transcoding". If I remember correctly this field enables a list of presets depending on what option you have selected. I would select something more appropriate like Recording (I believe recording is an option). I think when I used AMF for recording I actually left this option blank so that I could select my own custom settings.

I hope this helps. I'm not an expert like some of the great mods/admins around here but believe the color format is more likely to solve the problem.
Click to expand...

Hmm, interesting. I've tried every possible color format but none of them fixed my issue. I don't get what you mean by "Usage: Transcoding", but I don't think this is a settings problem, I think it has something to do the way my AMD settings are perhabs? Thanks for your reply anyway, I really appreciate your help.

Now I've reseted my OBS, and I get a pop-up when recording in Advanced mode (which was fine previously) that says: "Starting the output failed. Please check the log for details. Note: If you are using the NVENC or AMD encoders, make sure your video drivers are up to date."

I AM using the AMD encoder and my drivers are up to date as well, something weird is going on. Could it be that OBS isn't detecting my GPU? And how could I fix this?
 
K

Kurtalicious

Member
You are running two instances of obs. Check your task manager. Close all obs programs and reopen obs.

If that doesn't fix it then there is a setting that is not appropriate which is causing the output failed.

Last time I remember having this error it resulted in me having to reinstall my drivers/obs. But first close the obs programs, reopen, attempt to stream/record, and adjust settings.

I'm sorry about the confusion with the usage/preset: transcoding thing. Under the streaming tab toward the bottom in advanced mode there should be a field that allows you to select master or expert which will allow you to see the transcoding preset.

Furthermore, I wouldn't use simple mode at all because it's too restrictive. You kind of need the advanced mode so you can make the proper adjustments.
 
A

abdulapaydin

New Member
Kurtalicious said:
You are running two instances of obs. Check your task manager. Close all obs programs and reopen obs.

If that doesn't fix it then there is a setting that is not appropriate which is causing the output failed.

Last time I remember having this error it resulted in me having to reinstall my drivers/obs. But first close the obs programs, reopen, attempt to stream/record, and adjust settings.

I'm sorry about the confusion with the usage/preset: transcoding thing. Under the streaming tab toward the bottom in advanced mode there should be a field that allows you to select master or expert which will allow you to see the transcoding preset.

Furthermore, I wouldn't use simple mode at all because it's too restrictive. You kind of need the advanced mode so you can make the proper adjustments.
Click to expand...

Close all obs programs and reopen obs.

Yeah, I noticed OBS was running twice however after closing both instances it still didn't fix anything.

If that doesn't fix it then there is a setting that is not appropriate which is causing the output failed.

Last time I remember having this error it resulted in me having to reinstall my drivers/obs. But first close the obs programs, reopen, attempt to stream/record, and adjust settings.

I'm sorry about the confusion with the usage/preset: transcoding thing. Under the streaming tab toward the bottom in advanced mode there should be a field that allows you to select master or expert which will allow you to see the transcoding preset.

I downloaded a plugin for the new Radeon update that came out a few days ago which fixed the output issue! Thanks to @Suslik V .

Furthermore, I wouldn't use simple mode at all because it's too restrictive. You kind of need the advanced mode so you can make the proper adjustments.

I can't manage to find the right bitrate in advanced mode, because sometimes I want to record in 720p60, sometimes in 1080p30 and sometimes in 1080p60 and knowing what bitrate makes all three options look the same is kind of hard. Also, in FPS shooters I found out that the bitrate should be set higher as well than let's say just recording my desktop, all in all I find the quality in simple mode very nice.

Though I agree with you that I should get into advanced mode and I'll see what I can do to find the right bitrates.

Also, advanced mode now makes my OBS crash, but I'm sure someone will help me out. Thanks once again for the help.
 
K

Kurtalicious

Member
For optimal sharpness and performance I tend to use the following as far as bitrate goes:

720p30 @ 7500-10000
720p60 @ 10000-15000
1080p30 @ 15000-25000
1080p60 @ 30000-50000

Also, I don't remember there being a bitrate option when using constant frame rate or CQP. Play with your advanced settings more. It can be frustrating and will take time to find the right settings. Also, what works for me might not work for you. Almost everyone's settings are different.
 
You must log in or register to reply here.
Top