AMD HW H.264
The best way to setup OBS is running the Auto-configuration Wizard (tools menu) and apply the settings it gives. Then reastart OBS (this is important) and test it.
Now look inside Help menu. Upload the current log file and click on the Analize button to start troubleshooting common issues if any is presnt.