macOS VideoToolbox Custom Encoders 1.0.0

macOS VideoToolbox Custom Encoders is a third-party plugin for OBS Studio on macOS that adds custom versions of Apple's VideoToolbox H.264 and HEVC hardware encoders.

It exposes additional VideoToolbox options that are not configurable through OBS Studio's built-in encoders:

• RealTime encoding
• Prioritize Encoding Speed Over Quality
• Omit Expected Frame Rate: lets VideoToolbox encode each frame as fast as it can, which leaves more headroom when several outputs share the hardware encoder.
• Custom Lookahead Frame Count on macOS 15 Sequoia or newer
• Submit Frames From a Dedicated Thread: with a configurable queue depth, so an overloaded encoder drops its own frames instead of making OBS skip frames on every output.

You can also add as many additional encoders as you need, each with its own alias and configuration (for example, one tuned for streaming and one for recording).

The plugin also supports OBS Studio's Multitrack Video feature. When enabled, compatible Multitrack Video outputs can automatically use the custom VideoToolbox encoders instead of the standard Apple H.264/HEVC hardware encoders.

The plugin does not replace or modify OBS Studio's built-in encoders. The additional encoders are registered alongside them.




Requirements

• macOS 13 Ventura or newer
• Apple Silicon Mac
• OBS Studio 31.1.0 or newer
• macOS 15 Sequoia or newer for Custom Lookahead Frame Count

Installation

Download the latest .pkg installer from the GitHub Releases page, run the installer, and restart OBS Studio.

After installation, configuration is available under:

Tools → macOS VideoToolbox Encoders…



Each tab of the settings window holds one set of encoders, starting with Default. Click the + tab to add another H.264/HEVC pair with its own alias and settings. New encoders appear in OBS after restarting it. Multitrack Video always uses the Default encoders.

The custom encoders can also be selected manually from Settings → Output:




Diagnostics and updates

• Log encoder timing statistics writes per-encoder timing to the OBS log every 10 seconds. Use it to find out which encoder causes "skipped frames due to encoding lag".
• Check for updates when OBS starts (optional) looks at the project's GitHub releases at most once a day and tells you when a newer version exists. Nothing is downloaded or installed automatically, and no personal data is sent. You can turn it off in the settings window.

Source code



Bug reports

Please report bugs and compatibility issues through the GitHub Issues page:



License

GPL-2.0.


Third-party notice

This is an independent third-party plugin and is not developed, endorsed, or supported by the OBS Project or Apple.


AI usage disclosure:
AI coding tools, primarily Claude and Codex, were used extensively throughout development and generated a significant portion of the code. I designed and directed the plugin's functionality and development process, and performed the integration, debugging, testing, and validation of the resulting implementation.

The plugin has undergone more than 20 hours of real-world testing on a Mac Studio with Apple M4 Max, including live streaming, Multitrack Video streaming, and local recording tests, covering the VideoToolbox encoders and their integration with OBS Studio.