OBS Latest News

A new update is released and therefore a new progress report. The story of version 23 involves a whole lot of research and a whole lot of development.

Crowdfunding

Crowdfunding is something I now realize we should have done a long time ago. There's no reason why we shouldn't be pursuing this. As the project grows, and as more contributors come on board, I want to make sure that we can guarantee a future not just for myself but for the project and as many contributors as we possibly can.

After much discussion and looking at existing open source projects, we decided to create both a Patreon and Open Collective. Our goal is so we can ensure that not only can the project continue operating, but also have the ability to grow. Personally speaking, I want to ensure that not only can I work for the users, but that I can delegate important tasks to other contributors with experience working on core code and actually be able to pay them for doing so.

See the announcement blog post by Ben (dodgepong) for more details.

Development of browser-based widgets in 23.0

The goal of browser widgets was so that we could integrate services such as Twitch and allow the ability to display things like the user's chat directly within the program. In 22.0, I sort of had CEF-based Qt widgets working, but they had a number of issues that hadn't been resolved yet, so I put it aside for the time being and made the 22.0 release. However for 23.0, I wanted to get it finished, and get it right.

For those of you who aren't programmers, CEF is the Chromium Embedded Framework, an awesome library we use to allow us to use Chrome within OBS for things like the browser source. It's also what Spotify uses for their entire desktop app. Because of CEF, we were able to implement browser as UI with no extra resource cost than what OBS was already using when they had browser sources loaded.

High-DPI insanity

The first issue was DPI scaling support. The browser widget would not display correctly on monitors that had high-DPI scaling enabled. Eventually I thought I had thought found the fix: the browser subprocess needed to have high-DPI mode enabled as well as OBS. I made the commit eea74ff6, which I thought worked. However, another person on the OBS team discovered that when they had multiple monitors with different scaling, it would break when moving it across monitors! After what seemed like an entire week just investigating DPI issues and digging deep in to CEF/Chromium code, I discovered a special new API that Microsoft had added recently in Windows 10, which fixed it. The mystery was solved in commit 8521b2.

What an unprecedented amount of annoyance, and it was all just due to high-DPI scaling that the user can set on one or more of their monitors.

Cookies and Profiles

I implemented the ability to log in to your Twitch account, and eventually had Twitch chat working as a dockable panel as well. However, I soon realized that users may want different accounts across profiles; so I had to create a "cookie manager" specific to each profile, so that if a user switches from profile A with one account to profile B with another account, the chat windows/etc will appropriately go to that account's channel. That required storing cookies for each profile in a discrete and separate location from each other. It was a pain, but CEF provided all the tools to make my plans happen.

BTTV/FFZ support for Twitch Chat

Naturally, when I created the chat, I quickly realized there was another problem: too many users use BTTV and/or FFZ extensions to add custom chat features, such as extra emojis, animated emojis, or other quality-of-life features that many people have come to love. It's basically a must-have for Twitch chat these days, whether we like it or not. So, with the help of web pros on the OBS team, I implemented custom javascript that injected them into Twitch chat.

Popups and Other Unexpected Issues

When implementing service integration, I started with Twitch, which was the easiest service to implement. Mixer was similarly easy, but along the way, I discovered another annoyance and delay. When Mixer was acquired by Microsoft, they understandably started adding Microsoft features: one of which was logging in via your Microsoft account. However, when you log in with your Microsoft account on the OAuth login page, it creates a popup that starts from about:blank URL and is controlled via Mixer javascript to redirect it to the Microsoft login. I thought I had custom popup whitelisting working, but I then discovered I had to completely defer all whitelisted popups entirely to CEF and let it manage the popups itself, just to get that login working. It was a learning process.

Mac/Linux Browser Issues

Mac and Linux are another story entirely. I tried getting browser widgets working on macOS, but had a lot of crashes, so I had to put off Mac support for widgets for the 23.0 patch. I discovered what the issue was only a week ago, as well as the source behind many other crashes, but it still needs a lot of work before releasing. It'll have to wait until 23.1 or a patch after because of that. So in the future, macOS will finally get a little loving and some stability fixes.

Linux has also been pretty neglected. I plan on getting the browser source finally working on Linux, as well as browser widgets.

Service Integration

Which leads to service integration: when going in to Settings or Auto-Configuration, users can now simply connect their account, log in to their service and just use it right then and there, without having to search around for their stream key. We were going to get YouTube integrated, but its API is quite a bit more complex, so we put it on hold for now to get out a solid release first. Other services are such as Facebook, Restream.io, and others will also be coming. We may also add an external API so plugins can add their own integration separately in the future.

Note that if for whatever reason you'd prefer to use a stream key, you still can. It's completely optional for services that support stream keys.

NVENC Improvements

Originally, we used FFmpeg's implementation of NVENC to save time. It was less than a few hundred lines to implement, and like x264, it only required the raw frames on system RAM. However, I knew that if I implemented it myself and revamped the backend to where we could just give encoders textures directly, it would improve performance. The reason we didn't was behind the complexity of supporting Windows 7 as well. NVIDIA had contacted me asking me about it, and we talked back and forth on the matter. After discussions, I came up with a pretty simple plan: just forget Windows 7. If the user is on Windows 7, just simply fall back to the older version! It saved a lot of time, though not as much time as I'd hoped.

Multi-threading is very difficult to do right

I started off simple to get an initial implementation going: having the encoder the graphics thread (which is normally for rendering), but if either rendering or encoding lagged, it would cause a cascade of subsequent lag. My hope was that the encode call wouldn't stall, but unfortunately it turned out that it can stall, so the only solution was to separate the encoding to another thread, like we already did with software encoders. I had to implement texture sharing in commit b64d7d7. This allowed the ability to not only share textures (like we did for game capture), but also lock textures between multiple threads and graphics contexts to ensure frame synchronization.

After a lot of trial and error, I finally came up with a good threaded implementation in the libobs backend, which I implemented in commit 93ba6e7. It operates on a circular texture queue buffer of a few textures, and I was able to make a specific optimization where if an encoder that uses RAM data is not simultaneously active (e.g. x264), then I can just swap the NV12 texture directly in to the queue instead of having to do an extra texture copy. Finally, after painfully laying all that groundwork for texture-based encoding support in the backend, it was time to finalize my new custom implementation of NVENC, which was accomplished in commit ed0c7bc.

So needless to say, I am very happy with how I was able to implement it as well as the optimizations I was able to come up with. It was pretty fun.

Performance Benefits

The performance benefits of the new NVENC are pretty significant. Before, the process looked like this:

• OBS renders a frame
• OBS transfers that texture from GPU to RAM like it would for any other encoder
• FFmpeg NVENC uploads it to the GPU
• FFmpeg NVENC encodes it

Now, it looks like this:

• OBS renders a frame
• NVENC encodes it

This is not just a performance improvement of OBS, but also reduces the impact of OBS on any game you're playing while using NVENC. It's a must-have for anyone streaming or recording games with a single PC setup.

Pull Requests and New Features by Contributors

Of course, by no means am I the only one working on this project. The OBS developer community continued to produce a ton of great improvements while I was in the trenches working on some of these larger issues, many of which were suggested by you on our ideas page. In fact, I started enlisting the help of fellow contributor DDRBoxman to help maintain the project so I can delegate pull requests and focus on code and project management.

Here are some of the things added by our contributors this patch:

• DDRBoxman worked hard adding support for a long-standing feature request: Decklink output, including keyer support. This is an extremely useful feature for users who use OBS in higher-level productions.
• pkv was certainly busy this patch. He added support for recording multiple audio tracks when using FFmpeg output, which has been a long-standing feature request. He also worked on adding several new audio enhancements, including an audio limiter filter to protect your audio from clipping (requested here) and an audio expander filter for smoother noise gating (requested here). He also added support for adding PSD files as image sources.
• cg2121 continues to be one of our most active contributors. He made it possible to automatically remux recordings upon completion (requested here), so that you don’t have to remember to do it yourself if you don’t want to. He also implemented the ability to do stereo audio balancing in the advanced audio properties dialog, added the “About” dialog to show the license and list contributors, added an image limit to the slideshow source preventing users from using up too much memory if they need to use a lot of images, and finally, he added the ability to resize the OBS canvas to a specific source’s size (requested here), which makes it easier to record applications at exactly their native resolution.
• nleseul added support for batch file remuxing, so that you can remux several files from FLV to MP4 all at once.
• VAAPI support was finally merged in this patch, adding support for hardware encoding on Intel and AMD graphics cards on Linux. This was a long-time collaboration from several members of the OBS developer community, so huge thanks in particular to w23, reboot, kingargyle, kc5nra, GloriousEggroll and kkartaltepe for helping make this feature a reality.
• Andersama made the Stats window dockable, so that you can make it a more permanently visible part of your OBS UI (requested here).
• Dillon has continued his work adding usability improvements to OBS, this time revamping the the way source selections and locations are shown in the OBS preview. This makes it easier to see which source is selected, which source you might select if you click on the preview, and what parts of sources are currently outside the preview. He also added the ability to filter the hotkeys list to make it easier to find the hotkey you are looking to set.

And keep in mind, there are many features still in the works by many contributors. So many great people work on what was just my humble little project, adding features they want, adding little improvements they like, improving performance, and submitting fixes left and right. It's clear this project has become a project made by the people, for the people, and that's exactly how I wanted it to be. I love this community so much, and I can't wait to see what else we can accomplish in the future.

Hindsight, 23.1, 24.0, and beyond

Briefly, I want to share some of my plans for near and long-term future.

First of all I want to note that 23.0 wasn't meant to take as long as it did. I originally wanted to get it out in late November 2018. However we added too many features that were way too big all at once, did too much R&D, ran in to too many unexpected hurdles that we had to leap over, and it ended up taking months to complete. Fortunately, I don't see any major R&D of that level coming up any time soon again this year, so patch pace should greatly speed up once again.

We plan on releasing a 23.1 patch quickly after; the plan is to mostly just merge pull requests and make a quick minor release, hopefully no more than a few weeks. It should have a bunch of smaller new features and minor bug fixes, with at least one or two more services integrated.

My biggest focus near-term is to improve user experience, design, and first-time user onboarding. We need to make it easier for users to start streaming or recording for their first time and making it easier for them to understand and use OBS. The auto-configuration dialog was a big first step for that, but it’s clear there are still things we can do to improve the first-time experience. This year, especially for 24.0, expect to see not just more new features, but also improvements to user experience, onboarding, and design.

I also want to say that I plan to show some love for our recording-only users as well, such as the highly-requested pause recording feature. I also plan to spend some quality time on the macOS and Linux versions. Expect to see all those things in the near future.

For the long-term, there are a near endless number of features we want to implement, and a near endless number of features requested by the users. I myself have some plans I've wanted to see come to fruition for a long time, and finally it's looking like I can make it happen. For requested features, we can't always guarantee that we can get every feature right away, but if you have a feature you really want to see, please make sure to visit out ideas page, and upvote or submit a feature you'd like to see. Even if we can't always implement it right away, it helps us gauge what the majority of you want to see, and helps us prioritize things.

Saying goodbye to a long-time contributor and friend

Michel "Osiris" Snippe, a long-time contributor to the project, passed away unexpectedly February 15th, 2019. He first joined the project near the very beginning; he was always active with helping users and contributing bug fixes, and when the original author of the browser plugin had to move on to other projects a few years ago, Osiris took it upon himself to take over maintaining the plugin for a year or two, during a period of time where I was unable to dedicate time to maintaining it myself due to its complexity. For the longest time, he managed builds for the browser plugin: fixing minor bugs where he could, adding minor features, merging pull requests, and he took care of the plugin best he was able until I was finally able to refactor the project and make it a core plugin in 2018.

He was always kind to people, and always active in the community. No more than a week before we found out we lost him, he was chatting away in our Discord server, helping users with support and goofing around with other contributors, mods, and admins. To lose a friend so suddenly hurts. It's traumatizing. It makes me sad, and it makes me upset.

Thank you Michel, for helping us when we needed it, for being a part of our community, and for being a good friend. We'll all miss you.

Thank you

I want to thank you all for using our humble project, and I want to thank the community for taking it to the next level and making their own features, as well as being patient with me. It started off as a tool I wanted to create for fun out of boredom, and it completely turned not just my life around, but many others as well. It created a wonderful community of friends. The fact that so many of you are becoming successful thanks to the help of our humble tool makes me so happy to see. Thank you so much for using OBS, and I hope we can continue to work on it as long as possible.

Also, thank you for reading this giant blog post. There was a ton of stuff to go over, and I didn't even get to cover everything.

-Lain