Bug Report OBS memory leak randomly taking down my PC

[unfa]

Last month this happened for the first time - OBS was causing memory leaks filling up all my SWAP (16 GB) and all my RAM (32 GB) and taking down my system twice during a live event.

Today I'm doing another stream and the same is happening. It works well for a while, but the suddenly it starts filling up SWAP. I've tried many things and suspected a lot of programs, but it's clearly a problem with OBS.

If I SIGSTOP OBS, the memory stops filling up immediately.
If I kill it - the memory usage goes back to normal instantly.

Howere htop is not reporting this huge memory usage for any process. It shows the total system memory usage going up, but it doesn't show any process using this memory up in particular.

I've recently switched from a Nvidia GTX 1060 to a Radeon RX 580. The reason I switched was that I had flickers and glitches in screen capture in OBS using that card.
And now looks like the Radeon and OBS duo results in memory leaks.
I wonder if the memor yleak is actually from the GPU - since it's not reported to be memoy used by any particular process.

This is my system:

$ screenfetch

██████████████████ ████████ unfa@unfa-desktop
██████████████████ ████████ OS: Manjaro 18.1.3 Juhraya
██████████████████ ████████ Kernel: x86_64 Linux 5.3.13-1-MANJARO
██████████████████ ████████ Uptime: 1h 34m
████████ ████████ Packages: 1885
████████ ████████ ████████ Shell: bash 5.0.11
████████ ████████ ████████ Resolution: 3840x1080
████████ ████████ ████████ DE: KDE 5.64.0 / Plasma 5.17.3
████████ ████████ ████████ WM: KWin
████████ ████████ ████████ GTK Theme: Breath-Dark [GTK2/3]
████████ ████████ ████████ Icon Theme: Adwaita
████████ ████████ ████████ Disk: 3,7T / 4,4T (85%)
████████ ████████ ████████ CPU: AMD Ryzen 7 1700 Eight-Core @ 16x 3GHz
████████ ████████ ████████ GPU: Radeon RX 580 Series (POLARIS10, DRM 3.33.0, 5.3.13-1-MANJARO, LLVM 9.0.0)
RAM: 19368MiB / 32113MiB

$ inxi -F
System: Host: unfa-desktop Kernel: 5.3.13-1-MANJARO x86_64 bits: 64 Desktop: KDE Plasma 5.17.3 Distro: Manjaro Linux
Machine: Type: Desktop Mobo: ASUSTeK model: PRIME B350M-A v: Rev X.0x serial: <root required>
UEFI [Legacy]: American Megatrends v: 4207 date: 12/07/2018
CPU: Topology: 8-Core model: AMD Ryzen 7 1700 bits: 64 type: MT MCP L2 cache: 4096 KiB
Speed: 2720 MHz min/max: 1550/3000 MHz Core speeds (MHz): 1: 2720 2: 2932 3: 3043 4: 3076 5: 2630 6: 2636 7: 2422
8: 2032 9: 2837 10: 2874 11: 3003 12: 3073 13: 3188 14: 3159 15: 2899 16: 2860
Graphics: Device-1: Advanced Micro Devices [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] driver: amdgpu
v: kernel
Display: x11 server: X.Org 1.20.6 driver: amdgpu resolution: 1920x1080~60Hz, 1920x1080~60Hz
OpenGL: renderer: Radeon RX 580 Series (POLARIS10 DRM 3.33.0 5.3.13-1-MANJARO LLVM 9.0.0) v: 4.5 Mesa 19.2.6
Audio: Device-1: AMD Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] driver: snd_hda_intel
Device-2: Advanced Micro Devices [AMD] Family 17h HD Audio driver: snd_hda_intel
Device-3: Logitech HD Pro Webcam C920 type: USB driver: snd-usb-audio,uvcvideo
Device-4: BEHRINGER UMC202HD 192k type: USB driver: snd-usb-audio
Device-5: AKAI Professional M.I. type: USB driver: snd-usb-audio
Sound Server: ALSA v: k5.3.13-1-MANJARO
Network: Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet driver: r8169
IF: enp6s0 state: up speed: 1000 Mbps duplex: full mac: 60:45:cb:64:2c:ed
Drives: Local Storage: total: 2.05 TiB used: 1.73 TiB (84.1%)
ID-1: /dev/sda vendor: Western Digital model: WD20EZRZ-00Z5HB0 size: 1.82 TiB
ID-2: /dev/sdb vendor: Transcend model: TS256GMTS400 size: 238.47 GiB
Partition: ID-1: / size: 222.37 GiB used: 175.26 GiB (78.8%) fs: btrfs dev: /dev/sdb4
ID-2: /home size: 222.37 GiB used: 175.26 GiB (78.8%) fs: btrfs dev: /dev/sdb4
ID-3: swap-1 size: 15.62 GiB used: 466.5 MiB (2.9%) fs: swap dev: /dev/sdb2
Sensors: System Temperatures: cpu: 47.8 C mobo: N/A gpu: amdgpu temp: 39 C
Fan Speeds (RPM): cpu: 0 gpu: amdgpu fan: 1087
Info: Processes: 434 Uptime: 1h 35m Memory: 31.36 GiB used: 16.16 GiB (51.5%) Shell: bash inxi: 3.0.37


Does anyone have similar issues?

 

[Tuna]

Potentially relates to this: https://obsproject.com/mantis/view.php?id=1452

Seems to be more Mesa/Linux related rather than OBS.

For the NVIDIA card case - you probably had OpenGL flipping enabled. This could be easily disabled via nvidia-settings application.

 

[unfa]

@Tuna - I'll check that link, thanks!

As fro the flipping issue: I've tried all possible combinations of the flipping/vsync settings in the Nvidia panel and all available driver versions.
None worked properly, so I have sold my Nvidia GPU and got a Radeon - it wasn't a spontaneous action.

I'm now running latest Git stable release: OBS 24.0.3.r198.gc8ab8aec4 (installed through AUR PKBUILD). I've updated my kernel:

$ uname -a
Linux unfa-desktop 5.4.1-2-MANJARO #1 SMP PREEMPT Sat Nov 30 08:57:00 UTC 2019 x86_64 GNU/Linux

I'm now trying to reproduce it - so far so good, but last month it worked firn for an hour and then took my machine down twice in the span of 5 minutes.

 

[unfa]

@Tuna, this issue you've linked seems like exactly the issue I'm having. I've switch back from the Studio Mode and I hope keeping it this way will let me stream without issues.

I hope the Mesa guys can fix this quickly...

 

[LinuxReborn]

People are saying this is a Mesa problem and it probably is; however, if you download the unofficial app image of OBS studio then this memory leak does not happen. Anyone experiencing this issue might want to try the unofficial image seeing how as it stands currently the official OBS is sorta unusable. https://github.com/probonopd/obs-studio/releases/continuous

 

[Prate]

Same thing happening with OBS studio version 25.0.8-1 on Fedora 32 server.
My server running out of memory. I have to reboot server every alternate day
due to this issue.

Please fix this error with OBS or provide solution to fix this memory leak.

 

[jimmylimo]

OBS is using ALL 32gb RAM on my PC. All of a sudden OBS just disappeared. Checked task manager and it showed 31.9gb RAM being used. HELP !

 

[Tuna]

This is Linux support. You are in the wrong forum.