OBS Virtual Camera not showing/Failed to start solution

[suiba]

I did have to go to a wild ride to solve this, every post/person/guide that tries to help solving did not help me fix it, at the end i managed, although i am not certain what the root of all the problems was.
I asked gpt to make a step by step of what worked for me at least if it can help anyone.

Introduction​

If you’ve installed OBS Studio on Fedora but the virtual camera option is missing or you receive errors like "Failed to start virtual camera," you might be facing an issue related to the v4l2loopback kernel module and Secure Boot. This guide will walk you through installing, signing, and loading the v4l2loopback module, which is required for OBS Studio's virtual camera feature. We'll also address issues related to Secure Boot and how to handle them.

---

Prerequisites​

• Fedora 34 or later (tested on Fedora 40)
• OBS Studio installed (dnf install obs-studio)
• Admin (root) access

---

Step 1: Install RPMFusion and v4l2loopback Module​

1. Enable RPMFusion repository to install v4l2loopback:
   bash
   Copy code
   VERSION=$(lsb_release -r -s)
   sudo dnf install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-${VERSION}.noarch.rpm
2. Install v4l2loopback and its dependencies:
   bash
   Copy code
   sudo dnf install v4l2loopback kmod-v4l2loopback

---

Step 2: Disable Secure Boot Temporarily​

Since Fedora enforces module signing with Secure Boot, you need to temporarily disable Secure Boot to load unsigned kernel modules like v4l2loopback.

1. Check Secure Boot Status:
   bash
   Copy code
   mokutil --sb-state
   If Secure Boot is enabled, proceed to the next step.
2. Reboot and Disable Secure Boot:
   • Reboot your system and enter the BIOS/UEFI settings.
   • Disable Secure Boot (steps vary depending on the system, refer to your motherboard/laptop manual for details).
3. Boot back into Fedora.

---

Step 3: Load the v4l2loopback Module​

1. Load the v4l2loopback module manually:
   bash
   Copy code
   sudo modprobe v4l2loopback devices=1 max_buffers=2 exclusive_caps=1 card_label="VirtualCam"
   If it loads without errors, check OBS Studio to see if the virtual camera option is available. If Secure Boot is disabled, the virtual camera should work now.
2. If the module fails to load and you see errors like "Key was rejected by service," continue with the steps below to properly sign the module and enable Secure Boot again.

---

Step 4: Generate and Sign MOK (Machine Owner Key)​

To re-enable Secure Boot while allowing v4l2loopback to load, you need to sign the module with your own Machine Owner Key (MOK).

1. Install required packages:
   bash
   Copy code
   sudo dnf install mokutil openssl keyutils
2. Create a directory for kernel signing:
   bash
   Copy code
   mkdir ~/kernel-signing && cd ~/kernel-signing
3. Generate MOK keys:
   bash
   Copy code
   openssl req -new -x509 -newkey rsa:2048 -keyout MOK.priv -outform DER -out MOK.pem -nodes -days 36500 -subj "/CN=Your Name/"
4. Import the MOK key to the system:
   bash
   Copy code
   sudo mokutil --import ~/kernel-signing/MOK.pem
5. Reboot and enroll the MOK key:
   • During reboot, you'll be prompted to enroll the key.
   • Select Enroll MOK, use the arrow keys to select the MOK, and enter the password you set earlier.

---

Step 5: Sign the v4l2loopback Kernel Module​

Once Secure Boot is enabled, you'll need to sign the kernel module so it can load.

1. Locate the v4l2loopback module:
   bash
   Copy code
   modinfo v4l2loopback | grep filename
2. Sign the module:
   bash
   Copy code
   sudo /usr/src/kernels/$(uname -r)/scripts/sign-file sha256 ~/kernel-signing/MOK.priv ~/kernel-signing/MOK.pem $(modinfo -n v4l2loopback)
3. Load the signed module:
   bash
   Copy code
   sudo modprobe v4l2loopback devices=1 max_buffers=2 exclusive_caps=1 card_label="VirtualCam"
4. Check for errors (optional):
   bash
   Copy code
   sudo dmesg | grep v4l2loopback

---

Step 6: Test OBS Studio Virtual Camera​

Now, open OBS Studio, and the virtual camera feature should work without issues. You can test it in applications like Zoom, Google Meet, or any other video conferencing tool by selecting "OBS Virtual Camera" as the input device.

---

Step 7: Re-enable Secure Boot​

1. Reboot and enter the BIOS/UEFI settings.
2. Re-enable Secure Boot.
3. Boot back into Fedora.

Since the kernel module is signed with your MOK key, it should load properly even with Secure Boot enabled.