# [ Filter Request ] Remove Webcam Film Grain



## calvinvalerian (Apr 25, 2020)

My Logitech C922 is grainy and I want to remove it like this Adobe Photoshop filter; Filter>Noise>Reduce Noise.

Like this:
https://www.youtube.com/watch?v=JE1gvdoe_6o

I lack financial resources and I think using a little more of my Intel i7 CPU (4 cores 8 threads) is the best I can do.


----------



## dodgepong (Apr 25, 2020)

Usually the solution to grainy video is to improve your lighting conditions.

I'm not sure how good a live video denoiser could be in practice. Applying denoising to a single image is one thing, but applying denoising to 30 1080p images per second is another matter entirely.


----------



## calvinvalerian (Apr 26, 2020)

dodgepong said:


> Usually the solution to grainy video is to improve your lighting conditions.


I'll try this suggestion as I was already planning on doing it. I really hope it's just lighting problem and not that my C922 was designed with noise.



dodgepong said:


> I'm not sure how good a live video denoiser could be in practice. Applying denoising to a single image is one thing, but applying denoising to 30 1080p images per second is another matter entirely.


Since COVID-19 pandemic everybody must stay home for quite a long time and I guess many more people will want to become youtuber as they don't have anything else to do right now, and adding denoising will definitely help since everybody's financial condition is at stagnant and buying high-end cameras might not be an option.


----------



## dodgepong (Apr 26, 2020)

The existence of COVID-19 does not make people's computers run faster.


----------



## Shraeder (Jun 4, 2021)

I know this thread is ages old by this point, but it would be nearly impossible to implement a feature like this for the following reasons:

1.) You cannot increase the resolution or remove noise from a still image or video without introducing distortion (or unintended artifacts). Improvements made to AI and video processing has gotten good, albeit still not perfect. Even Photoshop's effects aren't perfect on still images and can produce artifacts from time to time.

2.) It would require a significant amount of system resources to upscale or even add post processing to every single video frame as it's being streamed or recorded. That kind of feature is always at the bleeding edge of technology. It's one of the main reasons video games require some of the most powerful hardware in order to get decent frame rates. Every single frame, your graphics card and cpu work together to calculate lighting, geometry, texture, post processing (ie. filters).

3.) assuming you have the proper hardware and software support to do anything remotely useful to improve video quality on the fly, you could afford a better camera that wouldn't require a "denoiser" to begin with. You would need an absolute powerhouse of a GPU to do something like that, and a lot of decent GPUs that can do things like that in Blender to increase picture quality/remove noise cost upwards of $1000 and would have to be paired with a paid software. there's no way any sort of free open source software for recording your screen would offer that, considering that it would be a revolutionary piece of software.

P.S.
A lot of webcams will look grainy in low light conditions because they will automatically adjust their ISO, which is basically the frame time, to capture more light per frame. The upside is that you don't need to have perfect lighting all the time to make out people and objects, but the downside is that the image can look grainy because some pixels of the imaging sensor can be overexposed. The more light you have on yourself, the lower the ISO will be on the camera, thus improving image quality.


----------



## Eki Halkka (Jun 21, 2021)

I've been toying around with the idea of a simple temporal noise reduction filter. I've dabbled with user-defined shaders, but writing proper plugins is beyond my expertise at this point - i'd write the plugin myself if i knew how. But here's some pseudo-code on how it would work, at it's simplest:

Create user-defined parameter "threshold"
Initiate a frame buffer "B"

Process frame
{
Load a video frame "A" from the source
For each pixel, calculate the difference between source "A" and buffer "B"
If the difference is above the threshold, use "A", else use "B"
Write the resulting image to buffer "B" (and output)
}

As far as i can tell, this should be quite easy on the processing resources.

I'd know how to code this as a user-defined shader, if i only knew a way to implement the "B" buffer. Perhaps it's not even possible in a shader. Unfortunately, i have not had as much time to spend on learning this as i would have liked to...

I recreated the idea in After Effects, and it seems to work pretty well when the threshold is set just above the noise level: in the areas of the image that do not move, noise is reduced by averaging multiple frames together, the areas that do move trigger the threshold and are used from the current frame of the source. They still have noise, but it's not really noticeable in moving subjects.

I did the tests using good quality source material (Prores 422 from BMCC and 100 Mbit mp4 from Sony A6300) - my aim is to get broadcast quality greenscreen results in obs*, and reducing the noise before keying greatly improves the semi-transparent areas - even a tiny amount of noise can be visible there. I'm not sure whether this approach would help much with lower quality sources, because the noise threshold is much higher, and raising it might cause visible "trailing" motion artifacts.

* I've written my own keyer as a user-defined shader. It works pretty well already, better than the built-in greenscreen key effect in some ways. Still not done with that though.


----------



## SolarSailor (Jul 2, 2021)

Eki Halkka said:


> I've been toying around with the idea of a simple temporal noise reduction filter.


Exactly the same I've been thinking. I'm not sure I understand the attitudes of people above because temporal denoising isn't terribly hard to implement (there are many implementations on shadertoy to use, there's also hqdn3D which is a workhorse in video players), works pretty well in practice (video players, 3d rendering etc), and doesn't consume lots of processing power at all even on CPU, and the overhead is negligible when running on a GPU. The "just add more light" line of thinking isn't exactly productive because cheap webcams always have noise, even if you drop several times their cost into lighting. (which you should do anyway)
Two main reasons to have it:

Greatly improve the keying quality with a green screen as the slightest amount of noise tends to mess up with the results, even with a good camera
For users on a budget, it's a great way to improve the quality of cheap webcams with only cheap standard three-point lighting
The only obstacle, as you said, is inability to access to previous frames from a shader. This should probably be either posted as a feature/pull request to StreamFX/StreamFX Plus plugins github, or implemented as a separate plugin. Sadly, I'm not experienced enough to write one either.

That said, temporal denoising isn't the only denoise type that would be nice to have. Basic color denoising (within the same buffer) would improve the quality a lot already, and doesn't need access to previous frames. Additional deblocking finter is also an easy and great way to improve the quality of cheap webcams as they all use low-bandwidth USB 2.0 and are using horribly compressed streams; MJPEG artifacts are typically very visible, especially if you apply some color grading or sharpen the image afterwards.

(I'd like to see your keyer as well if you are interested in publishing it, especially if you use hue transfer to reduce the screen spilling!)


----------



## Eki Halkka (Jul 3, 2021)

Yep, to all you wrote.

I've been toying around with spatial denoising too, but not yet with usable results...

My WIP shaders are here, some work, some don't, yet at least (i guess I should get into the whole GitHub thing, but drive it is for now):





						Halsu_shaderfilter - Google Drive
					






					drive.google.com
				




The latest keyer is Halsu_HybridKeyer_V014.

I actually made an overview video of an older version of the keyer, it's missing some features but all the essentials are there:


			https://drive.google.com/file/d/1ohvVTWoP4kGOCNgoF9IXf-Y7DGZDdsS5/view?usp=sharing
		


As far as the spill reduction part goes, I'm doing it in RGB space. I compare the green channel to a user-controlled mix of r and b channels, and if g is higher than the r/b mix, replace green with the r/b mix. Another option would be to use the maximum of r and b, but I like the control over the spill bias one gets with the mixing method. Maybe I'll add a checkbox to use this as an alternative method.

The keyer is also RGB, essentially just the maximum of rb minus g. I do some simple RGB preprocessing to improve the hue of the green before key, but better results could be obtained by rotating the green screen hue in HLS to pure green before keying. I'll probably implement this at some point.


----------



## recognitium (Sep 11, 2021)

@Eki Halkka thanks a lot for sharing your WIP shaders / keyers.  Has anybody translated them to GLSL ?  It would be great to have versions that worked on linux. Also, there is obs-shaderfilter-plus   now, seems little work to adapt it. I have done a little bit of trial and error, starting from zero knowledge on the subject, and haven't  managed to get any productive result.


----------



## SCREAMING_SNAKE_CASE (Dec 20, 2021)

I was able to get temporal denoising working in OBS by capturing my webcam with VLC through DirectShow, applying hqdn3d inside VLC, and piping it through NDI to OBS (which turned out to induce less latency and system load than the VLC video source).

Key takeaways:

hqdn3d works absolute wonders on lower end USB 2.0 webcams which stream terribly compressed and subsampled MJPEG garbage. Due to its temporal nature, it gives the appearance of deblocking and deringing without an actual filter, because of the lowered noise floor. So you can rely on sharpness filter in OBS instead of the trash built into the webcam itself, with much less worrying about compression artifacts becoming visible.
The temporal part of hqdn3d doesn't cause noticeable motion artifacts unless you are dancing before your camera. For a typical talking head scene, temporal denoising greatly improves the result. No excessive softness either.
Most noise seems to be chromatic. Luma denoising can be omitted for negligible quality loss.
Surprisingly, it doesn't hog the system too much at 1920x1080, 30fps.
As the users above mentioned, temporal denoising *MASSIVELY* improved my chroma keying quality. With a $50 chinese webcam from Aliexpress, I got the acceptable key under below average room lighting, with 99% of my hair preserved (something I never thought was possible with that kind of hardware). Huge thanks to @Eki Halkka , the shaders he posted above do an excellent job.

On quite a dated desktop system (i5-4670K @ 3.4GHz, GTX 970) the pipeline I described was using about 10% CPU and 20% GPU for OBS, plus ~16% CPU for VLC, all spread between the cores, running at 30fps and1920x1080 resolution. That's with a LUT, StreamFX color correction filter, @Eki Halkka 's hybrid keyer, and some Sharpness, all of this dumped into OBS virtual webcam output.

Worth mentioning that this is an ad hoc solution that does a lot of unnecessary processing and copying, applies the denoising to all channels, and does this on the CPU. With a native OBS plugin or a shader only applied to chroma, it must be possible to reduce the load further, so I totally agree on the usefulness of such a plugin.

(I don't even want to talk about what it took to beat VLC into submission with its directshow capture latency... ugh)


----------



## DefJeff (Oct 26, 2022)

Eki Halkka said:


> Yep, to all you wrote.
> 
> I've been toying around with spatial denoising too, but not yet with usable results...
> 
> ...


I know this is an old thread, but really wanting to use this with OBS v28 on MacOS. I have downgraded to OBS v27.1 to test this on Mac, but there is just a gray screen when I select the filter. My camera image does not even come through. This seems to be a great tool, I just cannot get it to work. Thanks in advance.


----------



## SolarSailor (Oct 30, 2022)

DefJeff said:


> I have downgraded to OBS v27.1 to test this on Mac, but there is just a gray screen when I select the filter. My camera image does not even come through.


I could be wrong but it probably needs to be rewritten in GLSL to make it work on Mac.


----------



## CurlyBlonde (Nov 21, 2022)

Eki Halkka said:


> Yep, to all you wrote.
> 
> I've been toying around with spatial denoising too, but not yet with usable results...
> 
> ...



Please pardon my incompetence since I'm a novice, how does one apply / use your shaders in obs ? 


> I'm using Version 28.1.2 & I've found Shaderfilter Plus but upon selecting your denoising shader for example it gives this error :
> 
> Could not create the effect due to the following error:
> 
> ...


----------



## cyberjunk (Nov 22, 2022)

Eki Halkka said:


> Yep, to all you wrote.
> 
> I've been toying around with spatial denoising too, but not yet with usable results...
> 
> ...


is there any chance that you might be able to convert your keyer into the new format for the new version of the Shader Filter: ShaderFilter Plus:  https://obsproject.com/forum/threads/obs-shaderfilter-plus.119594/


----------

