It seem somewhat fitting that given I’ve always wanted to do graphics “stuff” from the moment i touched a computer, I’d start off my posts with something related to graphics.
In this post, i will talk about a noise removal technique that revolves around “stacking” multiple similar images, extracting the Median value for each pixel and using that value as the value for a “denoised” version of the image.
Most of the code in the solution is just “glue” to load images, transform them into byte arrays in the format [width,height,channel] (RGB order), and the real processing is done with the CalculateMedianImage method, as follows:
public static byte[,,] CalculateMedianImage(List<byte[,,]> imagebuffers,int runlength)
{
byte[,,] retval;
int width = imagebuffers[0].GetLength(0);
int height = imagebuffers[0].GetLength(1);
retval = new byte[width, height, 3];
List compr = new List();
List compg = new List();
List compb = new List();
//
for (int y = 0; y < height - 1; y++)
{
for (int x = 0; x < width - 1; x++)
{
for (int rl = 0; rl < runlength - 1; rl++)
{
compr.Add(imagebuffers[rl][x, y, 0]);
compg.Add(imagebuffers[rl][x, y, 1]);
compb.Add(imagebuffers[rl][x, y, 2]);
}
//Sort the lists and extract the value at the middle
compr.Sort();
retval[x, y, 0] = compr[(int)(compr.Count/2.0)];
compg.Sort();
retval[x, y, 1] = compg[(int)(compg.Count / 2.0)];
compb.Sort();
retval[x, y, 2] = compb[(int)(compb.Count / 2.0)];
//
compr.Clear();compg.Clear();compb.Clear();
}
}
return retval;
}
The code is pretty much self explanatory. It gets the width and length from one of the input buffers, creates a byte array of the right size to return as result and three lists to temporarily store the values to process. It processes the image pixel by pixel, from top left to bottom right. For each pixel, it goes though each image buffer and stores the RGB components at that coordinate in the appropriate list. Then it sorts each component list, and places the value in middle of the list at the current coordinate in the return buffer.
If you look at the output images, you’ll see that the more frames it uses to determine the output, the less noise there will be in the output. Let’s think on why it works.
Your phone, webcamera or digital camera uses something called a CCD (Charge-Coupled Device) to capture images. Its individual elements are arranged as an array, and each captures a single “pixel”. In a perfect world, if neither your subject nor your camera moved, nor the subject changed, you’d get the same value for every “pixel” each time you captured an image. But… this is the real world. For each capture, every element of the CCD will measure a value that is ALWAYS sightly off its “real” value. Because a few more, or less, photons will hit it every time.
What Median Stacking does is simply calculate the value that is closest to most samples and take that as the “real value”. A side effect of the technique is that, over long sequences, as it uses the most frequent values, it will “eliminate” spurious object/motion, like a car moving along a road. On the other hand, subject like leaves on a windy day or moving water will generate so much variance that it will render the output unusable.
You might ask, why not just average the values, ie, use the mean value instead of the median? Two reasons. The mean value might not be in your recorded values. For example, the average of [1,3,3] is 2.33, which would round to 2, and 2 is not in our recorded values. The other reason is that, if the “outliers” are sufficiently far from the median, the mean will skew terribly. For example, given the recorded values of [100,100,128,128,129,130,131] the mean will be 120, when we can clearly see that the “real value” should be somewhere in the 128-131 range with 100 being outliers.
As a side note, over a large number of samples (images), the Law of large numbers tells us that the mean will converge to a true mean. The keyword is large. The benefits of using the mean value are vastly outweighed when taking into account the larger amount of samples needed and corresponding processing time.
But, as an exercise, you can add a CalculateMeanImage method that implements the mean, and use the CalculateDiff method to get the difference between the mean and the median outputs for (several) sample lengths.
As the saying goes, hope this was useful, or at least fun, and as Dave Plummer says, in the meantime and in between time, cya next time…