Monday, June 23, 2008
The Battle Between Noise & Sharpness
Shooting at high ISOs and tweaking sharpness in an image can introduce excessive noise. Balancing these two aesthetic elements is an art.
Software For Noise Control
Noise and sharpness have a yin-yang kind of relationship; you can't have one without the other. Nik Software and PictureCode offer comprehensive programs with simplified processing. PictureCode's Noise Ninja and Nik's Dfine 2.0 and Sharpener Pro 2.0 are the pro's alternatives to Photoshop for working with noise and sharpness.
Noise Inherent. "Noise is a general phenomena that exists in any analog-to-digital conversion process," says Josh Haftel, Product Manager at Nik. "In a digital image, random elements that occur due to the nature of semiconductors result in variation of lightness or luminosity, or of color. Luminance noise is typically the most evident, while chrominance noise, or color noise, is most likely to be identified as being an artifact of digital photography. Luminance noise is something that can be accepted, more or less, especially since luminous noise has a tendency to fade into photographic details."
Digital cameras also are inherently noisy in their design. The more pixels that are put onto a camera's image sensor, the less surface there is in any one pixel to collect enough light to give an accurate reading. The sensor, with its attendant Bayer pattern overlay, converts photons into RGB digital values, but when higher ISOs are utilized in a situation where there's less available light hitting the sensor, the camera increases the amplification of that reduced signal. That, in turn, leads to more noise in the images.
"Each of the pixels in the camera is trying to make an accurate reading of that color at that particular pixel site," explains Fernando Zapata, Senior Programmer for Noise Ninja, "and the less light you have hitting that spot, the less accurate that measurement is. The error in that measurement is the noise that you see in the image, so instead of getting the true color of that pixel, you're getting something else; and the more samples you have, the more accurate the reading will be. That's why you see higher amounts of noise at higher ISO settings, where less light is hitting the sensor, than at lower ISO settings where more light hits each sensor. You also have more noise in the shadows than you have in the highlights because there's less light in the shadows."
Filling In The Gaps. According to Jim Christian, creator and founder of Noise Ninja, Noise Ninja uses a form of wavelets to decompose the image into multiple spatial frequencies. This essentially means that the Noise Ninja algorithm breaks down the fine and coarse features of an image and then analyzes it to find out what's noise and what's detail. It's then able to modify the pixel locations in these decomposed bands and add little changes to the digital values, putting it back together again with reduced noise.
"Think of it as sort of repeated blurring of the image at different resolutions," explains Christian. "So I can go up to the fine frequency band and see the fine frequencies lacking low coarser features and so on. That gives us a little bit more flexibility and gives us a better sense of what's noise and what isn't. We can break it down by chroma, luminance and spatial frequency. And for each coordinate in that space, we can assign an estimate of what kind of noise we can expect to see for that particular camera and a particular brightness chroma, and spatial frequency of how much noise we should see."
Sharpening and noise reduction are similar in their efficacy, but are diametrically opposed. Sharpening at an abstract level is blurring a region around the pixel and comparing pixel value to the blurred value, then trying to amplify edges. Whereas with noise reduction, you're doing the opposite—you're comparing a pixel to the neighborhood of pixels around it and trying to make them similar. So sharpening is trying to push them apart, while noise reduction is trying to average things out.
"Sharpness is defined as the differentiation between one edge and how fast it transitions from one object to another," Haftel notes, "It's the same way that the human visual system works. We focus on sharpness, essentially ensuring that there's a well-defined edge between the two of those objects. Sharpening and noise reduction are opposites of each other. Noise reduction is trying to reduce the differentiation between objects because noise is simply the random unwanted differentiation between one pixel site and another, whereas sharpness increases the differentiation between objects."
How It Works. The Color Ranges method of Dfine 2.0, the Control Points of Dfine 2.0 and the Advanced Tab of Sharpener Pro 2.0 use Nik's U Point technology to distinguish different objects in an image with a more natural method than through the use of selections or masks. Users identify the object or area with a control point or through color range and then are able to quickly apply different amounts of noise reduction or sharpening.
Says Haftel, "Dfine 2.0 uses a wavelet-based algorithm to differentiate between noise and photographic detail by transforming an image from a spatial representation into a frequency-based representation. By focusing on reducing smaller frequencies in this other representation, which typically are noise, photographic details can be maintained. By then enabling the user to override the end results with the easy-to-use selective elements, like U Point, Dfine can offer the best of both worlds—best-in-class noise reduction with the added advantage of a human who can tell the difference between areas that need noise reduction and areas that don't."
Adds Haftel, "Nik Sharpener Pro 2.0, on the other hand, utilizes a unique Autoscan feature that analyzes the image to identify edges and problem areas, such as areas that could produce moiré patterns, halos or other artifacts. After analyzing the image, Sharpener Pro takes the input from the user, such as printer type, print size, viewing distance and paper type, and applies a proprietary sharpening algorithm to the image to ensure optimal sharpening, unique to each image. In this way, Sharpener Pro takes a holistic approach to image sharpening—taking the image, output device and viewer into consideration for the final result."
Experimentation aside, it's not possible for us to completely eliminate noise, nor would we want to. Every incremental change in noise reduction also reduces sharpness. There are images that definitely benefit from both sharpening and noise reduction, but the suggestion from Haftel is to operate selectively, applying tailored and localized manipulation to the areas that benefit the most.
"We try to educate people on that balance of looking for those things that are distracting," Haftel continues. "Don't just look for anything that could be noise; focus only on objects or areas where noise becomes a distraction, such as in an area that should otherwise be smooth, like the sky or an out-of-focus background. The most important thing, we believe, is to reduce chrominance noise. Chrominance noise is the telltale element of a digital file. When we see those red and green and blue splotches that appear in random, haphazard and unexpected areas, you can't make the mental connection between those color artifacts and the image itself, and your mind kind of says, 'Hey, that's unnatural.' And that's the point when you start eroding away the suspension of disbelief that we ask for when looking at photographs and art in general.
"Ultimately, it comes down to perception," Haftel concludes. "A computer can't differentiate by itself the difference between a noise structure and, for instance, the bark on a tree. It does require some human interaction or intervention to help it differentiate between different objects. And that's why we spend a lot of time and effort integrating unique tools and functionality that help the user balance noise reduction and sharpness."