Sensor

Megapixels

Digital camera sensors are made up of millions of tiny wells that collect photons of light. Each of these wells is called a photosite. During an exposure, each photosite will collect a certain number of photons based on the light characteristics of the scene and the length of the exposure. Each photo-site will transmit an electrical signal corresponding to the amount of photons it collected during an exposure.

These electrical signals are converted to luminance values and collected for all of the photosites. This aggregate of luminance values forms the basis of an image. If the sensor has 1500 photosites on one side and 3000 photosites on the other side, it has 1500 x 2000 or 3,000,000 photosites. Since each photosite represents a pixel in the final image, the sensor is said to have 3,000,000 pixels or 3 megapixels.

The number of pixels on a sensor can be directly translated into a print size by dividing the pixels by the pixels per inch (ppi) of the final output or output device. For instance, 1500 x 2000 pixels divided by 300 pixels per inch for printed output equals a print size of 5" x 6.7" at 300 ppi. If the output device is a computer screen which has a resolution of 72 ppi, the image at 100% will be 20.8" x 27.7".

Figure 1 The more megapixels, the larger the native print size. Digital image files can make larger prints than illustrated here by either reducing the print resolution- using 180 ppi instead of 300 ppi for instance, or by upresing. Digitally captured files can be upsampled up to 400% in many cases, as long as the original image is sharp and well exposed to begin with.

Sensor size

Digital sensors come in a wider variety of sizes than we had with film formats. They range from the tiny (5.76 x 4.29 mm) to the large (53.9 x 40.4 mm).

Figure 2 Larger sensors can hold more photosites (or more photosites that are larger). Both larger sensors and larger photosites are an advantage from an image quality standpoint. Another important factor for photographers to consider is whether the sensor is “full frame” or will the sensor size crop the angle of view of the lens. Equally important is the aspect ratio. Some photographers are used to and prefer the 35mm 1.5:1 ratio, while many others are wedded to the 1.33:1 ratio of medium-format. The 1.33:1 ratio fits normal publication page sizes better and is often preferred for that reason. The so-called “crop factor” was an impediment for many photographers in the early days of digital capture. Some have grown to like its advantages while others were relieved when full frame sensors allowed full use of wide-angle lenses. Since digital sensors are not tied to any particular film format and they only need to be compatible with lens coverage, we wonder why there hasn’t been more innovation in aspect ratios. Why wouldn’t we have a 1.33:1 or even a 1.25:1 DSLR sensor that uses the full 36 mm width, i.e. 36x28mm or 36x29mm?

Pixel pitch (relative size of the photosites/pixels)

Increasing the number of megapixels results in higher resolution but if the sensor size remains the same, it results in smaller photosites (pixels). Photosite size is referred to as the pixel pitch. Smaller photosites gather less light, so they have less signal strength. Less signal strength, all other things being equal, results in a less efficient signal to noise ratio, therefore more noise.

This effect is much more pronounced at higher ISO settings, because increasing the ISO setting requires cranking up the amplification of the signal, which, in turn, increases the noise level.

This is why point-and-shoot cameras produce noisy images at high ISO settings, and why the Nikon D3 with its large pixel pitch (8.4 µm) has much lower noise at high ISO settings than a point-and-shoot that might have a pixel pitch as small as 1.7 µm.

CCD chips, commonly used for medium and large format sensors, tend to generate more heat than CMOS sensors: more heat contributes to noise.

Some medium-format backs have active cooling to counteract that effect. CCD chips in cameras used by astronomers are cooled with liquid nitrogen to combat noise realized at the long exposures required for their specialized type of photography.

Pixel size and diffraction

Smaller photosites can also affect image quality due to an optical effect known as diffraction. Diffraction limits image resolution by nullifying the gains of higher megapixel counts when the lens is stopped down. As photosites get smaller, diffraction occurs at a larger f-stop. For instance, a point-and-shoot camera’s resolution can become diffraction limited between f/4.0 and f/5.6, whereas a full frame DSLR with large photosites will not become diffraction limited until between f/16 and f/22.

Keep in mind that cameras with smaller sensors have a higher depth of field at any given f/stop, so diffraction is not quite as limiting as it might appear. Still, the effect is noticeable enough that a review of the Canon D50 (15.1 megapixels) concluded that it required higher quality lenses to realize the same image quality as its predecessor, the Canon D40 (10.1 megapixels).

Medium-format sensors are becoming just as crowded as some higher resolution DSLR sensors. Since medium-format lenses are longer for any given angle of view, achieving great depth of field may result in detail loss at small apertures with high megapixel count sensors. Medium-format camera makers have joined DSLR manufacturers in touting new digital lenses that claim to have higher-resolving power and have wide angle lens designs that aim to send light waves at straighter angles back to ever smaller photo sensor sites.

A tutorial on diffraction issues associated with digital cameras is available at Cambridgeincolor.com. Another important discussion is available on the Luminous-Landscape site. The bottom line is that at a certain point, increasing resolution by adding more photosites without also increasing total sensor size will not add anything to image quality and, in fact, will decrease image quality, especially at small apertures.

Phase One has acknowledged that packing sensors with ever higher numbers of ever smaller photosites does create limitations in terms of ISO and f/stop settings by introducing a “binning” function in their new Sensor+ camera backs. For instance, the Phase P65+ back has 60 million 6 micron pixels. This gives extremely high resolution but at the expense of image quality if pushed much beyond ISO 200 and stopped down much past f/11. Combining 4 photosites into one through a proprietary technology results in a 15 MP sensor with relatively large 12 micron pixels. This function completely changes the character of the sensor, allowing faster image capture, higher ISO settings and likely less diffraction issues.

Sensor type

An interesting note about measuring megapixels is that megapixel counts vary in relation to image quality depending on whether the camera sensor uses color filter array technology, Foveon technology, multi-shot, tri-linear array technology, or beam splitter technology.

Color filter arrays are composed of red, green, and blue (and sometimes other colors) filters that are placed over the sensor photosites. The great majority of digital cameras use color filter array technology, specifically a Bayer-pattern color filter array (shown on the left). A few manufacturers have experimented with other types of arrays, notably Kodak and Sony (shown on the right).You'll notice that the Sony color filter array introduces a fourth color which they call emerald to the array.

	Figure 4 A normal RGB Bayer pattern array.
	Figure 5 The Sony Bayer array with emerald as a fourth color.

A feature of color filter array cameras is that for each of the photosites, only one color is actually measured- which is the color of the filter array over that photosite. The other two color values for that pixel are interpolated based on the measured color values of nearby photosites. This interpolation process is called demosaicing or replacing the mosaic color filter array pattern with continuous tone information. Although demosaiciing works very well, it can created some issues such as color artifacts and moire.

Foveon sensors don't require color filter arrays at all since they record all three colors of light at each photosite. Foveon sensors don't need as many photosites (megapixels) in order to achieve the same apparent image quality at any given output size. The issue with Foveon sensors is that the pixel interpolation required to make a Foveon image the same size as a color filter array camera image is roughly equivalent to the color interpolation that occurs with demosaicing.

Multi-shot cameras require three exposures, one each through red, green, and blue filters. Multi-shot cameras don't require color interpolation so their megapixel counts are understated in terms of image quality when compared to single shot color filter array cameras. The same can be said of tri-linear array cameras. These cameras are called scan-back cameras because they work the same as scanners, a three-part sensor array one each for red, green, and blue sweeps across the sensor area recording all three colors from the scene in the process. It has similar quality to a multishot camera although the means of recording the image is different.

Figure 6 The Foveon sensor captures all three RGB colors at each photosite.

Video

The final technology is the beam-splitter camera. This is in wide use in digital video cameras. The light from the lens passes through a dichroic mirrored prism that splits the light into red, green, and blue beams. Each beam is directed to a small CCD chip. The electrical signals from each CCD chip is combined to create a full color image. The use of three small CCD chips gives these types of video cameras a characteristic wide depth of field.

This contrasts with digital video shot with the comparatively large sensors used in DSLR cameras which can produce a shallow depth-of-field. Shallow depth of field and excellent image quality is leading even some film makers to use cameras such as the Canon 5D II for serious film projects.

Convergence is coming from the other end just as rapidly with the 4K RED One camera system. This camera can shoot 24 12 megapixel frames per second in a raw file format. The sensor at 24.4 x 13.7 mm is approximately the size of a Nikon DX sensor allowing for reasonably shallow depth of field with fast lenses. It has adaptors allowing the use of many standard 35mm format lenses. Its raw file format is ground breaking in several respects. It is adjustable in post production, the same as still digital raw formats, and it has two levels of visually lossless compression which allows practical on-camera recording of such high definition files.

The RED One shares another characteristic with DSLR CMOS cameras that can record video: the rolling shutter effect. Since these cameras read the data off of the sensor line by line and not all at once (like CCD sensors do), moving objects, or camera panning can result in vertical objects that appear to lean. This jiggly effect is more pronounced in some cameras, such as the Nikon D90 than it is in the Canon 5D II or the RED One. To complete the convergence picture, The RED One is capable of producing single frames with high enough resolution to use for magazine covers.

CMOS vs. CCD

Although CCD (Charge Coupled Device) preceded CMOS (Complementary Metal Oxide Semi-Conductor) chips to market, most smaller sensor cameras (up to full-frame 35mm) use CMOS chips. The larger sensors found in medium/large format cameras are currently all CCD technology.

At this point, both technologies can achieve the highest image quality. The Foveon sensor is of the CMOS variety although it's method of collecting light is entirely different from color filter array sensors. The fundamental difference between CMOS and CCD technology is that CMOS integrated circuits have more processing functions on the chip itself.

CCD integrated circuits have the single function of collecting light and converting it to electrical signals. CMOS chips tend to use less power, generate less heat, and provide faster read-out of the electrical signal data. In addition, CMOS sensors can provide additional signal processing such as noise reduction on the chip itself. This explains to some extent why CMOS chipped cameras tend to provide more highly processed, less "raw" data than the CCD chips normally found in medium format backs.

Micro lenses and filters

Figure 7 Digital sensor photosites have significantly more depth than the relatively flat surface of film. The use of micro lenses to focus incoming light from the lens down into the sensor has become a common way to increase sensor efficiency and how to avoid problems caused by the angled light path that commonly occurs with wide-angle lenses. Until recently, most medium-format sensors did not have micro lenses, which is why medium-format backs did not work as well with wide angles, and why the use of shifting lenses often resulted in lens cast (a magenta/green color shift across the image). The use of micro lenses has greatly improved the efficiency of digital sensors while also helping to nullify the effects of squeezing ever more resolution without increasing sensor size. In addition, effective micro lenses can help to alleviate diffraction limits by capturing the light that would have fallen outside the photosites.

Filters in front of the sensor (Low-Pass, Anti-Aliasing, Anti-Moiré):

Digital sensors that use Bayer pattern arrays (which is most of them) are subject to aliasing (jaggies) and moiré (a colorful pattern appearing when a scene contains a repetitive pattern that lines up with the Bayer pattern overlaying the sensor).

Moiré can be alleviated by:

• using higher resolution sensors
• moving the camera closer or farther from the subject
• most reliably by using a filter that creates a one pixel wide blur over the image

Various strategies can be used to increase the effectiveness of this filter in suppressing aliasing and moiré while minimizing detail loss due to image blurring. Use of a filter in front of the sensor is the main reason you need to apply sharpening to digital images.

Quality of the analog to digital converter (ADC) and its read speed

Similar to the race to higher megapixels and the need to keep digital noise at bay, there is tension between the need for higher frame rates and the higher image quality gained by giving the ADC more time to collect data from the sensor.

As usual, manufacturers are working towards doing both. The quality available from medium-format sensors is partly due to their slower frame rates (in some cases by a factor of 10).

The extent to which your photography requires high frame rates will factor into your speed vs. quality equation. The ADC also determines the sampling bit depth, as it converts the analog voltage to luminance levels. The ADC needs to be matched to the dynamic range of the sensor, thus defining the sensor bit depth for a camera.

Sensor bit depth (8-bits, 12-bits, 14-bits, 16-bits)

The trend has been to increase bit depth from the 8-bits available in JPEG capture to 12-bits, 14-bits, and even 16-bits.Higher bit depth has a positive impact on image quality—especially dynamic range. Going from 8-bits to 12-bits results in the greatest gain.

At the time of this writing, no 35 mm DSLR cameras that have 14-bit capture ability clearly show an image quality advantage over 12-bit capture.

Some medium-format sensor makers claim an advantage with 16-bit capture. However, we have never seen a study (other than the manufacturer’s) that shows higher bit depth translates into higher image quality based on 16-bit capture alone. In general, the difference between 14-bit and 16-bit capture would not be visible (to humans anyway) unless a severely steep tone curve was applied to the image (on the order of 6-7 stops).