7 ■
IEEE CIRCUITS & DEVICES MAGAZINE
■
MAY/JUNE 2005
developments and future directions in
this area. We begin with a brief
description of a typical digital imaging
system pipeline. We also discuss
image-sensor operation and describe
the most popular CMOS image-sensor
architectures. We note the main non-
idealities that limit CMOS image sen-
sor performance, and specify several
key performance measures. One of the
most important advantages of CMOS
image sensors over CCDs is the ability
to integrate sensing with analog and
digital processing down to the pixel
level. Finally, we focus on recent devel-
opments and future research directions that are enabled by
pixel-level processing, the applications of which promise to
further improve CMOS image sensor performance and broaden
their applicability beyond current markets.
IMAGING SYSTEM PIPELINE
An image sensor is one of the main building blocks in a digital
imaging system such as a digital still or video camera. Figure
1 depicts a simplified block diagram of an imaging-system
architecture. First, the scene is focused on the image sensor
using the imaging optics. An image sensor comprising a two-
dimensional array of pixels converts the light incident at its
surface into an array of electrical signals.
To perform color imaging, a color-filter-
array (CFA) is typically deposited in a cer-
tain pattern on top of the image sensor
pixel array (see Figure 2 for a typical red-
green-green-blue Bayer CFA). Using such
a filter, each pixel produces a signal corre-
sponding to only one of the three colors,
e.g., red, green, or blue. The analog pixel
data (i.e., the electrical signals) are read
out of the image sensor and digitized by
an analog-to-digital converter (ADC). To
produce a full color image, i.e., one with
red, green and blue color values for each
pixel, a spatial interpolation operation
known as demosaicking is used. Further
digital-signal processing is used to perform white balancing
and color correction as well as to diminish the adverse effects
of faulty pixels and imperfect optics. Finally, the image is
compressed and stored in memory. Other processing and con-
trol operations are also included for performing auto-focus,
auto-exposure, and general camera control.
Each component of an imaging system plays a role in
determining its overall performance. Simulations [1] and
experience, however, show that it is the image sensor that
often sets the ultimate performance limit. As a result, there
has been much work on improving image sensor performance
through technology and architecture enhancements as dis-
cussed in subsequent sections.
IMAGE-SENSOR ARCHITECTURES
An area image sensor consists of an array of pixels, each con-
taining a photodetector that converts incident light into pho-
tocurrent and some of the readout circuits needed to convert
the photocurrent into electric charge or voltage and to read it
off the array. The percentage of area occupied by the photode-
tector in a pixel is known as fill factor. The rest of the readout
circuits are located at the periphery of the array and are mul-
tiplexed by the pixels. Array sizes can be as large as tens of
megapixels for high-end applications, while individual pixel
sizes can be as small 2
×
2
µ
m. A microlens array is typically
deposited on top of the pixel array to increase the amount of
light incident on each photodetector.
Figure 3 is a scanning electron micro-
scope (SEM) photograph of a CMOS
image sensor showing the color filter and
microlens layers on top of the pixel array.
The earliest solid-state image sensors
were the bipolar and MOS photodiode
arrays developed by Westinghouse, IBM,
Plessy, and Fairchild in the late 1960s [2].
Invented in 1970 as an analog memory
device, CCDs quickly became the domi-
nant image sensor technology. Although
several MOS image sensors were reported
in the early 1980s, today’s CMOS image
sensors are based on work done starting
around the mid 1980s at VLSI Vision Ltd
and the Jet Propulsion Laboratory. Up until the early 1990s,
the passive pixel sensor (PPS) was the CMOS image sensor
technology of choice [3]. The feature sizes of the available
CMOS technologies were too large to accommodate more
than the single transistor and three interconnect lines in a
PPS pixel. PPS devices, however, had much lower perfor-
mance than CCDs, which limited their applicability to low-end
machine-vision applications. In the early 1990s, work began
on the modern CMOS active pixel sensor (APS), conceived
originally in 1968 [4], [5]. It was quickly realized that adding
an amplifier to each pixel significantly increases sensor speed
and improves its signal-to-noise ratio (SNR), thus overcoming
the shortcomings of PPS. CMOS technology feature sizes,
1. The imaging system pipeline.
