基于互信息的图像配准

IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 16, NO. 2, APRIL 1997 187

Multimodality Image Registration by

Maximization of Mutual Information

Frederik Maes,* Andr´e Collignon, Dirk Vandermeulen, Guy Marchal, and Paul Suetens, Member, IEEE

Abstract— A new approach to the problem of multimodality

medical image registration is proposed, using a basic concept

from information theory, mutual information (MI), or relative

entropy, as a new matching criterion. The method presented

in this paper applies MI to measure the statistical dependence

of the modalities involved. The accuracy of the MI criterion

is validated for rigid body registration of computed tomog-

raphy (CT), magnetic resonance (MR), and photon emission

tomography (PET) images by comparison with the stereotactic

registration solution, while robustness is evaluated with respect

to implementation issues, such as interpolation and optimization,

and image content, including partial overlap and image degra-

dation. Our results demonstrate that subvoxel accuracy with

respect to the stereotactic reference solution can be achieved

completely automatically and without any prior segmentation,

HE geometric alignment or registration of multimodality

images is a fundamental task in numerous applications in

three-dimensional (3-D) medical image processing. Medical

diagnosis, for instance, often benefits from the complemen-

tarity of the information in images of different modalities.

In radiotherapy planning, dose calculation is based on the

computed tomography (CT) data, while tumor outlining is of-

ten better performed in the corresponding magnetic resonance

(MR) scan. For brain function analysis, MR images provide

anatomical information while functional information may be

*F. Maes is with the Laboratory for Medical Imaging Research,

Katholieke Universiteit Leuven, ESAT/ Radiologie, Universitair Ziekenhuis

Gasthuisberg, Herestraat 49, B-3000 Leuven, Belgium. He is an Aspirant

of the Belgian National Fund for Scientific Research (NFWO) (e-mail:

Frederik.Maes@uz.kuleuven.ac.be).

A. Collingnon, D. Vandermeulen, G. Marchal, and P. Suetens are with the

Laboratory for Medical Imaging Research, Katholieke Universiteit Leuven,

ESAT/Radiologie, Universitair Ziekenhuis Gasthuisberg, Herestraat 49, B-

3000 Leuven, Belgium.

Publisher Item Identifier S 0278-0062(97)02397-5.

obtained from positron emission tomography (PET) images,

as with any external point landmark-based method, while

anatomical point landmark-based methods are usually labor-

intensive and their accuracy depends on the accurate indication

of corresponding landmarks in all modalities. Surface-based

registration requires delineation of corresponding surfaces

in each of the images separately. But surface segmentation

algorithms are generally highly data and application dependent

and surfaces are not easily identified in functional modalities

such as PET. Voxel-based (VSB) registration methods optimize

a functional measuring the similarity of all geometrically cor-

been proposed that optimize some global measure of the

absolute difference between image intensities of corresponding

voxels within overlapping parts or in a region of interest (ROI)

[5], [11], [19], [26]. These criteria all rely on the assumption

that the intensities of the two images are linearly correlated,

which is generally not satisfied in the case of intermodality

registration. Crosscorrelation of feature images derived from

the original image data has been applied to CT/MR matching

using geometrical features such as edges [15] and ridges [24]

or using especially designed intensity transformations [25].

tiresolution optimization strategy is therefore required, which

is not necessarily a disadvantage, as it can be computationally

attractive.

[13], misregistration is measured by the dispersion of the

two-dimensional (2-D) histogram of the image intensities of

corresponding voxel pairs, which is assumed to be minimal

in the registered position. But the dispersion measures they

188 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 16, NO. 2, APRIL 1997

propose are largely heuristic. Hill’s criterion requires seg-

notion of mutual information (MI) or relative entropy [8],

[22] to describe the dispersive behavior of the 2-D histogram.

MI is a basic concept from information theory, measuring

the statistical dependence between two random variables or

the amount of information that one variable contains about

the other. The MI registration criterion presented here states

that the MI of the image intensity values of corresponding

voxel pairs is maximal if the images are geometrically aligned.

Because no assumptions are made regarding the nature of the

and Wells et al. [27], [28] and by Studholme et al. [21]. The

theoretical concept of MI is presented in Section II, while the

implementation of the registration algorithm is described in

Section III. In Sections IV, V, and VI we evaluate the accuracy

and the robustness of the MI matching criterion for rigid body

CT/MR and PET/MR registration. Section VII summarizes our

current findings, while Section VIII gives some directions for

further work. In the Appendexes, we discuss the relationship

of the MI registration criterion to other multimodality VSB

criteria.

, are statistically independent if

, while they are maximally dependent if they are

related by a one-to-one mapping

:

. MI, , measures the degree of depen-

dence of

and by measuring the distance between the

joint distribution

and the distribution associated to

the case of complete independence

TABLE I

image center. The trace on the left is obtained from an original

MR image and shows a single sharp optimum with a rather

broad attraction basin. The trace on the right is obtained from

the same image after having reduced the spatial correlation of

the image intensity by repeatedly swapping pairs of randomly

selected pixels. This curve shows many local maxima and

the attraction basin of the global maximum is also much

smaller, which deteriorates the optimization robustness. Thus,

although the formulation of the MI criterion suggests that

spatial dependence of image intensity values is not taken into

with its origin positioned in a corner of the image, with the

(a) (b)

Fig. 2. Spatial correlation of image intensity values increases MI registration

robustness. Top: (a) original 256

256 2-D MR image and (b) image of (a)

shuffled by swapping 30 000 randomly selected pixel pairs. Both images have

the same image content. Bottom: MI registration traces obtained using partial

volume distribution (PV) interpolation for in-plane rotation of each image

over itself. Local maxima are marked with “*”.

the reference image,

. can be the set of grid points of

image might be used to increase speed performance, while

supersampling aims at increasing accuracy. For each value

of the registration parameter

angles

distances

(measured in millimeters). Transformation

of image coordinates

to from the image to image

is given by

with

and being 3 3 diagonal matrixes representing the

190 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 16, NO. 2, APRIL 1997

and update the corresponding joint histogram entry, while PV interpolation

distributes the contribution of this sample over multiple histogram entries

defined by its NN intensities, using the same weights as for TRI interpolation.

at position and the intensity at the transformed

position in the reference image

the floating and the reference image intensities are first linearly

rescaled to the range

and , respectively,

being the total number of bins in the joint histogram.

Typically, we use

.

and interpolation of the reference image is needed to

obtain the image intensity value

are originally not present in the reference image, leading to

unpredictable changes in the marginal distribution

of

the reference image for small variations of

. To avoid this

problem, we propose to use trilinear partial volume distribution

(PV) interpolation to update the joint histogram for each voxel

pair

. Instead of interpolating new intensity values in

, the contribution of the image intensity of the sample

distributions

by normalization of

The MI registration criterion

are aligned and have the same orientation. Powell’s multidi-

mensional direction set method is then used to maximize

,

using Brent’s one-dimensional optimization algorithm for the

line minimizations [18]. The direction matrix is initialized with

unit vectors in each of the parameter directions. An appropriate

choice for the order in which the parameters are optimized

needs to be specified, as this may influence optimization

robustness. For instance, when matching images of the brain,

the horizontal translation and the rotation around the vertical

.

However, as the optimization proceeds, the Powell algorithm

may introduce other optimization directions and change the

order in which these are considered.

station (AIX 4.1.3, 58 MHz, 185 SPECfp92; source code

is available on request). The computation time required for

one evaluation of the MI criterion varies linearly with the

number of samples taken from the floating image. While TRI