method [24] into the image fusion field. However, the image
fusion method based on a cross bilateral filter just used one
source image characteristic to structure the other source image
filter kerne l; it did not consider the directionality of the source
images. Thus, it is not quite satisfied with the performance
of the fused image. Similarly, the image fusion method based
on a guided filter did not consider the directionality of the
source images. The image fusion method based on a multiscale
directional bilateral filter combined the advantages of multi-
scale analysis and image filtering theory; meanwhile it used
a directional bank to capture the directionality of the images.
It obtains an efficient fusion effect. Nevertheless, the fused
image for infrared and visible images is not clear yet since it
has not well preserved the contour information of the source
images when it smoothed the image by bilateral filtering.
To improve the filtering performance of the existing
image filter analysis, a novel image filtering method is proposed
in this paper called the multiscale directional nonlocal means
(MDNLM) filter. It combines the feature of preserving edge
information by the nonlocal means (NLM) filter with the
capacity of capturing directionality of images by the directional
filter bank (DFB). Thus, it can efficiently represent the intrinsic
geometric structure of images and an effective method for
infrared and visible image fusion can be designed by the
MDNLM.
In this paper, an image fusion method for infrared and vis-
ible images based on the MDNLM filter is proposed. First, the
MDNLM is used to decompose the source infrared and visible
images to obt ain detail subbands at different scales and approxi-
mate subbands. Then, an effective fusion strategy is applied to
the subbands to obtain the fused subbands. Finally, the inverse
MDNLM is implemented to the fused subbands to acquire the
final fused image. Comparison experiments have proven that
the MDNLM is effective in infrared and visible image fusion
as described in this paper.
This paper is organized as follows. Section
2 reviews the
principle of the NLM filter and gives the basic principle of the
MDNLM filter. Our proposed MDNLM filter fusion-based
method is discussed in detail in Section
3. Section 4 concretely
describes the experimental results and the performance analysis.
Finally, we conclude this paper in Section
5.
2. RELATED WORK
In this section, we briefly introduce the principle of the NLM
filter. At the same time, we propose the MDNLM filter.
A. Nonlocal Means Filter
The local smoothing methods and the frequency domain filters
aim at a noise reduction at a reconstruction of the main geo-
metrical configurations, but not at the preservation of the fine
structure, details, and texture. In order to overcome the draw-
backs of the local smoothing method or the frequency domain
filtering, the NLM filter was put forward by Buades in [
25]. It
takes advantage of the high degree of redundancy of natural
images to smooth images and preserves the fine structure, de-
tails, and texture of the images. In order to facilitate this, we
assume the image to be processed is u fuxjx ∈ I g, where I
is a discrete grid. The NLM filter is defined as follows:
u
0
x
X
y∈ I
ωx;yuy; (1)
with
ωx;y
1
Zx
exp−‖uN
x
− uN
y
‖
2
2;α
∕k
2
; (2)
uN
x
uy;y ∈ N
x
; (3)
where uN
x
and uN
y
are the intensity gray level, the
similarity of uN
x
and uN
y
determines the similarity be-
tween two pixels x and y, Z x is the normalizing factor
Z x
P
y
exp−‖uN
x
− uN
y
‖
2
2;α
∕k
2
, and the param-
eter k controls the decay of the exponential function. G
α
exp−‖uN
x
− uN
y
‖
2
2;α
∕k
2
represents a spatial Gaussian
function, where α is the standard deviation of Gaussian
function G
α
.
B. Multiscale Directional Nonlocal Means Filter
The NLM filter can be applied to decompose an image into an
approximate subband and a detail subband. The approximate
subband is acquired by applying Eq. (
1) to the source images,
and the detail subband can be easily obtained by the approxi-
mate subband subtracting from the source images. But the
single level of the image decomposition is not comprehensively
extracting the important source image information because
some important features are reflected in different levels. Thus,
we propose the multiscale nonlocal means (MNLM) filter, and
define it as follows:
u
s1
x
X
y∈ I
s
ω
s
x;yu
s
y; (4)
with
ω
s
x; y
1
Z
s
x
exp−‖u
s
N
x
− u
s
N
y
‖
2
2;α
∕k
2
; (5)
Z
s
x
X
y
exp−‖u
s
N
x
− u
s
N
y
‖
2
2;α
∕k
2
; (6)
u
1
x
X
y∈I
ωx;yuy; (7)
where s represents the sth decomposition level and Z
s
is also the
normalization factor. As a matter of fact, the main idea is to
decompose the approximate subband several times in order
to obtain the final approximate subband by Eq. (
4). After S
levels of the MNLM filter, the source image is decomposed
into an approximate subband and S detail subbands.
In order to facilitate this, Eq. (
4) can be simplified to
T
s
u
s
: (8)
Then the detail subband can be defined as
D
s1
T
s
− T
s1
; (9)
with
D
1
u − u
1
: (10)
However, one of the basic tasks in computer vision is to get
the directional representation of the features from an image.
Generally, these features are the edges and the irregular or
anisotropic lines. Thus, it is necessary to construct a directional
4300 Vol. 54, No. 13 / May 1 2015 / Applied Optics
Research Article