PLBP：具有金字塔表示的有效局部二进制模式纹理描述符资源-CSDN文库

Local

binary

51 浏览量 2021-02-23 01:58:29 上传评论收藏 2.24MB PDF 举报

资源推荐

资源详情

资源评论

PLBP: An effective local binary patterns texture descriptor with

pyramid representation

Xueming Qian

, Xian-Sheng Hua

, Ping Chen

, Liangjun Ke

School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, China

Microsoft Research Asia, Beijing 10080, China

article info

Article history:

Received 8 August 2010

Received in revised form

11 March 2011

Accepted 30 March 2011

Available online 8 April 2011

Keywords:

Local binary pattern

Pyramid transform

Texture

Gaussian ﬁlter

Wavelet

abstract

Local binary pattern (LBP) is an effective texture descriptor which has successful applications in texture

classiﬁcation and face recognition. Many extensions are made for conventional LBP descriptors. One of

the extensions is dominant local binary patterns which aim at extracting the dominant local structures

in texture images. The second extension is representing LBP descriptors in Gabor transform domain

(LGBP). The third extension is multi-resolution LBP (MLBP). Another extension is dynamic LBP for video

texture extraction. In this paper, we extend the conventional local binary pattern to pyramid transform

domain (PLBP). By cascading the LBP information of hierarchical spatial pyramids, PLBP descriptors take

texture resolution variations into account. PLBP descriptors show their effectiveness for texture

representation. Comprehensiv e comparisons are made for LBP, MLBP, LGBP, and PLBP. Performances

of no sampling, partial sampling and spatial pyramid sampling approaches for the construction of PLBP

texture descriptors are compared. The inﬂuences of pyramid generation approaches, and pyramid levels

to PLBP based image categorization performances are discussed. Compared to the existing multi-

resolution LBP descri ptors, PLBP is with satisfactory performances and with low computational costs.

1. Introduction

Recently, Bag-of-Words (BoW) models have been shown their

effectiveness in image classiﬁcation and retrieval [1–11]. BoW

based scene categorization approaches model objects in an image

as geometric-free structures. Thus the BoW based approaches are

robust to the illumination, occlusion, rotation, and resolution

variations. In many applications, the co-occurrences, dependences

and linkages of BoW are modeled [3,4,8–10]. However, the BoW

models are less discriminative for texture classiﬁcation. How to

classify the texture images effectively is a challenging problem.

Usually, most texture classiﬁcation approaches are based on the

assumption that the speciﬁed texture images to be classiﬁed are

identical to the training images in resolution, contrast, orienta-

tion, and other visual appearances. However, in the real world,

textures can occur at arbitrary spatial resolutions and orienta-

tions. Texture is also subjected to the variations of illuminations

and imaging conditions. The texture patches closing to camera

are usually with high resolutions while those far from camera

are with low resolutions. This can be shown from the examples

in Fig. 1.

In order to carry out effective texture classiﬁcation, robust

texture descriptors are required. GIST is an effective texture

descriptor [12,13]. It is represented in multi-resolution and

multi-orientation Gabor transform domain. The GIST descriptor

cascades the magnitudes of all the sub-bands. Hierarchical wave-

let packet transform (HWVP) is utilized for texture representation

in [14]. HWVP and GIST descriptors have shown their effective-

ness in image categorization [15,31].

Despite of utilizing multi-resolution based texture representa-

tion [11–15], Ojala et al. represent texture features by observing

the statistical distributions of local binary patterns [17]. Uniform

and rotation-invariant uniform LBP descriptors are extended from

original LBP to extract the uniform and rotation stable local

binary patterns [17]. Various extensions are made for the con-

ventional LBP descriptors [19–25,28–30]. Due to their excellent

performances, LBP and its extensions have been successfully

utilized in image classiﬁcation and face recognition. Zhang et al.

[21] represent local binary pattern in Gabor transform domain.

The marriage of Gabor transform and LBP descriptors further

improves the discriminative power of LBP descriptors. First,

multi-scale and multi-orientation Gabor ﬁltering is carried out

for a given image, and then local binary patterns are extracted on

Gabor ﬁltered images. Finally, all the LBP histograms of the Gabor

ﬁltered sub-bands are concatenated into a single histogram

sequence [21]. By inheriting the advantages of Gabor ﬁltering,

the local Gabor binary patterns (LGBP) are rotation invariant and

Contents lists available at ScienceDirect

journal homepage: www.elsevier.com/locate/pr

Pattern Recognition

doi:10.1016/j.patcog.2011.03.029

Corresponding author. Tel.: þ86 29 82667771.

E-mail addresses: qianxm@mail.xjtu.edu.cn,

qianxueming@gmail.com (X. Qian), xshua@microsoft.com,

huaxiansheng@gmail.com (X.-S. Hua), kelj163@163.com (L. Ke).

Pattern Recognition 44 (2011) 2502–2515

less sensitive to illumination variations. A volume local binary

patterns (VLBP) based dynamic texture recognition method is

proposed for video sequences [22,23]. Compared to the basic LBP,

VLBP uses both the spatial information and the temporal informa-

tion. It is a 3D-LBP which has advantages for representing

dynamic textures by extending conventional LBP from 2D image

to 3D video sequence. In Ref. [25], Lei et al. further extend VLBP to

Gabor transform domain. A spatial-temporal local binary pattern

is utilized to model dynamic background. Dominant local binary

pattern (DLBP) is aiming at capturing the dominating patterns in

texture images [19]. DLBP is extended from uniform LBP by

counting the occurrence frequencies of all rotation invariant

patterns deﬁned in LBP groups. The DLBP is effective in describing

texture images with complicated shapes, curvature edges, cross-

ing boundaries and corners [19]. Similar to uniform LBP, DLBP is

constructed by sorting the occurrence frequencies of the patterns

in descending order.

Although many extensions have been made for conventional

LBP [19–25,28–30], their basic assumption is that texture resolu-

tion of an image is ﬁxed as shown in Fig. 1(a). Actually, texture

patches in an image can be with various resolutions as shown in

Fig. 1(b). Each of the three examples shown in Fig. 1(a) is with

same resolution. There are resolution variations among the

images. The texture resolutions of each images shown in

Fig. 1(b) are varying signiﬁcantly. Usually, texture information

in a ﬁxed resolution does not have signiﬁcant discriminative

power. For example, the LBP labels of the center pixels of the

three patches are all mapped to the same bin at the ﬁrst

resolution (resolution #1), while the LBP labels are mapped to

different bins at the other resolutions as shown in Fig. 2. That is

to say, the coming of LBP labels of the other resolutions improves

the discriminative power. Thus considering resolution variations

can be contributive for texture classiﬁcation. The rest of this

paper is organized as follows: In Section 2 the conventional

LBP and its extensions are brieﬂy overviewed, and then LBP

with pyramid representation is introduced in detail. In Section 3

performance evaluation approach is given. In Section 4 experimental

results and discussions are given. Finally, conclusions are drawn in

Section 5.

2. Local binary pattern with pyramid representation

First, the conventional LBP descriptor and its extensions are

brieﬂy reviewed, and then the proposed local binary pattern with

pyramid representation and its relationships with existing multi-

resolution LBP descriptors are illustrated in detail.

2.1. Conventional local binary pattern and its extensions

Ojala et al. represent the texture information by deﬁning

texture T in a local neighborhood of a gray level image as the

joint distribution of the gray levels of P image pixels as follows:

T ¼tðg

,...,g

P1

Þtðsðg

g

Þ,...,sðg

P1

g

ÞÞ ð1Þ

where gray value g

corresponds to the gray value of the center

pixel of a local neighborhood and g

ðp ¼0,...,P1Þ correspond to

Fig. 1. Texture images with various resolutions. In (a) each image is with same resolution, images are with same content but with various resolutions. (b) The texture

resolution are varying in each image.

LBP

(Resolution #1)

LBP

(Resolution #2)

LBP

(Resolution # n)

Local

Patch

Fig. 2. Diagram of LBP labels in different resolutions. The LBP labels of the center

pixels of the three patches are the same at the ﬁrst resolution (Resolution #1),

thus are hard to be discriminated at resolution #1, while they are discriminative

with the coming of the LBP labels in the following resolutions.

X. Qian et al. / Pattern Recognition 44 (2011) 2502–2515 2503

the gray values of P equally spaced pixels on a circle of radius

R(R4 0) that form a circularly symmetric neighbor set. And

sðxÞ¼

0,xZ 0

1,xo 0

(

. Finally, according to the rule utilized in Ref.

[17], a binary factor 2

is assigned to each neighbor. The original

LBP value of a pixel is represented by

LBP ¼

P1

p ¼ 0

ðsðg

g

Þ2

Þð2Þ

The LBP descriptor labels the pixels of an image by determin-

ing the gray levels of the P neighbors (with radius R) of the center

pixel as shown in Fig. 3. Finally, the histogram of the labels is

utilized for texture description.

Ojala et al. [18] extend the original LBP descriptor to multi-

resolution LBP by using neighbors of different sizes. Using circular

neighborhoods and interpolating the pixel values allow any

radius and number of pixels in the neighborhood as shown in

Fig. 3. Thus the ﬁnal multi-resolution LBP (denoted MLBP)is

constructed as follows:

MLBP ¼[

P,R

LBP

P,R

¼/LBP

;...;LBP

S ð3Þ

where P

and R

(i¼1,y,S) denote the neighbor number and

radius. The combination can improve the discriminative power

of the texture descriptors. However MLBP descriptors are

obtained from the same image with ﬁxed resolution. MLBP

represents multi-resolution texture information by sparse sam-

pling [29]. Maenpaa et al. [28,29] have pointed out that this

representation approach is sensitive to noise, because sampling is

made at a single pixel position rather than an effective region.

Moreover, direct sampling usually causes aliasing effects [28,29].

To overcome the above two shortcomings, multi-resolution LBP

descriptor are proposed by utilizing low-pass ﬁltering (LBPF) and

local averaging [28–30]. In LBPF, each sample in the neighborhood

can be made to collect intensity information from a large effective

area rather than a single pixel as shown in Fig. 6. The relationship

of LBPF and the proposed PLBP is deeply analyzed in Section 2.3.

Local Gabor binary pattern is extracted by carrying out Gabor

transform for the image with n-scale and m-orientation ﬁltering

[21]. This method improves the discriminative power of original

LBP [17]. In this paper, we also represent LGBP in spatial pyramid

domain. Experimental results are given in Section 4.2.

2.2. Local binary pattern represented in spatial pyramid domain

Pyramid transform is an effective multi-resolution analysis

approach. In this paper, we represent local binary pattern in

spatial pyramid domain.

During pyramid transform, each pixel in the low spatial

pyramid is obtained by down sampling from its adjacent low-

pass ﬁltered high resolution image as shown in Fig. 4(b). Thus in

the low-resolution images, a pixel corresponds to a region in its

high-resolutions. In Refs. [28,29], Maenpaa et al. call the region as

‘‘effective area’’. Sequential pyramid images are constructed as

shown in Fig. 5. Each neighboring two images are with resolution

variation rate 2. That is to say, the down sampling ratios in x- and

y-directions are both

ﬃﬃﬃ

. Pyramid image can be generated by low-

pass ﬁlters of wavelet transform, Gaussian smooth ﬁltering

[28,29], symmetric weighting and block averaging [30].

The pyramid generation approach consists of low-pass ﬁltering

and down sampling images of the preceding pyramid level. Let f(x, y)

(8,2)(8,1) (12,2) (8,3)

Fig. 3. Four Examples of the LBP. (a) the circular (8,1) with P¼8 and R ¼1; (b) the

circular (8,2) with P¼8 and R¼2; (c) the circular (12,2) with P¼12 and R¼2; and

(d) the circular (8,3) with P ¼8 and R ¼3. The pixel values are interpolated

whenever the sampling point is not in the center of a pixel.

Level 1

Level 2

Level 3

Level 4

Fig. 4. Diagram of pyramid transform and spatial pyramid sampling. (a) A four level spatial pyramids are shown. (b) The diagram of pyramid sampling in neighboring

3 resolutions. The down sampling ratios in x- and y- directions are both 2. The resolution variation of neighboring two pyramids is with a factor 4.

Fig. 5. Gaussian Pyramid images. The original image (level 1) and their Gaussian pyramids shown from left to right. The resolution variation of two neighboring pyramid is