IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 10, OCTOBER 2015 1811
No Reference Quality Assessment for
Multiply-Distorted Images Based on
an Improved Bag-of-Words Model
Yanan Lu, Fengying Xie, Tongliang Liu, Zhiguo Jiang, and Dacheng Tao, Fellow, IEEE
Abstract—Multiple distortion assessment is a big challenge in
image quality assessment (IQA). In this letter, a no reference IQA
model for multiply-distorted images is proposed. The features,
whicharesensitivetoeachdistortiontypeeveninthepresence
of other distortions, are first selected from three kinds of NSS
features. An improved Bag-of-Words (BoW) model is then applied
to encode the selected features. Lastly, a simple yet effective line
ar
combination is used to map the image features to the quality score.
The combination weights are obtained through lasso regression. A
series of experiments show that the feature selection strat
egy and
the improved BoW model are effective in improving the accuracy
of quality prediction for multiple distortion IQA. Compared with
other algorithms, the proposed method delivers the best r
esult for
multiple distortion IQA.
Index Terms—Feature encoding, feature selection, image quality
assessment, multiple distortions, no reference.
I. INTRODUCTION
N
O REFERENCE image quality assess
ment (IQA) is of
crucial importance in many image processing and anal-
ysis applications for its ability to predict image quality without
access to a reference image
. Over the past decade, numerous no
reference IQA algorithms for general purpose or specific distor-
tion types have been proposed, among which the general pur-
pose no reference IQA, w
hich predict the visual quality of im-
ages without access to reference images or prior knowledge of
the distortion types, are the most challenging. Researchers have
developed many stat
e-of-the-art algorithms for general purpose
no reference IQA. Being required to reflect the visual quality,
feature is one of the major issues in developing a robust IQA
algorithm. Moo
rthy et al. [1] and Saad et al. [2] extracted fea-
tures in image transformation domains. Mittal et al. [3] used
Manuscript received March 06, 2015; revised May 17, 2015; accepted May
20, 2015. Date of publication May 22, 2015; date of current version June 03,
2015. This work was supported by the National Natural Science Foundation of
China under Grants 61471016 and 61371134, and by the Australian Research
Council Projects DP-140102164, FT-130101457, and LP-140100569. The as-
sociate editor coordinating the review of thi smanuscript and apprroving it for
publication was Prof. Tolga Tasdizen.
Y. Lu, F. Xie, and Z. Jiang are with the Image Processing Center, School of
Astronautics, Beihang University, Beijing, China (e-mail: yn_lu@buaa.edu.cn;
xfy_73@buaa.edu.cn; jiangzg@buaa.edu.cn).
T. Liu and D. Tao are with the Center for Quantum Computation & Intelligent
System and the Faculty of Engineering and Information Technology, Univer-
sity of Technology Sydney, Sydney, Australia (e-mail: tliang.liu@gmail.com;
dacheng.tao@uts.edu.cn).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/LSP.2015.2436908
features in the spatial domain. He et al. [4] introduced sparse
representation to IQA, and Kang et al. [5] used convolutional
neural networks to learn features directly from the raw image
pixels instead of using the handcrafted features to assess image
quality. These methods can predict image quality in high corre-
lation with subjective scores.
Nevertheless, an important problem still exists in IQA re-
search. The distorted images in the majority of publicly available
IQA datasets such as LIVE II [6], IVC [7], CSIQ [8], MICT [9],
and TID [10] suffer from a single distortion type. That
is to say,
most of the existing IQA algorithms are tested on images with
a single type of distortion. However, the images available to
consumers usually reach them after going through s
everal stages
including acquisition, compression, transmission and reception.
In this pipeline, they may suffer multiple distortions. The Lab-
oratory for Image & Video Engineering (LIV
E) recently built
a multiply-distorted image quality dataset LIVE MD [11], for
IQA research. This new multiple dataset consists of two parts:
blur followed by JPEG compression and
blur followed by noise.
Experiments in [11] indicate that even some state-of-the-art
IQA algorithms show poorer performance on this new dataset
compared to their performance o
n single distortion datasets. In
[12], Chandler pointed out that multiply-distorted images are a
big challenge for IQA because an IQA algorithm must not only
consider the joint effects o
f these distortions on the image, but
must also consider the effects of these distortions on each other.
Compared with IQA images of single distortion type, IQA for
multiply-distorted ima
ges has thusfar received less attention. For
example, in [13], Li et al. proposed a method named SHANIA
using statistical features in the Shearlet domain. In [14], Qian et
al. proposed to use t
he multi-scale representation of structure for
IQA. In [15], Gao et al. proposed a model of learning to rank for
IQA. In [16] and [17], Gu et al. respectively proposed a five-step
andasix-stepbl
ind metric (FISBLIM and SISBLIM). The criteria
show that the performance of IQA algorithms for multiple distor-
tions ([14]–[17]) is much poorer than that of IQA algorithms for
single disto
rtion types ([1]–[5]). A sizable gap exists for multiple
distortion IQA research, as it is necessary to learn how to assess
the quality of multiply-distorted images precisely and efficiently.
Feature d
escription for image quality assessment is impor-
tant. In IQA methods, all the assessment metrics are designed
according to features that are related to image quality. The
more s
ensitive the feature is to changes in image quality, the
more effective the assessment metric will be. In this letter,
a no reference IQA method for multiply-distorted images is
pro
posed based on an improved Bag-of-Wo rds (Bo W) model
using Selected Features. The algorithm is called BoWSF for
short. In the next section, the proposed method is described in
1070-9908 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.