TWO DIMENSIONAL SYNTHESIS SPARSE MODEL
Na Qi
1
, Yunhui Shi
1
, Xiaoyan Sun
2
,Jingdong Wang
2
,Baocai Yin
1
1
Beijing Key Laboratory of Multimedia and Intelligent Software Technology
College of Computer Science and Technology
Beijing University of Technology, Beijing, China
2
Microsoft Research Asia, Beijing, China
q1987n@emails.bjut.edu.cn,{syhzm,ybc}@bjut.edu.cn,{xysun,jingdw}@microsoft.com
ABSTRACT
Sparse representation has been proved to be very efficient in
machine learning and image processing. Traditional image
sparse representation formulates an image into a one dimen-
sional (1D) vector which is then represented by a sparse lin-
ear combination of the basis atoms from a dictionary. This
1D representation ignores the local spatial correlation inside
one image. In this paper, we propose a two dimensional
(2D) sparse model to much efficiently exploit the horizon-
tal and vertical features which are represented by two dictio-
naries simultaneously. The corresponding sparse coding and
dictionary learning algorithm are also presented in this pa-
per. The 2D synthesis model is further evaluated in image
denoising. Experimental results demonstrate our 2D synthe-
sis sparse model outperforms the state-of-the-art 1D model in
terms of both objective and subjective qualities.
Index Terms— Synthesis Sparse Model, Sparse Repre-
sentation, 2D-KSVD, Dictionary Learning, Image Denoising
1. INTRODUCTION
Sparse representation has been widely studied and provided
promising performance in numerous signal processing tasks
such as image denoising, texture synthesis, audio processing,
and image classification. Modeling a signal x ∈ R
d
by sparse
representation involves two ways, the synthesis sparse mod-
eling and analysis sparse modeling [1]. The synthesis model
can be defined as:
x = Db, s.t.kbk
0
= k, (1)
where D ∈ R
d×n
is an over-complete dictionary in which
each column denotes an atom, b ∈ R
n
is a sparse vector. The
sparsity is measured by the l
0
norm k · k
0
which is valued by
the number of nonzero entries k of a vector. In this model, x
is synthesized by a linear combination of certain atoms from
D [2]. Correspondingly, the analysis sparse model is defined
as:
b = Ωx, s.t.kbk
0
= p − l, (2)
where Ω ∈ R
p×d
is a linear operator (also called as a dic-
tionary), and l denotes the co-sparsity of the signal x. The
analysis representative vector b is sparse with l zeros. The
zeros in b denote the low-dimensional subspace to which the
signal x belongs.
The dictionaries in both analysis and synthesis models
play an important role in sparse representation. They can
be roughly classified into two categories, analytic dictionaries
and learned dictionaries, according to the generation methods.
The analytic dictionaries are predefined and built by mathe-
matics models. The typical predefined dictionaries to natural
images include Wavelets [3], Curvelet [4], Contourlets [5],
and Bandelets [6]. Differently, the learned dictionaries are
produced by training samples. Compared with the analytic
dictionaries which have limited expressions, the learned dic-
tionaries can adaptively represent a wider range of signal
characteristic.
Different learning algorithms, e.g. K-SVD [7] and sparse
coding [8], have been proposed to generate learned dictionar-
ies from a set of samples. Sparse coding, as a fundamen-
tal method in dictionary learning, provides a class of algo-
rithms for finding sparse representations of an training set.
Many algorithms, such as matching pursuit(MP), orthogonal
MP(OMP) [9], Lasso, and proximal method [10] have been
proposed to solve the sparse pursuit problem. Lee. et al.
in [8] formulate the sparse coding algorithms as a combina-
tion of two convex optimization problems: the L1-regularized
least squares problem solved by feature-sign search to learn
coefficients, and the L2-constrained least squares problem
solved by a Lagrange dual method to learn the bases for any
sparsity penalty function. The K-SVD is an iterative method
that alternates between sparse coding of the examples based
on the current dictionary and an updating process of the dic-
tionary atoms so as to better fit the data [7]. It is a two-phase
block coordinate-relaxation approach.
We observe that all the previous sparse models with
learned dictionaries treat an input signal as a 1D vector. When
dealing with 2D signals, they are reshaped into 1D vector. For
natural images, this kind of reshape breaks the local correla-