IEEE SIGNAL PROCESSING MAGAZINE [90] SEPTEMBER 2005
particular, employ dynamic Bayesian networks (DBNs) and a
DBN extension using the Graphical Model Toolkit’s (GMTK’s)
basic template, a dynamic graphical model representation that
is more suitable for speech and language systems. While this
article will concentrate on speech recognition, it should be
noted that many of the ideas presented here are also applicable
to natural language processing and general time-series analysis.
This article assumes some familiarity with basic speech-to-
text concepts [16], [25], [26], [42], [46], [47] and the basics of
graphical models [28], [29], [32], [41], including notions such as
hidden and observed variables, evidence, and factorization and
conditional independence. Moreover, we will use the MATLAB-
like notation
1:N
to denote the set of integers
{1, 2,...,N}
. A
set of
N
random variables (RVs) is denoted as
X
1:N
. Given any
subset
S ⊆ 1:N
, where
S ={S
1
, S
2
,...,S
|S|
}
, the subset of
random variables is denoted as
X
S
={X
S
1
, X
S
2
,...,X
S
|S|
}
.
Lastly, we use upper case letters (such as
X
and
Q
) to refer to
random variables, and lower case letters (such as
x
and
q
) to
refer to random variable values.
DYNAMIC GRAPHICAL MODELS
Graphical models [27], [29], [32], [41] are a set of formalisms,
each of which describes families of probability distributions.
There are many different types of graphical models [27], [31],
[32], [41], each having its own semantics [10] that govern how
the graph specifies a set of factorization constraints on multi-
variate probability distributions. Of course, factorization and
conditional independence go hand in hand; thus, factorization
constraints typically (but not always) involve conditional inde-
pendence properties. A Bayesian network (BN) is one type of
graphical model where the graphs are directed and acyclic. In
a BN, the probability distribution over a set of variables
X
1:N
factorizes with respect to a directed acyclic graph (DAG) as
p(x
1:N
) =
i
p(x
i
|x
π
i
)
, where
π
i
⊂ 1:N
are the set of indices
of
X
i
’s immediate parents according to the BN’s DAG. This
factorization is called the directed factorization property [32].
There are many additional (and provably equivalent)
characterizations of BNs, including the notion of d-separation
[32], [41], but this one suffices for our discussion. It should be
clear that because of the strong relationship between factoriza-
tion and conditional independence, the above factorization
implies that a BN expresses a large number of conditional
independence statements to the extent that it has missing
edges in the graph. Moreover, it should be clear that it is the
common factorization properties of the family of probability
distributions that makes for efficient probabilistic inference
[28], [29], [32], [41].
Speech is inherently a temporal process, and any graphi-
cal model for speech must take this into account.
Accordingly, dynamic graphical models [5] are graphs that
represent the temporal evolution of the statistical properties
of a speech signal, ideally in such a way as to improve auto-
matic speech recognition (ASR) accuracy. For speech recog-
nition, DBNs [10], [15], [37], [49]–[51] have been most
successfully used. DBNs are simply BNs with a repeated
“template” structure over time. Other than this regularity,
however, DBNs have exactly the same semantics as BNs.
Specifically, a DBN of length
T
is a DAG
G = (V, E) =
(
T
t=1
V
t
, E
T
∪
T−1
t=1
E
t
∪E
→
t
)
with node set
V
and edge set
E
comprising pairs of nodes. If
uv ∈ E
for
u, v ∈ V
, then
uv
is an
edge of
G
. The sets
V
t
are the nodes at time slice
t
,
E
t
are the
intraslice edges between nodes in
V
t
, and
E
→
t
are the interslice
edges between nodes in
V
t
and
V
t+1
. A DBN, however, does not
typically have this much flexibility. That is, a DBN is specified
using a “rolled up” template giving nodes that are repeated in
each slice, the intraslice edges among those nodes, and the
interslice edges between nodes of adjacent slices. In other
words,
V
t
and
V
t+τ
have the same set of random variables that
are different only in that the time indexes of the variables differ
by
τ
. The same is true for
E
t
and
E
t+τ
, as well as for
E
→
t
and
E
→
t+τ
. The DBN template is then unrolled to any desired length
T
to yield the DBN
G
. As in any BN, the collection of edges
pointing into a node corresponds to a conditional probability
function (CPF). In a DBN, the CPF of a node is shared (or tied)
with the CPF of all other nodes that have come from the same
underlying node in the DBN template. If
V
t
∈ V
t
with parents
V
π
t
,
then
p(V
t
= v|V
π
t
= v
π
) = p(V
τ
= v|V
π
τ
= v
π
)
for all
t,τ
and
for all scalar values
v
and vector values
v
π
. Therefore, it is possi-
ble to represent a DBN of unbounded length but with only a
finite description length and a finite number of parameters.
It is well known that the hidden Markov model (HMM) is
one type of DBN [44]. Even given its success and flexibility,
however, the HMM is only one small model within the enor-
mous family of statistical techniques represented by DBNs. Like
an HMM, a DBN makes a temporal Markov assumption, mean-
ing that the future is independent of the past given the present.
In fact, it is true that many (but not all, see section titled
“Architectures over Observed Variables”) DBNs can be “flat-
tened” into a corresponding HMM, but staying within the DBN
framework has several advantages. First, in DBN form, there
can be exploitable computational advantages since the DBN
explicitly represents factorization properties and factorization is
the key to tractable probabilistic inference [29]. These factor-
izations, however, are lost when the model is flattened. Second,
the factorization specified by a DBN implies that there are con-
straints that the model must obey. For example, consider
Figure 1, which shows a two-Markov-chain DBN with chains
(Q
1
t
, Q
2
t
)
. A flattened HMM would have one chain
R
t
≡ (Q
1
t
, Q
2
t
)
with transition probabilities set
[FIG1] A simple two-stream Markov chain.
Authorized licensed use limited to: HEFEI UNIVERSITY OF TECHNOLOGY. Downloaded on December 24, 2008 at 00:49 from IEEE Xplore. Restrictions apply.
4星 · 超过85%的资源 需积分: 9 99 浏览量
2009-05-17
20:10:47
上传
评论
收藏 745KB PDF 举报 
guangying01142012-11-29说明的太详细了，就是太长了我看不懂啊看不懂
