Old and New Matrix Algebra Useful for Statistics
Thomas P. Minka
Contents
1 Derivatives 1
2 Kronecker pro duct and vec 6
3 Vec-transp ose 7
4 Multilinear forms 8
5 Hadamard pro duct and diag 10
6 Inverting partitioned matrices 12
7 Polar decomp osition 14
8 Hessians 15
Warning: This pap er contains a large number of matrix identities which cannot be absorbed
by mere reading. The reader is encouraged to take time and check each equation by hand and
work out the examples. This is advanced material; see Searle (1982) for basic results.
1 Derivatives
Maximum-likeliho od problems almost always require derivatives. There are six kinds of deriva-
tives that can be expressed as matrices:
Scalar Vector Matrix
Scalar
d
y
d
x
d
y
d
x
=
h
@ y
i
@ x
i
d
Y
d
x
=
h
@ y
ij
@ x
i
Vector
d
y
d
x
=
h
@ y
@ x
j
i
d
y
d
x
=
h
@ y
i
@ x
j
i
Matrix
d
y
d
X
=
h
@ y
@ x
ji
i
The partials with resp ect to the numerator are laid out according to the shape of
Y
while the
partials with respect to the denominator are laid out according to the transpose of
X
. Each
of these derivatives can be tediously computed via partials, but this section shows how they
instead can b e computed with matrix manipulations. The material is based on Magnus and
Neudecker (1988).
Dene the dierential d
y
(
x
) to be that part of
y
(
x
+ d
x
)
?
y
(
x
) which is linear in d
x
. Unlike the
classical denition in terms of limits, this denition applies even when
x
or
y
are not scalars.
For example, this equation:
y
(
x
+ d
x
) =
y
(
x
) +
A
d
x
+ (higher order terms) (1)
1
剩余18页未读，继续阅读
资源评论
