张志华机器学习导论课件资源-CSDN文库

共20个文件

pdf：20个

机器学习导论

5星 · 超过95%的资源需积分: 35 84 浏览量 2018-11-26 16:59:34 上传评论 2 收藏 3.41MB RAR 举报

资源推荐

资源详情

资源评论

收起资源包目录

机器学习导论课件(张志华).rar （20个子文件）

机器学习导论课件(张志华)

lecture12.pdf 233KB

lecture13.pdf 70KB

lecture2.pdf 201KB

lecture11.pdf 208KB

lecture3.pdf 234KB

lecture7.pdf 62KB

homework2_solution.pdf 172KB

homework2.pdf 125KB

Supplemental Materials

scale_mixture_distribution.pdf 188KB

matrix.pdf 649KB

homework1.pdf 117KB

lecture9.pdf 212KB

lecture4.pdf 205KB

lecture14.pdf 142KB

lecture10.pdf 209KB

lecture6.pdf 218KB

lecture5.pdf 180KB

lecture8.pdf 202KB

homework1_solution.pdf 42KB

lecture1.pdf 70KB

Matrix Review

Luo Luo

March 18, 2014

Contents

1 Basic Concepts and Notations 2

2 Matrix Operations and Properties 3

2.1 Matrix Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1.1 Vector-Vector Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1.2 Matrix-Vector Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.1.3 Matrix-Matrix Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.4 Kronecker Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.5 Block Matrix Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 The Transpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.3 Symmetric Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.4 The Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.5 Vector Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.6 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.7 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.8 The Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.9 Range and Nullspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.10 QR Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.11 Matrix Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.11.1 Induced Matrix Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.11.2 General Matrix Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.12 The Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.13 The Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.14 Schur Complement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Eigenvalues and Eigenvectors 16

3.1 Deﬁnitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.2 General Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.3 Spectral Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.3.1 Spectral Decomposition Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.3.2 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4 Singular Value Decomposition 19

4.1 Deﬁnitions and Basic Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.2 Low-Rank Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.3 SVD vs. Spectral Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5 Quadratic Forms and Deﬁniteness 22

6 Matrix Calculus 24

6.1 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

6.2 The Gradient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

6.3 The Hessian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6.4 Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6.5 Digression: Generalized Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

6.6 Gradients of the Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

6.7 Eigenvalues as Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

6.8 General Derivatives of Matrix Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

7 Bibliographic Remarks 29

1 Basic Concepts and Notations

We use the following notation:

• By A ∈ R

m×n

we denote a matrix with m rows and n columns, where the entries of A are real

numbers.

• By x ∈ R

, we denote a vector with n entries.

• An n-dimensional vector is often thought of as a matrix with n rows and 1 column, known as

column vector. A row vector is a matrix with 1 row and n columns, which we typically write

denote the transpose of x, which we will deﬁne shortly).

• The ith element of a vector x is denoted x

x =













• We use notation a

to denote the entry of A in the ith row and jth column:

A =







. . . a







= (a

• We denote the jth columns of A by a

A =





. . .

. . . a

. . .





• We denote the ith row of A by a

A =







(1)

(2)

(m)







or A =













• The meanings of the notations usually should be obvious from its use when they are ambiguous.

• A matrix written in terms of its sub-matrices is called a partitioned matrix:

A =







. . . A







where A

∈ R

r×s

, m = pr, and n = qs. Based on the Matlab colon notation, we have A



(i − 1)r + 1 : ir, (j − 1)s + 1 : s



Table 1: Particular matrices and types of matrix

Name Deﬁnition Notation

Scalar m = n = 1 a, b . . .

Column vector n = 1 x, y . . .

All-ones vector [1, . . . , 1]

1 or 1

Null a

= 0 0

Square m = n A ∈ R

n×n

Diagonal m = n, a

= 0 if i 6= j diag(a

, . . . , a

) or diag([a

, . . . , a

]

)

Identity diag(1) I or I

Symmetric a

= a

A ∈ S

Table 2: Basic matrix operations

Operation Restrictions Notations and Deﬁnitions

Addition / Subtraction A, B of the same order A ± B = (a

± b

)

Scalar multiplication cA = c(a

)

Inner product x, y of the same order x

y =

Multiplication #columns of A equals #rows of B AB = (a

(i)

)

Kronecker Product A ⊗ B = (a

Transpose A

= [a

(1)

, . . . , a

(m)

]

Trace A is square tr(A) =

Determinant A is square |A| or det(A)

Inverse A is sqaure and det(A) 6= 0 AA

−1

= A

−1

A = I

Pseudo-inverse A

†

2 Matrix Operations and Properties

2.1 Matrix Multiplication

The product of two matrices A ∈ R

m×n

and B ∈ R

n×p

is the matrix

C = AB ∈ R

m×p

where

k=1

Note that in order for the matrix product to exist, the number of columns in A must equal the number

of rows in B. There are many ways of looking at matrix multiplication. We’ll start by examining a few

special cases.

2.1.1 Vector-Vector Products

Given two vectors x, y ∈ R

, the quantity x

y, sometimes called the inner product or dot product of

the vectors, is a real number given by

y ∈ R =



. . . x















i=1

Note that it is always the case that x

y = y

Given x ∈ R

, y ∈ R

(not for the same size), xy

is called the outer product of the vectors. It is

a matrix whose entries are given by (xy

)

= x

∈ R

m×n















. . . y









. . . x







Consider the matrix A ∈ R

m×n

whose columns are equal to some vector x ∈ R

. Using outer products,

we can represent A compactly as,

A =





x x . . . x











. . . x





















1 1 . . . 1



= x1

2.1.2 Matrix-Vector Products

Given a matrix A ∈ R

m×n

and a vector x ∈ R

, their product is a vector y = Ax ∈ R

. There are a

couple ways of looking at matrix-vector multiplication, and we will look at each of them in turn.

If we write A by rows, then we can express Ax as,

y = Ax =







(1)

(2)

(m)







x =







(1)

(2)

(m)







In other words, the ith entry of y is equal to the inner product of the ith row of A and x, y

= a

(i)

Alternatively, lets write A in column form. In this case we see that,

y = Ax =





. . .

. . . a

. . .

































+ ··· +









In other words, y is a linear combination of the columns of A, where the coeﬃcients of the linear

combination are given by the entries of x.

So far we have been multiplying on the right by a column vector. We also have similar interpretation

of multiplying on the left by a row vector. For A ∈ R

m×n

and a vector x ∈ R

, which gives

= x

A = x





. . .

. . . a

. . .

















= x



. . . x









(1)

(2)

(m)







= x

(1)

+ x

(2)

+ ··· + x

(n)

2.1.3 Matrix-Matrix Products

Armed with this knowledge, we can now look at four diﬀerent (but, of course, equivalent) ways of viewing

the matrix-matrix multiplication C = AB.

• We can view matrix-matrix multiplication as a set of vector-vector products. The most obvious

viewpoint, which follows immediately from the deﬁnition, is that the (i, j)th entry of C is equal to

the inner product of the ith row of A and the jth row of B.

C = AB =







(1)

(2)

(m)











. . .

. . . b

. . .





A =







(1)

. . . a

(1)

(2)

. . . a

(2)

(m)

. . . a

(m)







• Alternatively, we can represent A by columns, and B by rows. This representation leads to a much

trickier interpretation of AB as a sum of outer products.

C = AB =





. . .

. . . a

. . .











(1)

(2)

(n)







i=1

• We can also view matrix-matrix multiplication as a set of matrix-vector products. Speciﬁcally, if

we represent B by columns, we can view the columns of C as matrix-vector products between A

and the columns of B.

C = AB = A





. . .

. . . b

. . .









. . .

. . . Ab

. . .





• We have the analogous viewpoint, where we represent A by rows, and view the rows of C as the

matrix-vector product between the rows of A and C.

C = AB =







(1)

(2)

(m)







B =







(1)

(2)

(m)







In addition to this, it is useful to know a few basic properties of matrix multiplication at a higher level:

• Matrix multiplication is associative: (AB)C = A(BC).

• Matrix multiplication is distributive: A(B + C) = AB + AC.

• Matrix multiplication is, in general, not commutative; that is, it can be the case that AB 6= BA.

(For example, if A ∈ R

m×n

and B ∈ R

m×q

, the matrix product BA does not even exist if m and

q are not equal!)

2.1.4 Kronecker Products

The Kronecker product, denoted by ⊗, is an operation on two matrices of arbitrary size resulting in a

block matrix. It is a generalization of the outer product from vectors to matrices The Kronecker product

should not be confused with the usual matrix multiplication, which is an entirely diﬀerent operation.

If A is an m × n matrix and B is a p × q matrix, then the Kronecker product A ⊗ B is the mp × nq

block matrix:

A ⊗ B =







B ··· a







评论收藏

内容反馈

CS-er

2019-08-31

非常感谢分享宝贵资源
alto1394

2019-08-27

张志华的机器学习导论的课件,2017年，英文pdf。

lzgyxc8204

粉丝: 0
资源: 4

张志华机器学习导论课件

张志华 - 机器学习导论讲义-良心资料

张志华课堂课件

张志华《机器学习导论》笔记和补充材料

张志华 机器学习讲义+作业 内含Aipaydin的机器学习导论

机器学习导论PPT

第1章机器学习导论.pptx

机器学习导论中文笔记整理.zip

吴恩达机器学习视频百度网盘（视频+PPT+个人笔记+作业）

统计机器学习讲义

机器学习课堂讲义及答案

统计机器学习笔记.zip

《算法导论》开放课程教学视频讲义02

学习：数据是石油的未来，挖掘数据的潜在价值非常有意义。 这个图书馆记录了我的机器学习之路

机器学习课件概论-概念

机器学习介绍PPT

Reinforcement Learning

数字经济下企业财务管理模式的创新_张志华_会计财务管理毕业论文范文模板、范本.pdf

Deep Learning Book (深度学习)中文版

C#设计模式（中文版） PDF

Deep Learning book

论文研究-基于Bayes方法的初始备件调整方法研究.pdf

ChatGPT教程（终极版）最全整理

博客中Kmeans以及FCM算法数据（免积分）

基于Python+pytorch的图像处理+附完整代码图像处理，能够轻松实现图像的读取、显示、裁剪等还有机器学习等操作

hugging face的models-openai-clip-vit-large-patch14文件夹

神经网络回归预测--气温数据集

XGBoost+LightGBM+LSTM-光伏发电量预测

最新资源

张志华机器学习讲义+作业内含Aipaydin的机器学习导论

学习：数据是石油的未来，挖掘数据的潜在价值非常有意义。这个图书馆记录了我的机器学习之路