没有合适的资源?快使用搜索试试~ 我知道了~
Bayesian reasoning and machine learning电子书
5星 · 超过95%的资源 需积分: 11 44 下载量 107 浏览量
2013-12-24
11:58:12
上传
评论
收藏 13.64MB PDF 举报
温馨提示
试读
672页
2013年12月版《Bayesian reasoning and machine learning》一书的电子版
资源推荐
资源详情
资源评论
Bayesian Reasoning and Machine Learning
David Barber
c
2007,2008,2009,2010,2011,2012,2013
Notation List
V a calligraphic symbol typically denotes a set of random variables . . . . . . . . 7
dom(x)
Domain of a variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
x = x
The variable x is in the state x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
p(x = tr) probability of event/variable x being in the state true . . . . . . . . . . . . . . . . . . . 7
p(x = fa) probability of event/variable x being in the state false . . . . . . . . . . . . . . . . . . . 7
p(x, y) probability of x and y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
p(x ∩ y) probability of x and y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
p(x ∪ y) probability of x or y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
p(x|y) The probability of x conditioned on y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
X ⊥⊥Y|Z Variables X are independent of variables Y conditioned on variables Z . 11
X>>Y|Z Variables X are dependent on variables Y conditioned on variables Z . . 11
R
x
f(x)
For continuous variables this is shorthand for
R
f(x)dx and for discrete vari-
ables means summation over the states of x,
P
x
f(x) . . . . . . . . . . . . . . . . . . 18
I [S] Indicator : has value 1 if the statement S is true, 0 otherwise . . . . . . . . . . 19
pa (x) The parents of node x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
ch (x)
The children of node x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
ne (x) Neighbours of node x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
dim (x) For a discrete variable x, this denotes the number of states x can take . . 34
hf(x)i
p(x)
The average of the function f(x) with respect to the distribution p(x) .158
δ(a, b)
Delta function. For discrete a, b, this is the Kronecker delta, δ
a,b
and for
continuous a, b the Dirac delta function δ(a − b) . . . . . . . . . . . . . . . . . . . . . . 160
dim x The dimension of the vector/matrix x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
] (x = s, y = t) The number of times x is in state s and y in state t simultaneously . . . 197
]
x
y
The number of times variable x is in state y . . . . . . . . . . . . . . . . . . . . . . . . . . 278
D
Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .291
n
Data index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
N Number of dataset training points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
S Sample Covariance matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
σ(x) The logistic sigmoid 1/(1 + exp(−x)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
erf(x) The (Gaussian) error function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
x
a:b
x
a
, x
a+1
, . . . , x
b
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455
i ∼ j The set of unique neighbouring edges on a graph . . . . . . . . . . . . . . . . . . . . . .585
I
m
The m × m identity matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605
II DRAFT December 9, 2013
Preface
The data explosion
We live in a world that is rich in data, ever increasing in scale. This data comes from many different
sources in science (bioinformatics, astronomy, physics, environmental monitoring) and commerce (customer
databases, financial transactions, engine monitoring, speech recognition, surveillance, search). Possessing
the knowledge as to how to process and extract value from such data is therefore a key and increasingly
important skill. Our society also expects ultimately to be able to engage with computers in a natural manner
so that computers can ‘talk’ to humans, ‘understand’ what they say and ‘comprehend’ the visual world
around them. These are difficult large-scale information processing tasks and represent grand challenges
for computer science and related fields. Similarly, there is a desire to control increasingly complex systems,
possibly containing many interacting parts, such as in robotics and autonomous navigation. Successfully
mastering such systems requires an understanding of the processes underlying their behaviour. Processing
and making sense of such large amounts of data from complex systems is therefore a pressing modern day
concern and will likely remain so for the foreseeable future.
Machine Learning
Machine Learning is the study of data-driven methods capable of mimicking, understanding and aiding
human and biological information processing tasks. In this pursuit, many related issues arise such as how
to compress data, interpret and process it. Often these methods are not necessarily directed to mimicking
directly human processing but rather to enhance it, such as in predicting the stock market or retrieving
information rapidly. In this probability theory is key since inevitably our limited data and understanding
of the problem forces us to address uncertainty. In the broadest sense, Machine Learning and related fields
aim to ‘learn something useful’ about the environment within which the agent operates. Machine Learning
is also closely allied with Artificial Intelligence, with Machine Learning placing more emphasis on using data
to drive and adapt the model.
In the early stages of Machine Learning and related areas, similar techniques were discovered in relatively
isolated research communities. This book presents a unified treatment via graphical models, a marriage
between graph and probability theory, facilitating the transference of Machine Learning concepts between
different branches of the mathematical and computational sciences.
Whom this book is for
The book is designed to appeal to students with only a modest mathematical background in undergraduate
calculus and linear algebra. No formal computer science or statistical background is required to follow the
book, although a basic familiarity with probability, calculus and linear algebra would be useful. The book
should appeal to students from a variety of backgrounds, including Computer Science, Engineering, applied
Statistics, Physics, and Bioinformatics that wish to gain an entry to probabilistic approaches in Machine
Learning. In order to engage with students, the book introduces fundamental concepts in inference using
III
only minimal reference to algebra and calculus. More mathematical techniques are postponed until as and
when required, always with the concept as primary and the mathematics secondary.
The concepts and algorithms are described with the aid of many worked examples. The exercises and
demonstrations, together with an accompanying MATLAB toolbox, enable the reader to experiment and
more deeply understand the material. The ultimate aim of the book is to enable the reader to construct
novel algorithms. The book therefore places an emphasis on skill learning, rather than being a collection of
recipes. This is a key aspect since modern applications are often so specialised as to require novel methods.
The approach taken throughout is to describe the problem as a graphical model, which is then translated
into a mathematical framework, ultimately leading to an algorithmic implementation in the BRMLtoolbox.
The book is primarily aimed at final year undergraduates and graduates without significant experience in
mathematics. On completion, the reader should have a good understanding of the techniques, practicalities
and philosophies of probabilistic aspects of Machine Learning and be well equipped to understand more
advanced research level material.
The structure of the book
The book begins with the basic concepts of graphical models and inference. For the independent reader
chapters 1,2,3,4,5,9,10,13,14,15,16,17,21 and 23 would form a good introduction to probabilistic reasoning,
modelling and Machine Learning. The material in chapters 19, 24, 25 and 28 is more advanced, with the
remaining material being of more specialised interest. Note that in each chapter the level of material is of
varying difficulty, typically with the more challenging material placed towards the end of each chapter. As
an introduction to the area of probabilistic modelling, a course can be constructed from the material as
indicated in the chart.
The material from parts I and II has been successfully used for courses on Graphical Models. I have also
taught an introduction to Probabilistic Machine Learning using material largely from part III, as indicated.
These two courses can be taught separately and a useful approach would be to teach first the Graphical
Models course, followed by a separate Probabilistic Machine Learning course.
A short course on approximate inference can be constructed from introductory material in part I and the
more advanced material in part V, as indicated. The exact inference methods in part I can be covered
relatively quickly with the material in part V considered in more in depth.
A timeseries course can be made by using primarily the material in part IV, possibly combined with material
from part I for students that are unfamiliar with probabilistic modelling approaches. Some of this material,
particularly in chapter 25 is more advanced and can be deferred until the end of the course, or considered
for a more advanced course.
The references are generally to works at a level consistent with the book material and which are in the most
part readily available.
Accompanying code
The BRMLtoolbox is provided to help readers see how mathematical models translate into actual MAT-
LAB code. There are a large number of demos that a lecturer may wish to use or adapt to help illustrate
the material. In addition many of the exercises make use of the code, helping the reader gain confidence
in the concepts and their application. Along with complete routines for many Machine Learning methods,
the philosophy is to provide low level routines whose composition intuitively follows the mathematical de-
scription of the algorithm. In this way students may easily match the mathematics with the corresponding
algorithmic implementation.
IV DRAFT December 9, 2013
1: Probabilistic Reasoning
2: Basic Graph Concepts
3: Belief Networks
4: Graphical Models
5: Efficient Inference in Trees
6: The Junction Tree Algorithm
7: Making Decisions
8: Statistics for Machine Learning
9: Learning as Inference
10: Naive Bayes
11: Learning with Hidden Variables
12: Bayesian Model Selection
13: Machine Learning Concepts
14: Nearest Neighbour Classification
15: Unsupervised Linear Dimension Reduction
16: Supervised Linear Dimension Reduction
17: Linear Models
18: Bayesian Linear Models
19: Gaussian Processes
20: Mixture Models
21: Latent Linear Models
22: Latent Ability Models
23: Discrete-State Markov Models
24: Continuous-State Markov Models
25: Switching Linear Dynamical Systems
26: Distributed Computation
27: Sampling
28: Deterministic Approximate Inference
Graphical Models Course
Probabilistic Machine Learning Course
Approximate Inference Short Course
Time-series Short Course
Probabilistic Modelling Course
Part I:
Inference in Probabilistic Models
Part II:
Learning in Probabilistic Models
Part III:
Machine Learning
Part IV:
Dynamical Models
Part V:
Approximate Inference
Website
The BRMLtoolbox along with an electronic version of the book is available from
www.cs.ucl.ac.uk/staff/D.Barber/brml
Instructors seeking solutions to the exercises can find information at the website, along with additional
teaching materials.
DRAFT December 9, 2013 V
剩余671页未读,继续阅读
资源评论
- turgunn2015-01-14This is very good book about probablistic graphic theory.
- hang10272018-10-05nice!感谢分享!
videoandimage08
- 粉丝: 0
- 资源: 8
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功