Volume 2
Andrew Glassner
DEEP LEARNING:
From Basics
to Practice
Deep Learning:
From Basics to Practice
Volume 2
Copyright (c) 2018 by Andrew Glassner
www.glassner.com / @AndrewGlassner
All rights reserved. No part of this book, except as noted below, may be reproduced,
stored in a retrieval system, or transmitted in any form or by any means, without
the prior written permission of the author, except in the case of brief quotations
embedded in critical articles or reviews.
The above reservation of rights does not apply to the program les associated with
this book (available on GitHub), or to the images and gures (also available on
GitHub), which are released under the MIT license. Any images or gures that are
not original to the author retain their original copyrights and protections, as noted
in the book and on the web pages where the images are provided.
All software in this book, or in its associated repositories, is provided “as is,” with-
out warranty of any kind, express or implied, including but not limited to the
warranties of merchantability, tness for a particular pupose, and noninfringe-
ment. In no event shall the authors or copyright holders be liable for any claim,
damages or other liability, whether in an action of contract, tort, or otherwise,
arising from, out of or in connection with the software or the use or other dealings
in the software.
First published February 20, 2018
Version 1.0.1 March 3, 2018
Version 1.1 March 22, 2018
Published by The Imaginary Institute, Seattle, WA.
http://www.imaginary-institute.com
Contact: andrew@imaginary-institute.com
For Niko,
who’s always there
with a smile
and a wag.
Contents of Both Volumes
Volume 1
Preface ....................................................................i
Chapter 1: An Introduction ...................................1
1.1 Why This Chapter Is Here ...............................3
1.1.1 Extracting Meaning from Data ............................ 4
1.1.2 Expert Systems ..................................................... 6
1.2 Learning from Labeled Data ..........................9
1.2.1 A Learning Strategy .............................................. 10
1.2.2 A Computerized Learning Strategy ................... 12
1.2.3 Generalization ...................................................... 16
1.2.4 A Closer Look at Learning ................................... 18
1.3 Supervised Learning ........................................21
1.3.1 Classication ......................................................... 21
1.3.2 Regression ............................................................. 22
1.4 Unsupervised Learning ...................................25
1.4.1 Clustering .............................................................. 25
1.4.2 Noise Reduction ................................................... 26
1.4.3 Dimensionality Reduction .................................. 28
1.5 Generators ........................................................32
1.6 Reinforcement Learning .................................34
1.7 Deep Learning ..................................................37
1.8 What’s Coming Next ....................................... 43
References ..............................................................44
Image credits ................................................................. 45
Chapter 2: Randomness and Basic Statistics .....46
2.1 Why This Chapter Is Here ...............................48
2.2 Random Variables ........................................... 49
2.2.1 Random Numbers in Practice............................. 57
2.3 Some Common Distributions ........................59
2.3.1 The Uniform Distribution ................................... 60
2.3.2 The Normal Distribution .................................... 61
2.3.3 The Bernoulli Distribution ................................. 67
2.3.4 The Multinoulli Distribution .............................. 69
2.3.5 Expected Value .................................................... 70
2.4 Dependence ....................................................70
2.4.1 i.i.d. Variables ........................................................ 71
2.5 Sampling and Replacement ...........................71
2.5.1 Selection With Replacement .............................. 73
2.5.2 Selection Without Replacement ....................... 74
2.5.3 Making Selections ............................................... 75
2.6 Bootstrapping .................................................76
2.7 High-Dimensional Spaces ..............................82
2.8 Covariance and Correlation ...........................85
2.8.1 Covariance ............................................................ 86
2.8.2 Correlation ........................................................... 88
2.9 Anscombe’s Quartet .......................................93
References ..............................................................95
