Probabilistic Graphical Models
Convolutional and Sequential
Modeling in DL
Xiaodan Liang
Lecture 20, 2017
Reading: see class website
1
© Eric Xing @ CMU, 2017
Overview
l Convolutional neural networks
l Recurrent neural networks
l Memory and attention mechanisms
l Deep Reinforcement Learning
l Conclusion
© Eric Xing @ CMU, 2017
2
Overview
l Convolutional neural networks
l Recurrent neural networks
l Memory and attention mechanisms
l Deep Reinforcement Learning
l Applications in computer vision
© Eric Xing @ CMU, 2017
3
Convolutional neural networks
l Biologically-inspired variants of MLPs
l Receptive field: visual cortex contains a complex arrangement of
cells. These cells are sensitive to small sub-regions of the visual
field. [Hubel & Wiesel 1962] & [Fukushima 1982] (Neocognitron)
l The sub-regions are tiled to cover the entire visual field ---
Hierarchical Representation
© Eric Xing @ CMU, 2017
4
Local Filters
exploit the strong spatially local correlation present in natural images
the figure is courtesy of Yann LeCun
[LeCun et al. NIPS 1989]
Convolutional neural networks
© Eric Xing @ CMU, 2015
5
l Sparse Connectivity
l Shared weights
l Increasingly “global” receptive fields
Feature maps ! "#
Feature maps !
Feature maps ! $%
Ø simple cells detect local features
Ø complex cells “pool” the outputs of simple cells within a retinotopic
neighborhood.
