Deep Learning in neural networks An overview-85

Contents 1. Introduction to Deep Learning (DL) in Neural Networks (NNs)...................................................................................................................................... 86 2. Event-oriented notation for activation spreading in NNs ............................................................................................................................................... 87 3. Depth of Credit Assignment Paths (CAPs) and of problems ............................................................................................................................................ 88 4. Recurring themes of Deep Learning.................................................................................................................................................................................. 88 4.1. Dynamic programming for Supervised/Reinforcement Learning (SL/RL).......................................................................................................... 88 4.2. Unsupervised Learning (UL) facilitating SL and RL.............................................................................................................................................. 89 4.3. Learning hierarchical representations through deep SL, UL, RL ......................................................................................................................... 89 4.4. Occam’s razor: compression and Minimum Description Length (MDL) ............................................................................................................ 89 4.5. Fast Graphics Processing Units (GPUs) for DL in NNs.......................................................................................................................................... 89 5. Supervised NNs, some helped by unsupervised NNs....................................................................................................................................................... 89 5.1. Early NNs since the 1940s (and the 1800s).......................................................................................................................................................... 90 5.2. Around 1960: visual cortex provides inspiration for DL (Sections 5.4, 5.11) .................................................................................................... 90 5.3. 1965: deep networks based on the Group Method of Data Handling................................................................................................................ 90 5.4. 1979: convolution + weight replication + subsampling (Neocognitron)......................................................................................................... 90 5.5. 1960–1981 and beyond: development of backpropagation (BP) for NNs ......................................................................................................... 90 5.5.1. BP for weight-sharing feedforward NNs (FNNs) and recurrent NNs (RNNs)...................................................................................... 91 5.6. Late 1980s–2000 and beyond: numerous improvements of NNs ...................................................................................................................... 91 5.6.1. Ideas for dealing with long time lags and deep CAPs........................................................................................................................... 91 5.6.2. Better BP through advanced gradient descent (compare Section 5.24).............................................................................................. 92 5.6.3. Searching for simple, low-complexity, problem-solving NNs (Section 5.24)..................................................................................... 92 5.6.4. Potential benefits of UL for SL (compare Sections 5.7, 5.10, 5.15)....................................................................................................... 92 5.7. 1987: UL through Autoencoder (AE) hierarchies (compare Section 5.15) ......................................................................................................... 93 5.8. 1989: BP for convolutional NNs (CNNs, Section 5.4) ........................................................................................................................................... 93 5.9. 1991: Fundamental Deep Learning Problem of gradient descent ...................................................................................................................... 93 5.10. 1991: UL-based history compression through a deep stack of RNNs................................................................................................................. 94 5.11. 1992: Max-Pooling (MP): towards MPCNNs (compare Sections 5.16, 5.19) ..................................................................................................... 94 5.12. 1994: early contest-winning NNs......................................................................................................................................................................... 95 5.13. 1995: supervised recurrent very Deep Learner (LSTM RNN).............................................................................................................................. 95 5.14. 2003: more contest-winning/record-setting NNs; successful deep NNs........................................................................................................... 96 5.15. 2006/7: UL for deep belief networks/AE stacks fine-tuned by BP ...................................................................................................................... 96 5.16. 2006/7: improved CNNs/GPU-CNNs/BP for MPCNNs/LSTM stacks .................................................................................................................... 96 5.17. 2009: first official competitions won by RNNs, and with MPCNNs.................................................................................................................... 97 5.18. 2010: plain backprop (+ distortions) on GPU breaks MNIST record ................................................................................................................. 97 5.19. 2011: MPCNNs on GPU achieve superhuman vision performance .................................................................................................................... 97 5.20. 2011: Hessian-free optimization for RNNs .......................................................................................................................................................... 98 5.21. 2012: first contests won on ImageNet, object detection, segmentation............................................................................................................ 98 5.22. 2013-: more contests and benchmark records .................................................................................................................................................... 98 5.23. Currently successful techniques: LSTM RNNs and GPU-MPCNNs...................................................................................................................... 99 5.24. Recent tricks for improving SL deep NNs (compare Sections 5.6.2, 5.6.3)......................................................................................................... 99 5.25. Consequences for neuroscience............................................................................................................................................................................100 5.26. DL with spiking neurons? .....................................................................................................................................................................................100 6. DL in FNNs and RNNs for Reinforcement Learning (RL) ..................................................................................................................................................100 6.1. RL through NN world models yields RNNs with deep CAPs ...............................................................................................................................100 6.2. Deep FNNs for traditional RL and Markov Decision Processes (MDPs)..............................................................................................................101 6.3. Deep RL RNNs for partially observable MDPs (POMDPs) ....................................................................................................................................101 6.4. RL facilitated by deep UL in FNNs and RNNs........................................................................................................................................................102 6.5. Deep hierarchical RL (HRL) and subgoal learning with FNNs and RNNs............................................................................................................102 6.6. Deep RL by direct NN search/policy gradients/evolution ...................................................................................................................................102 6.7. Deep RL by indirect policy search/compressed NN search .................................................................................................................................103 6.8. Universal RL............................................................................................................................................................................................................103 7. Conclusion and outlook .....................................................................................................................................................................................................103 Acknowledgments .............................................................................................................................................................................................................104 References...........................................................................................................................................................................................................................104