Deep Neural Networks in a Mathematical Framework-Springer(2018)....

Over the past decade, Deep Neural Networks (DNNs) have become very popular models for problems involving massive amounts of data. The most successful DNNs tend to be characterized by several layers of parametrized linear and nonlinear transformations, such that the model contains an immense number of parameters. Empirically, we can see that networks structured according to these ideals perform well in practice. However, at this point we do not have a full rigorous understanding of why DNNs work so well, and how exactly to construct neural networks that perform well for a specific problem. This book is meant as a first step towards forming this rigorous understanding: we develop a generic mathematical framework for representing neural networks and demonstrate how this framework can be used to represent specific neural network architectures. We hope that this framework will serve as a common mathematical language for theoretical neural network researchers—something which currently does not exist—and spur further work into the analytical properties of DNNs. We begin in Chap. 1 by providing a brief history of neural networks and exploring mathematical contributions to them. We note what we can rigorously explain about DNNs, but we will see that these results are not of a generic nature. Another topic that we investigate is current neural network representations: we see that most approaches to describing DNNs rely upon decomposing the parameters and inputs into scalars, as opposed to referencing their underlying vector spaces, which adds a level of awkwardness into their analysis. On the other hand, the framework that we will develop strictly operates over these vector spaces, affording a more natural mathematical description of DNNs once the objects that we use are well defined and understood.