神经网络和统计学习(Neural networks and statistical learning) by K.-L. Du and M.N.s. Swamy

所需积分/C币:32 2018-06-21 13:33:58 13.11MB PDF

神经网络和统计学习(Neural networks and statistical learning) by K.-L. Du and M.N.s. Swamy
Ke-Lin Du. M. N.s. Swamy Neural networks and Statistical Learning 空 Springer Ke-Lin du M.N.S. Sw Enjoyor labs Department of electrical and computer Enjoyor Inc. Engineering Hangzhou Concordia university China Montreal, QC Canada ang Department of Electrical and Computer Engineering Concordia University Montreal, QC anada Additionalmaterialtothisbookcanbedownloadedfromhttp://extras.springer.com. ISBN978-1-4471-5570-6 ISBN978-1-4471-5571-3( eBook) DOI10.1007/978-1-4471-5571-3 Springer London Heidelberg New York Dordrecht Library of Congress Control Number: 2013948860 o Springer-Verlag London 2014 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publishers location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general u While the advice and information in this book are believed to be true and accurate at the date o publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein Printed on acid-free paper SpringerispartofSpringerScience+businessMedia(www.springer.com) In memory of my grandparents K-L. D To my family M.N.S. Swamy To all the researchers with original contri butions to neural networks and machine learnin g K-L Du, and m.n. s swamy Preface The human brain, consisting of nearly 10 neurons, is the center of human intelligence. Human intelligence has been simulated in various ways. Artificial intelligence(AD) pursues exact logical reasoning based on symbol manipulation Fuzzy logics model the highly uncertain behavior of decision making. Neural networks model the highly nonlinear infrastructure of brain networks. evolu tionary computation models the evolution of intelligence. Chaos theory models the highly nonlinear and chaotic behaviors of human intelligence Softcomputing is an evolving collection of methodologies for the representation of ambiguity in human thinking; it exploits the tolerance for imprecision and uncertainty, approximate reasoning, and partial truth in order to achieve tracta bility, robustness, and low-coSt solutions. The major methodologies of softcom- puting are fuzzy logic, neural networks, and evolutionary computation Conventional model-based data-processing methods require experts knowl- edge for the modeling of a system. Neural network methods provide a model-free, adaptive, fault tolerant, parallel and distributed processing solution. A neural network is a black box that directly learns the internal relations of an unknown system, without guessing functions for describing cause-and-effect relationships The neural network approach is a basic methodology of information processing Neural network models may be used for function approximation, classification nonlinear mapping, associative memory, vector quantization, optimization, feature extraction, clustering, and approximate inference. Neural networks have wide applications in almost all areas of science and engineering Fuzzy logic provides a means for treating uncertainty and computing with words. This mimics human recognition which skillfully copes with uncertainty Fuzzy systems are conventionally created from explicit knowledge expressed in the form of fuzzy rules, which are designed based on experts'experience. a fuzzy system can explain its action by fuzzy rules. Neurofuzzy systems, as a synergy of fuzzy logic and neural networks, possess both learning and knowledge represen- tation capabilities This book is our attempt to bring together the major advances in neural net- works and machine learning and to explain them in a statistical framework while some mathematical details are needed we emphasize the practical aspects of the models and methods rather than the theoretical details. To us. neural networks are merely some statistical methods that can be represented by graphs and networks Preface They can iteratively adjust the network parameters. As a statistical model, a neura network can learn the probability density function from the given samples, and then predict, by generalization according to the learnt statistics, outputs for new samples that are not included in the learning sample set The neural network approach is a general statistical computational paradigm Neural network research solves two problems: the direct problem and the inverse problem. The direct problem employs computer and engineering techniques to model biological neural systems of the human brain. This problem is investigated by cognitive scientists and can be useful in neuropsychiatry and neurophysiology The inverse problem simulates biological neural systems for their problem-solving capabilities for application in scientific or engineering fields. Engineering and computer scientists have conducted extensive investigation in this area. This book concentrates mainly on the inverse problem, although the two areas often shed light on each other. The biological and psychological plausibility of the neural network models have not been seriously treated in this book though some back ground material is discussed This book is intended to be used as a textbook for advanced undergraduate and graduate students in engineering, science, computer science, business, arts, and medicine. It is also a good reference book for scientists, researchers, and practi tioners in a wide variety of fields and assumes no previous knowledge of neural network or machine learning concepts This book is divided into 25 chapters and two appendices. It contains almost all the major neural network models and statistical learning approaches. We also give an introduction to fuzzy sets and logic, and neurofuzzy models. Hardware mplementations of the models are discussed. Two chapters are dedicated to the applications of neural network and statistical learning approaches to biometrics/ bioinformatics and data mining. Finally, in the appendices, some mathematical preliminaries are given, and benchmarks for validating all kinds of neural network methods and some web resources are provided First and foremost we would like to thank the supporting staff from Springer London, especially Anthony Doyle and Grace Quinn for their enthusiastic and professional support throughout the period of manuscript preparation K.-L. Du also wishes to thank Jiabin Lu guangdong University of Technology China), Jie Zeng(Richcon MC, InC, China), Biaobiao Zhang and Hui wang (Enjoyor, Inc, China), and many of his graduate students including Na Shou Shengfeng Yu, Lusha Han, Xiaolan Shen, Yuanyuan Chen, and Xiaoling Wang (Zhejiang University of Technology, China) for their consistent assistance In addition, we should mention at least the following names for their help Omer Morgul(Bilkent University, Turkey ), Yanwu Zhang (Monterey Bay Aquarium Research Institute, USA), Chi Sing Leung(City University of Hong Kong, Hong Kong), M. Omair Ahmad and Jianfeng Gu( Concordia University Canada), Li Yu, Limin Meng, Jingyu Hua, Zhijiang Xu, and Luping Fang(zhe jiang University of Technology, China), Yuxing Dai (Wenzhou university, China), and Renwang Li(zhejiang sci-Tech University, China). Last, but not Preface least, we would like to thank our families for their support and understanding during the course of writing this book a book of this length is certain to have some errors and omissions. Feedback is welcome via email at kldu @ieee. org or swamy @encs. concordia. ca. MATLAB code for the worked examples is downloadable from the website of this book Hangzhou, China K-L. Du Montreal Canada M. Ns Swamy Contents 1 ntroduction Major Events in Neural Networks Research 1.2 Neurons 1.2.1 The McCulloch-Pitts Neuron model 1. 2. 2 Spiking Neuron Models 1. 3 Neural Networks 5682 1.4 Scope of the book References 13 2 Fundamentals of Machine Learning 15 1 Learning Methods 2.2 Learning and Generalization 19 2.2.1 Generalization error 21 2.2.2 Generalization by Stopping Criterion 21 2.2.3 Generalization by regularization 2.2.4 Fault Tolerance and generalization 24 2.2.5 Sparsity Versus Stability 2. 3 Model selection 5 2.3.1 Crossvalidation 26 2.3.2 Complexity Criteria 28 2.4 Bias andⅤ ariance. 29 2.5 Rob g 2. 6 Neural Network processors 33 2.7 Criterion functions 36 2.8 Computational Learning Theory 39 2.8.1 Vapnik-Chervonenkis Dimension 40 2.8.2 Empirical risk-Minimization Principle 41 2.8.3 Probably Approximately Correct Learning 2.9 No-Free-Lunch th 44 2.10 Neural Networks as universal machines 45 2.10.1 Boolean Function Approximation 45 2.10.2 Linear Separability and Nonlinear Separability.. 47 2.10.3 Continuous Function Approximation 49 2. 10.4 Winner-TakeS-All 50 Xll Contents 2.11 Compressed Sensing and Sparse Approximation 51 2.11.1 Compressed Sensing 51 2. 11.2 Sparse Approximation 53 2.11.3 LASSO and greedy pursuit 54 2. 12 Bibliographical Notes References 59 3 Perceptrons 3.1 One-Neuron Perceptro 67 3.2 Single-Layer Perceptron 68 3.3 Perceptron Learning Algorithm 3.4 Least-Mean Squares (LMS) Algorithm P-Delta rule 74 3.6 Other Learning Algorithms 76 References 79 4 Multilayer Perceptrons: Architecture and Error backpropagation 4.1 Introduction 4.2 Universal Approximation 84 4.3 Backpropagation Learning Algorithm 85 4.4 Incremental Learning Versus Batch Learning Activation Functions for the Output Layer 4.6 Optimizing network structure 96 4.6.1 Network Pruning Using Sensitivity Analysis 96 4.6.2 Network Pruning USing Regularization 4.6.3 Network Growing 4.7 Speeding Up Learning Process 102 4.7.1 Eliminating Premature Saturation 102 4.7.2 Adapting Learning Par 10 4.7.3 Initializing Weights .108 4.7.4 Adapting Activation Function .110 4.8 Some Improved bp algorithms 112 4.8.1 BP with global descent 113 4.8.2 Robust bp algorithms 115 4.9 Resilient Propagation(RProp) 115 References 119 5 Multilayer Perceptrons: Other Learning Techniques 127 5.1 Introduction to Second-Order learning methods 127 5.2 Newton's methods 128 5.2.1 Gauss-Newton Method 129 5.2.2 Levenberg-Marquardt Method .130


评论 下载该资源后可以进行评论 1

妖孽横生 看了一下。还不错,有一些还是有参考价值的

关注 私信 TA的资源