没有合适的资源？快使用搜索试试~ 我知道了~

文库首页行业教育An introduction to RBM

An introduction to RBM

Restricted

Boltzman

Machine

需积分: 8 5 下载量 35 浏览量 2012-11-06 12:59:03 上传评论收藏 2.14MB PDF 举报

温馨提示

试读

130页

a complement material to bengio's deep machine learning for AI, detailed in restricted botlzman machine.

资源推荐

资源详情

资源评论

Foundations and Trends



Machine Learning

Vol. 2, No. 1 (2009) 1–127

 2009 Y. Bengio

DOI: 10.1561/2200000006

Learning Deep Architectures for AI

By Yoshua Bengio

Contents

1 Introduction 2

1.1 How do We Train Deep Architectures? 5

1.2 Intermediate Representations: Sharing Features and

Abstractions Across Tasks 7

1.3 Desiderata for Learning AI 10

1.4 Outline of the Paper 11

2 Theoretical Advantages of Deep Architectures 13

2.1 Computational Complexity 16

2.2 Informal Arguments 18

3 Local vs Non-Local Generalization 21

3.1 The Limits of Matching Local Templates 21

3.2 Learning Distributed Representations 27

4 Neural Networks for Deep Architectures 30

4.1 Multi-Layer Neural Networks 30

4.2 The Challenge of Training Deep Neural Networks 31

4.3 Unsupervised Learning for Deep Architectures 39

4.4 Deep Generative Architectures 40

4.5 Convolutional Neural Networks 43

4.6 Auto-Encoders 45

5 Energy-Based Models and Boltzmann Machines 48

5.1 Energy-Based Models and Products of Experts 48

5.2 Boltzmann Machines 53

5.3 Restricted Boltzmann Machines 55

5.4 Contrastive Divergence 59

6 Greedy Layer-Wise Training of Deep

Architectures 68

6.1 Layer-Wise Training of Deep Belief Networks 68

6.2 Training Stacked Auto-Encoders 71

6.3 Semi-Supervised and Partially Supervised Training 72

7 Variants of RBMs and Auto-Encoders 74

7.1 Sparse Representations in Auto-Encoders

and RBMs 74

7.2 Denoising Auto-Encoders 80

7.3 Lateral Connections 82

7.4 Conditional RBMs and Temporal RBMs 83

7.5 Factored RBMs 85

7.6 Generalizing RBMs and Contrastive Divergence 86

8 Stochastic Variational Bounds for Joint

Optimization of DBN Layers 89

8.1 Unfolding RBMs into Inﬁnite Directed

Belief Networks 90

8.2 Variational Justiﬁcation of Greedy Layer-wise Training 92

8.3 Joint Unsupervised Training of All the Layers 95

9 Looking Forward 99

9.1 Global Optimization Strategies 99

9.2 Why Unsupervised Learning is Important 105

9.3 Open Questions 106

10 Conclusion 110

Acknowledgments 112

References 113

Foundations and Trends



Machine Learning

Vol. 2, No. 1 (2009) 1–127

 2009 Y. Bengio

DOI: 10.1561/2200000006

Learning Deep Architectures for AI

Yoshua Bengio

Dept. IRO, Universit´e de Montr´eal, C.P. 6128, Montreal, Qc, H3C 3J7,

Canada, yoshua.bengio@umontreal.ca

Abstract

Theoretical results suggest that in order to learn the kind of com-

plicated functions that can represent high-level abstractions (e.g., in

vision, language, and other AI-level tasks), one may need deep architec-

tures. Deep architectures are composed of multiple levels of non-linear

operations, such as in neural nets with many hidden layers or in com-

plicated propositional formulae re-using many sub-formulae. Searching

the parameter space of deep architectures is a diﬃcult task, but learning

algorithms such as those for Deep Belief Networks have recently been

proposed to tackle this problem with notable success, beating the state-

of-the-art in certain areas. This monograph discusses the motivations

and principles regarding learning algorithms for deep architectures, in

particular those exploiting as building blocks unsupervised learning of

single-layer models such as Restricted Boltzmann Machines, used to

construct deeper models such as Deep Belief Networks.

Introduction

Allowing computers to model our world well enough to exhibit what

we call intelligence has been the focus of more than half a century of

research. To achieve this, it is clear that a large quantity of informa-

tion about our world should somehow be stored, explicitly or implicitly,

in the computer. Because it seems daunting to formalize manually all

that information in a form that computers can use to answer ques-

tions and generalize to new contexts, many researchers have turned

to learning algorithms to capture a large fraction of that information.

Much progress has been made to understand and improve learning

algorithms, but the challenge of artiﬁcial intelligence (AI) remains. Do

we have algorithms that can understand scenes and describe them in

natural language? Not really, except in very limited settings. Do we

have algorithms that can infer enough semantic concepts to be able to

interact with most humans using these concepts? No. If we consider

image understanding, one of the best speciﬁed of the AI tasks, we real-

ize that we do not yet have learning algorithms that can discover the

many visual and semantic concepts that would seem to be necessary to

interpret most images on the web. The situation is similar for other AI

tasks.

剩余129页未读，继续阅读

评论收藏

内容反馈

资源评论

资源反馈

评论星级较低，若资源使用遇到问题可联系上传者，3个工作日内问题未解决可申请退款~

lw860123

粉丝: 0
资源: 2

上传资源快速赚钱

我的内容管理展开

我的资源快来上传第一个资源

我的收益

登录查看自己的收益

我的积分登录查看自己的积分

我的C币登录后查看C币余额

我的收藏

我的下载

下载帮助

前往需求广场，查看用户热搜

An introduction to RBM

introduction

r introduction

RBM受限玻尔兹曼机

rbm _RBM代码_researchwyi_rbm代码思路_rbm_pyhon_

RBM代码实现MATLAB

matlab实现的RBM模型

RBM.m hinton

an introduction to R

RBM 读书笔记

深度学习笔记之RBM

AIS评测RBM训练性能

RBM 受限玻尔兹曼机

RBM相关论文，起源到综述

RBM-on-Classification RBM在分类上的应用

An introduction to R

introduction to R

波尔兹曼机RBM

CAN introduction

Maltab-RBM学习.rar

RBM程序协同过滤

Hiton RBM源程序

RBM_Tensorflow.py

RBM的python代码实现

最新资源