anintroductiontoreinforcementlearningbySutton资源-CSDN文库

Sutton

强化学习

需积分: 9 177 浏览量 2017-11-11 20:26:54 上传评论 1 收藏 49.14MB PDF 举报

资源推荐

资源详情

资源评论

Reinforcement Learning:

An Introduction

Second edition, in progress

****Complete Draft****

November 5, 2017

Richard S. Sutton and Andrew G. Barto

 2014, 2015, 2016, 2017

The text is now complete, except possibly for one more case study to be added to Chapter 16.The

references still need to be thoroughly checked, and an index still needs to b e added. Please send any

errors to rich@richsutton.com and barto@cs.umass.edu. We are also very interested in correcting any

important omissions in the “Bibliographical and Historical Remarks” at the end of each chapter. If

you think of something that really should have been cited, please let us know and we can try to get it

corrected before the ﬁnal version is printed.

ABradfordBook

The MIT Press

Cambridge, Massachusetts

London, England

iv CONTENTS

3.3 Returns and Episodes ..................................... 43

3.4 Uniﬁed Notation for Episodic and Continuing Tasks .................... 45

3.5 Policies and Value Functions ................................. 46

3.6 Optimal Policies and Optimal Value Functions ....................... 50

3.7 Optimality and Approximation ................................ 54

3.8 Summary ............................................ 55

4 Dynamic Programming 59

4.1 Policy Evaluation (Prediction) ................................ 60

4.2 Policy Improvement ...................................... 62

4.3 Policy Iteration ......................................... 64

4.4 Value Iteration ......................................... 67

4.5 Asynchronous Dynamic P r ogram mi ng ............................ 69

4.6 Generalized Policy Iteration .................................. 70

4.7 Eﬃciency of Dynamic Programming ............................. 71

4.8 Summary ............................................ 71

5 Monte Carlo Methods 75

5.1 Monte Carlo Predicti on .................................... 76

5.2 Monte Carlo Estimation of Action Values .......................... 79

5.3 Monte Carlo Control ...................................... 80

5.4 Monte Carlo Control without Exploring Starts ....................... 82

5.5 O↵-policy Prediction via Importance Sampling ....................... 84

5.6 Incremental Implementation .................................. 89

5.7 O↵-policy Monte Carlo Control ................................ 90

5.8 *Discounting-aware Importance Sampling .......................... 92

5.9 *Per-reward Importance Sampling .............................. 93

5.10 Summary ............................................ 94

6 Temporal-Di↵ eren ce Learning 97

6.1 TD Prediction ......................................... 97

6.2 Advantages of TD Prediction Methods ............................ 101

6.3 Optimality of TD(0) ...................................... 103

6.4 Sarsa: On-policy TD Control ................................. 105

6.5 Q-learning: O↵-policy TD Control .............................. 107

6.6 Expected Sarsa ......................................... 109

6.7 Maximization Bias and Double Learni ng ........................... 110

6.8 Games, Afterstates, and Other Special Cases ........................ 112

6.9 Summary ............................................ 113

7 n-step Bootstrapping 115

7.1 n-step TD Prediction ...................................... 115

CONTENTS v

7.2 n-step Sarsa ........................................... 119

7.3 n-step O↵-policy Learning by Importance Sampling .................... 121

7.4 *Per-reward O↵-policy Methods ................................ 122

7.5 O↵-policy Learning Without Importance Sampling:

The n-step Tree Backup Algorithm .............................. 124

7.6 *A Unifying Algorithm: n-step Q() ............................. 126

7.7 Summary ............................................ 129

8 Planning and Learning with Tabular M eth ods 131

8.1 Models and Planni n g ...................................... 131

8.2 Dyna: Integrati n g Planning, Acting, and Learning ..................... 133

8.3 When the Model Is Wrong ................................... 137

8.4 Prioritized Sweeping ...................................... 139

8.5 Expected vs. Sample Updates ................................. 142

8.6 Trajectory Sampling ...................................... 144

8.7 Real-time Dynamic Programming ............................... 146

8.8 Planning at Decision Time ................................... 149

8.9 Heuristic Search ........................................ 150

8.10 Rollout Algorithms ....................................... 152

8.11 Monte Carlo Tree Search .................................... 153

8.12 Summary of the Chapter .................................... 155

8.13 Summary of Part I: Dimensions ................................ 156

II Approximate Solution Me thods 160

9 On-p ol icy Prediction with Approximation 161

9.1 Value-function Approximation ................................. 161

9.2 The Prediction Objective (VE) ................................ 162

9.3 Stochastic-gradient and Semi-gradient Methods ....................... 164

9.4 Linear Methods ......................................... 167

9.5 Feature Construction for Linear Methods .......................... 171

9.5.1 Polynomials ....................................... 172

9.5.2 Fourier Basis ...................................... 173

9.5.3 Coarse Coding ..................................... 175

9.5.4 Tile Coding ....................................... 177

9.5.5 Radial Basis Function s ................................. 181

9.6 Nonlinear Function Appr oximation: Artiﬁcial Neural Networks .............. 182

9.7 Least-Squares TD ....................................... 186

9.8 Memory-based Function Ap p roximation ........................... 187

9.9 Kernel-based Function Approximation ............................ 189

9.10 Looking Deeper at On-policy Learning: Interest and Emphasis .............. 190

剩余444页未读，继续阅读

评论收藏

内容反馈

littleredhat31415

粉丝: 0
资源: 11

an introduction to reinforcement learning by Sutton

最新资源

an introduction to reinforcement learning by Sutton

An Introduction to Reinforcement Learning

增强学习导论（Reinforcement Learning An Introduction_Sutton）

reinforcement learning

Reinforcement Learning: An Introduction，Richard S. Sutton and Andrew G. Barto

Reinforcement Learning: An Introduction最新版习题解答（第一版本）

强化学习 reinforcement learning

Reinforcement Learning 2nd(Richard_S._Sutton).pdf

Reinforcement Learning(Second edition)-sutton (draft2018)

Reinforcement learning合集

reinforcement learning: an introduction

reinforcement learning：an introduction代码

reinforcement-learning-an-introduction-master.rar

reinforcement-learning-an-introduction-master.zip

reinforcement-learning-an-introduction-chinese:《强化学习

Reinforcement learning an introduction 2nd edition

Reinforcement Learning高清英文原版

Introduction to Machine Learning - Nils J Nilsson.pdf

reinforcement learning: an introduction / 强化学习在阿里的技术演进与业务创新

reinforcement-learning-an-introduction-PDF和源码（2017版）

reinforcement-learning-an-introduction-java:强化学习的Java实现

reinforcement learning: an introduction (2nd edition) 第二版正式版

reinforcement learning ：an introduction 2018最新版book pdf格式

reinforcement-learning-notes:Sutton和Barto 2015年书籍《强化学习》的笔记和练习解决方案

reinforcement-learning-an-introduction(2018年3月最新版550页)+配套源码+配套David Silver RL课件

YOLOv8-deepsort 实现智能车辆目标检测+车辆跟踪+车辆计数

YOLOv8网络结构图，自制visio文件，yolov8.vsds，需要的自取，在原有的基础上直接改就行了

yolov8(2023年8月版本),已经下好yolov8s.pt和yolov8n.pt

Transformer模型实现长期预测并可视化结果（附代码+数据集+原理介绍）

最新资源