■ 4.8 Summary
■ 4.9 Bibliographical and Historical Remarks
❍ 5. Monte Carlo Methods
■ 5.1 Monte Carlo Policy Evaluation
■ 5.2 Monte Carlo Estimation of Action Values
■ 5.3 Monte Carlo Control
■ 5.4 On-Policy Monte Carlo Control
■ 5.5 Evaluating One Policy While Following Another
■ 5.6 Off-Policy Monte Carlo Control
■ 5.7 Incremental Implementation
■ 5.8 Summary
■ 5.9 Bibliographical and Historical Remarks
❍ 6. Temporal-Difference Learning
■ 6.1 TD Prediction
■ 6.2 Advantages of TD Prediction Methods
■ 6.3 Optimality of TD(0)
■ 6.4 Sarsa: On-Policy TD Control
■ 6.5 Q-Learning: Off-Policy TD Control
■ 6.6 Actor-Critic Methods
■ 6.7 R-Learning for Undiscounted Continuing Tasks
■ 6.8 Games, Afterstates, and Other Special Cases
■ 6.9 Summary
■ 6.10 Bibliographical and Historical Remarks
● III. A Unified View
❍ 7. Eligibility Traces
■ 7.1 -Step TD Prediction
■ 7.2 The Forward View of TD( )
■ 7.3 The Backward View of TD( )
■ 7.4 Equivalence of Forward and Backward Views
■ 7.5 Sarsa( )
■ 7.6 Q( )
■ 7.7 Eligibility Traces for Actor-Critic Methods
■ 7.8 Replacing Traces
■ 7.9 Implementation Issues
■ 7.10 Variable
■ 7.11 Conclusions
■ 7.12 Bibliographical and Historical Remarks
❍ 8. Generalization and Function Approximation
■ 8.1 Value Prediction with Function Approximation
■ 8.2 Gradient-Descent Methods
http://www.cs.ualberta.ca/%7Esutton/book/ebook/the-book.html (3 di 4)22/06/2005 9.04.27
