Foundations and Trends
R
in
Machine Learning
Vol. 1, No. 4 (2008) 403–565
c
2009 S. Mahadevan
DOI: 10.1561/2200000003
Learning Representation and Control
in Markov Decision Processes: New Frontiers
By Sridhar Mahadevan
Contents
1 Introduction 404
1.1 Motivation 404
1.2 Laplacian Operators 409
1.3 Dimensionality Reduction of MDPs 412
1.4 Roadmap to the Paper 416
2 Sequential Decision Problems 418
2.1 Markov Decision Processes 419
2.2 Exact Solution Methods 429
2.3 Simulation-Based Methods 432
3 Laplacian Operators and Markov Decision
Processes 435
3.1 Laplacian Operators 436
3.2 Laplacian Matrices in MDPs 437
3.3 Generalized Inverses of the Laplacian 439
3.4 Positive-Semidefinite Laplacian Matrices 450
4 Approximating Markov Decision Processes 458
4.1 Linear Value Function Approximation 458
