viii Contents
3.2.2. Abel Convergence 55
3.2.3.
Structure of Stationary Distributions 57
3.2.4. A Small Improvement 59
3.2.5.
The Mean Ergodic Theorem Again 61
3.2.6. A Refinement in The Aperiodic Case 62
3.2.7. Periodic Structure 65
3.3.
Exercises 67
Chapter 4 Markov Processes in Continuous Time 75
4.1.
Poisson Processes 75
4.1.1.
The Simple Poisson Process 75
4.1.2.
Compound Poisson Processes on
7L
d
77
4.2.
Markov Processes with Bounded Rates 80
4.2.1.
Basic Construction 80
4.2.2.
The Markov Property 83
4.2.3.
The Q-Matrix and Kolmogorov's Backward Equation .... 85
4.2.4. Kolmogorov's Forward Equation 86
4.2.5.
Solving Kolmogorov's Equation 86
4.2.6.
A Markov Process from its Infinitesimal Characteristics ... 88
4.3.
Unbounded Rates 89
4.3.1.
Explosion 90
4.3.2.
Criteria for Non-explosion or Explosion 92
4.3.3.
What to Do When Explosion Occurs 94
4.4.
Ergodic Properties 95
4.4.1.
Classification of States 95
4.4.2.
Stationary Measures and Limit Theorems 98
4.4.3.
Interpreting
%a
101
4.5.
Exercises 102
Chapter 5 Reversible Markov Processes 107
5.1.
Reversible Markov Chains 107
5.1.1.
Reversibility from Invariance 108
5.1.2. Measurements in Quadratic Mean 108
5.1.3.
The Spectral Gap 110
5.1.4. Reversibility and Periodicity 112
5.1.5. Relation to Convergence in Variation 113
5.2. Dirichlet Forms and Estimation of
/3
115
5.2.1.
The Dirichlet Form and Poincare's Inequality 115
5.2.2. Estimating /?+ 117
5.2.3.
Estimating /?_ 119
5.3.
Reversible Markov Processes in Continuous Time 120
5.3.1.
Criterion for Reversibility 120
5.3.2. Convergence in
L
2
(TV)
for Bounded Rates 121
5.3.3.
£
2
(-7r)-Convergence Rate in General 122
