BayesianFiltering：FromKalmanFilterstoParticleFilters,andBeyond_贝叶斯信号处理:经典、现代与粒子滤波方法(第二版)资源-CSDN文库

需积分: 50 56 浏览量 2018-03-01 09:51:21 上传评论 2 收藏 1.19MB PDF 举报

贝叶斯滤波是信号处理和统计学领域中一种重要的算法，它从卡尔曼滤波发展到粒子滤波，并不断拓展新的应用方向。本文将详细探讨贝叶斯滤波的演变历程及其在现代技术中的应用。贝叶斯滤波的根源可以追溯到随机滤波理论。随机滤波理论强调了在非线性和非高斯噪声环境下的信号滤波问题。在贝叶斯框架下，根据不同的场景可以发展出不同的贝叶斯滤波技术。在一种特定的线性二次高斯环境下，著名的卡尔曼滤波器可以被推导出来。卡尔曼滤波器是基于最小均方误差估计而设计的，它假设系统噪声和观测噪声都服从高斯分布。贝叶斯滤波技术的进一步发展包括了非线性滤波技术。文章详细讨论了基于序贯蒙特卡洛采样（Sequential Monte Carlo Sampling）的贝叶斯滤波方法，即粒子滤波器（Particle Filters）。粒子滤波器通过构建一组在状态空间中传播的随机样本（粒子），以近似后验概率密度函数，从而实现对动态系统状态的估计。由于其能够处理复杂的非线性与非高斯问题，粒子滤波器在许多实际应用中表现出了其优越性。文章还深入探讨了粒子滤波器的多种变体以及它们各自的特点（优点和缺点）。例如，自适应粒子滤波器能够通过增减粒子数量来应对状态空间的动态变化，而重采样粒子滤波器则通过增加在高似然区域的粒子密度来改善滤波效果。除此之外，文章还探讨了与粒子滤波相关的理论和实际问题，例如粒子退化和粒子稀疏问题，以及如何通过改进算法来缓解这些问题。文章最后还探索了一些贝叶斯滤波的其他新方向，比如贝叶斯网络和深度学习在贝叶斯滤波中的应用。贝叶斯网络可以为状态空间模型提供结构化表示，从而捕捉变量之间的依赖关系。而深度学习，特别是深度神经网络，可以用来提取数据的复杂特征，从而进一步提高滤波器的性能。在数学预备和问题公式化方面，文档首先介绍了随机滤波理论的预备知识，包括随机过程、滤波问题和非线性滤波。非线性随机滤波被视为一种不适定的逆问题，这意味着它通常不存在唯一解。文章接着讨论了随机微分方程及其与滤波的关系。在贝叶斯统计和贝叶斯估计方面，文档探讨了贝叶斯理论的基本原理，并对如何在不同的情况下应用贝叶斯推断进行了说明。贝叶斯推断利用先验概率和样本数据来计算后验概率，它是一种描述不确定性并从数据中学习的强有力工具。文章还涉及了蒙特卡洛方法和蒙特卡洛滤波。蒙特卡洛方法是一种通过随机采样来近似计算概率分布的方法。在贝叶斯滤波中，蒙特卡洛方法允许通过粒子集合来近似后验概率，从而在状态空间模型中实现有效的序贯状态估计。文档强调了贝叶斯理论在现代统计推断中的重要性，并引用了贝叶斯和C.J.Bradfield的观点来说明贝叶斯理论的核心理念。贝叶斯理论认为，事件的概率是基于该事件发生时所期望的某个量的值与该量实际值之间的比值。而统计的核心在于无需承认自己是错误的，因为任何两个统计学家之间总会有方差存在。本文详细地回顾了贝叶斯滤波的发展历程和其在理论与实践中的应用。通过对卡尔曼滤波器和粒子滤波器的深入分析，读者可以对贝叶斯滤波的原理和技术有更全面的理解。同时，文章还提供了其他新兴方向的展望，为贝叶斯滤波的进一步研究指明了方向。

资源推荐

资源详情

资源评论

MANUSCRIPT 1

Bayesian Filtering: From Kalman Filters to

Particle Filters, and Beyond

ZHE CHEN

Abstract— In this self-contained survey/review paper, we system-

atically investigate the roots of Bayesian ﬁltering as well as its rich

leaves in the literature. Stochastic ﬁltering theory is brieﬂy reviewed

with emphasis on nonlinear and non-Gaussian ﬁltering. Following

the Bayesian statistics, diﬀerent Bayesian ﬁltering techniques are de-

veloped given diﬀerent scenarios. Under linear quadratic Gaussian

circumstance, the celebrated Kalman ﬁlter can be derived within the

Bayesian framework. Optimal/suboptimal nonlinear ﬁltering tech-

niques are extensively investigated. In particular, we focus our at-

tention on the Bayesian ﬁltering approach based on sequential Monte

Carlo sampling, the so-called particle ﬁlters. Many variants of the

particle ﬁlter as well as their features (strengths and weaknesses) are

discussed. Related theoretical and practical issues are addressed in

detail. In addition, some other (new) directions on Bayesian ﬁltering

are also explored.

Index Terms— Stochastic ﬁltering, Bayesian ﬁltering,

Bayesian inference, particle ﬁlter, sequential Monte Carlo,

sequential state estimation, Monte Carlo methods.

“The probability of any event is the ratio between the

value at which an expectation depending on the happening

of the event ought to be computed, and the value of the

thing expected upon its happening.”

— Thomas Bayes (1702-1761), [29]

“Statistics is the art of never having to say you’re

wrong. Variance is what any two statisticians are at.”

—C.J.Bradﬁeld

Contents

I Introduction 2

I-A StochasticFilteringTheory ............... 2

I-B BayesianTheoryandBayesianFiltering ........ 2

I-C Monte Carlo Methods and Monte Carlo Filtering . . . 2

I-D OutlineofPaper ..................... 3

II Mathematical Preliminaries and Problem Formula-

tion 4

II-APreliminaries ....................... 4

II-BNotations ......................... 4

II-CStochasticFilteringProblem .............. 4

II-D Nonlinear Stochastic Filtering Is an Ill-posed Inverse

Problem.......................... 5

II-D.1InverseProblem ................. 5

II-D.2 Diﬀerential Operator and Integral Equation . . 6

II-D.3RelationstoOtherProblems .......... 7

II-EStochasticDiﬀerentialEquationsandFiltering .... 7

III Bayesian Statistics and Bayesian Estimation 8

III-ABayesianStatistics .................... 8

III-BRecursiveBayesianEstimation ............. 9

The work is supported by the Natural Sciences and Engineering

Research Council of Canada. Z. Chen was also partially supported

by Clifton W. Sherman Scholarship.

The author is with the Communications Research Laboratory,

McMaster University, Hamilton, Ontario, Canada L8S 4K1, e-

mail: zhechen@soma.crl.mcmaster.ca, Tel: (905)525-9140 x27282,

Fax:(905)521-2922.

IV Bayesian Optimal Filtering 9

IV-AOptimalFiltering..................... 10

IV-BKalmanFiltering..................... 11

IV-COptimumNonlinearFiltering .............. 13

IV-C.1Finite-dimensionalFilters ............ 13

V Numerical Approximation Methods 14

V-A Gaussian/Laplace Approximation ............ 14

V-BIterativeQuadrature................... 14

V-C Mulitgrid Method and Point-Mass Approximation . . 14

V-D Moment Approximation ................. 15

V-E Gaussian Sum Approximation . ............. 16

V-F Deterministic Sampling Approximation . . ....... 16

V-G Monte Carlo Sampling Approximation . . ....... 17

V-G.1ImportanceSampling .............. 18

V-G.2RejectionSampling................ 19

V-G.3SequentialImportanceSampling ........ 19

V-G.4Sampling-ImportanceResampling ....... 20

V-G.5StratiﬁedSampling................ 21

V-G.6MarkovChainMonteCarlo ........... 22

V-G.7HybridMonteCarlo ............... 23

V-G.8Quasi-MonteCarlo................ 24

VI Sequential Monte Carlo Estimation: Particle Filters 25

VI-ASequentialImportanceSampling(SIS)Filter ..... 26

VI-BBootstrap/SIRﬁlter ................... 26

VI-CImprovedSIS/SIRFilters ................ 27

VI-DAuxiliary Particle Filter ................. 28

VI-ERejectionParticleFilter ................. 29

VI-F Rao-Blackwellization ................... 30

VI-GKernelSmoothingandRegularization ......... 31

VI-HDataAugmentation ................... 32

VI-H.1 Data Augmentation is an Iterative Kernel

SmoothingProcess................ 32

VI-H.2 Data Augmentation as a Bayesian Sampling

Method ...................... 33

VI-I MCMC Particle Filter .................. 33

VI-JMixtureKalmanFilters ................. 34

VI-KMixtureParticleFilters ................. 34

VI-LOtherMonteCarloFilters................ 35

VI-MChoicesofProposalDistribution ............ 35

VI-M.1PriorDistribution ................ 35

VI-M.2Annealed Prior Distribution ........... 36

VI-M.3Likelihood..................... 36

VI-M.4Bridging Density and Partitioned Sampling . . 37

VI-M.5Gradient-BasedTransitionDensity....... 38

VI-M.6EKFasProposalDistribution.......... 38

VI-M.7UnscentedParticleFilter ............ 38

VI-NBayesianSmoothing ................... 38

VI-N.1Fixed-pointsmoothing.............. 38

VI-N.2Fixed-lagsmoothing ............... 39

VI-N.3Fixed-intervalsmoothing ............ 39

VI-OLikelihoodEstimate ................... 40

VI-PTheoreticalandPracticalIssues............. 40

VI-P.1ConvergenceandAsymptoticResults ..... 40

VI-P.2Bias-Variance................... 41

VI-P.3Robustness .................... 43

VI-P.4AdaptiveProcedure ............... 46

MANUSCRIPT 2

VI-P.5EvaluationandImplementation......... 46

VIIOther Forms of Bayesian Filtering and Inference 47

VII-AConjugate Analysis Approach .............. 47

VII-BDiﬀerential Geometrical Approach . .......... 47

VII-CInteractingMultipleModels............... 48

VII-DBayesian Kernel Approaches ............... 48

VII-EDynamicBayesianNetworks............... 48

VIIISelected Applications 49

VIII-ATargetTracking...................... 49

VIII-BComputerVisionandRobotics ............. 49

VIII-CDigitalCommunications................. 49

VIII-DSpeechEnhancementandSpeechRecognition..... 50

VIII-EMachineLearning..................... 50

VIII-FOthers........................... 50

VIII-GAnIllustrativeExample:Robot-ArmProblem..... 50

IX Discussion and Critique 51

IX-AParameterEstimation .................. 51

IX-BJointEstimationandDualEstimation......... 51

IX-CPrior............................ 52

IX-DLocalizationMethods .................. 52

IX-EDimensionalityReductionandProjection ....... 53

IX-FUnansweredQuestions.................. 53

X Summary and Concluding Remarks 55

I. Introduction

HE contents of this paper contain three major scien-

tiﬁc areas: stochastic ﬁltering theory, Bayesian theory,

and Monte Carlo methods. All of them are closely discussed

around the subject of our interest: Bayesian ﬁltering. In

the course of explaining this long story, some relevant the-

ories are brieﬂy reviewed for the purpose of providing the

reader a complete picture. Mathematical preliminaries and

background materials are also provided in detail for the

self-containing purpose.

A. Stochastic Filtering Theory

Stochastic ﬁltering theory was ﬁrst established in the

early 1940s due to the pioneering work by Norbert Wiener

[487], [488] and Andrey N. Kolmogorov [264], [265], and it

culminated in 1960 for the publication of classic Kalman

ﬁlter (KF) [250] (and subsequent Kalman-Bucy ﬁlter in

1961 [249]),

though many credits should be also due to

some earlier work by Bode and Shannon [46], Zadeh and

Ragazzini [502], [503], Swerling [434], Levinson [297], and

others. Without any exaggeration, it seems fair to say

that the Kalman ﬁlter (and its numerous variants) have

dominated the adaptive ﬁlter theory for decades in signal

processing and control areas. Nowadays, Kalman ﬁlters

have been applied in the various engineering and scientiﬁc

areas, including communications, machine learning, neu-

roscience, economics, ﬁnance, political science, and many

others. Bearing in mind that Kalman ﬁlter is limited by its

assumptions, numerous nonlinear ﬁltering methods along

Another important event in 1960 is the publication of the cele-

brated least-mean-squares (LMS) algorithm [485]. However, the LMS

ﬁlter is not discussed in this paper, the reader can refer to [486], [205],

[207], [247] for more information.

its line have been proposed and developed to overcome its

limitation.

B. Bayesian Theory and Bayesian Filtering

Bayesian theory

was originally discovered by the British

researcher Thomas Bayes in a posthumous publication in

1763 [29]. The well-known Bayes theorem describes the

fundamental probability law governing the process of log-

ical inference. However, Bayesian theory has not gained

its deserved attention in the early days until its modern

form was rediscovered by the French mathematician Pierre-

Simon de Laplace in Th´eorie analytique des probailit´es.

Bayesian inference [38], [388], [375], devoted to applying

Bayesian statistics to statistical inference, has become one

of the important branches in statistics, and has been ap-

plied successfully in statistical decision, detection and es-

timation, pattern recognition, and machine learning. In

particular, the November 19 issue of 1999 Science mag-

azine has given the Bayesian research boom a four-page

special attention [320]. In many scenarios, the solutions

gained through Bayesian inference are viewed as “optimal”.

Not surprisingly, Bayesian theory was also studied in the

ﬁltering literature. One of the ﬁrst exploration of itera-

tive Bayesian estimation is found in Ho and Lee’ paper

[212], in which they speciﬁed the principle and procedure

of Bayesian ﬁltering. Sprangins [426] discussed the itera-

tive application of Bayes rule to sequential parameter esti-

mation and called it as “Bayesian learning”. Lin and Yau

[301] and Chien an Fu [92] discussed Bayesian approach

to optimization of adaptive systems. Bucy [62] and Bucy

and Senne [63] also explored the point-mass approximation

method in the Bayesian ﬁltering framework.

C. Monte Carlo Methods and Monte Carlo Filtering

The early idea of Monte Carlo

can be traced back to

the problem of Buﬀon’s needle when Buﬀon attempted

in 1777 to estimate π (see e.g., [419]). But the modern

formulation of Monte Carlo methods started from 1940s

in physics [330], [329], [393] and later in 1950s to statis-

tics [198]. During the World War II, John von Neumann,

Stanislaw Ulam, Niick Metropolis, and others initialized

the Monte Carlo method in Los Alamos Laboratory. von

Neumann also used Monte Carlo method to calculate the

elements of an inverse matrix, in which they redeﬁned the

“Russian roulette” and “splitting” methods [472]. In recent

decades, Monte Carlo techniques have been rediscovered in-

dependently in statistics, physics, and engineering. Many

new Monte Carlo methodologies (e.g. Bayesian bootstrap,

hybrid Monte Carlo, quasi Monte Carlo) have been reju-

venated and developed. Roughly speaking, Monte Carlo

A generalized Bayesian theory is the so-called Quasi-Bayesian the-

ory (e.g. [100]) that is built on the convex set of probability distribu-

tions and a relaxed set of aximoms about preferences, which we don’t

discuss in this paper.

An interesting history of Thomas Bayes and its famous essay is

found in [110].

The method is named after the city in the Monaco principality,

because of a roulette, a simple random number generator. The name

was ﬁrst suggested by Stanislaw Ulam.

MANUSCRIPT 3

technique is a kind of stochastic sampling approach aim-

ing to tackle the complex systems which are analytically

intractable. The power of Monte Carlo methods is that

they can attack the diﬃcult numerical integration prob-

lems. In recent years, sequential Monte Carlo approaches

have attracted more and more attention to the researchers

from diﬀerent areas, with many successful applications in

statistics (see e.g. the March special issue of 2001 Annals

of the Institute of Statistical Mathematics), sig-

nal processing (see e.g., the February special issue of 2002

IEEE Transactions on Signal Processing), machine

learning, econometrics, automatic control, tracking, com-

munications, biology, and many others (e.g., see [141] and

the references therein). One of the attractive merits of se-

quential Monte Carlo approaches lies in the fact that they

allow on-line estimation by combining the powerful Monte

Carlo sampling methods with Bayesian inference, at an ex-

pense of reasonable computational cost. In particular, the

sequential Monte Carlo approach has been used in parame-

ter estimation and state estimation, for the latter of which

it is sometimes called particle ﬁlter.

The basic idea of

particle ﬁlter is to use a number of independent random

variables called particles,

sampled directly from the state

space, to represent the posterior probability, and update

the posterior by involving the new observations; the “par-

ticle system” is properly located, weighted, and propagated

recursively according to the Bayesian rule. In retrospect,

the earliest idea of Monte Carlo method used in statisti-

cal inference is found in [200], [201], and later in [5], [6],

[506], [433], [258], but the formal establishment of particle

ﬁlter seems fair to be due to Gordon, Salmond and Smith

[193], who introduced certain novel resampling technique

to the formulation. Almost in the meantime, a number

of statisticians also independently rediscovered and devel-

oped the sampling-importance-resampling (SIR) idea [414],

[266], [303], which was originally proposed by Rubin [395],

[397] in a non-dynamic framework.

The rediscovery and

renaissance of particle ﬁlters in the mid-1990s (e.g. [259],

[222], [229], [304], [307], [143], [40]) after a long dominant

period, partially thanks to the ever increasing computing

power. Recently, a lot of work has been done to improve

the performance of particle ﬁlters [69], [189], [428], [345],

[456], [458], [357]. Also, many doctoral theses were devoted

to Monte Carlo ﬁltering and inference from diﬀerent per-

spectives [191], [142], [162], [118], [221], [228], [35], [97],

[365], [467], [86].

It is noted that particle ﬁlter is not the only leaf in the

Bayesian ﬁltering tree, in the sense that Bayesian ﬁltering

can be also tackled with other techniques, such as diﬀeren-

Many other terminologies also exist in the literature, e.g., SIS ﬁl-

ter, SIR ﬁlter, bootstrap ﬁlter, sequential imputation, or CONDEN-

SATION algorithm (see [224] for many others), though they are ad-

dressed diﬀerently in diﬀerent areas. In this paper, we treat them as

diﬀerent variants within the generic Monte Carlo ﬁlter family. Monte

Carlo ﬁlters are not all sequential Monte Carlo estimation.

The particle ﬁlter is called normal if it produces i.i.d. samples;

sometimes it is deliberately to introduce negative correlations among

the particles for the sake of variance reduction.

The earliest idea of multiple imputation due to Rubin was pub-

lished in 1978 [394].

tial geometry approach, variational method, or conjugate

method. Some potential future directions, will be consid-

ering combining these methods with Monte Carlo sampling

techniques, as we will discuss in the paper. The attention

of this paper, however, is still on the Monte Carlo methods

and particularly sequential Monte Carlo estimation.

D. Outline of Paper

In this paper, we present a comprehensive review of

stochastic ﬁltering theory from Bayesian perspective. [It

happens to be almost three decades after the 1974 publica-

tion of Prof. Thomas Kailath’s illuminating review paper

“A view of three decades of linear ﬁltering theory” [244],

we take this opportunity to dedicate this paper to him who

has greatly contributed to the literature in stochastic ﬁlter-

ing theory.] With the tool of Bayesian statistics, it turns

out that the celebrated Kalman ﬁlter is a special case of

Bayesian ﬁltering under the LQG (linear, quadratic, Gaus-

sian) circumstance, a fact that was ﬁrst observed by Ho

and Lee [212]; particle ﬁlters are also essentially rooted

in Bayesian statistics, in the spirit of recursive Bayesian

estimation. To our interest, the attention will be given to

the nonlinear, non-Gaussian and non-stationary situations

where we mostly encounter in the real world. Generally for

nonlinear ﬁltering, no exact solution can be obtained, or the

solution is inﬁnite-dimensional,

hence various numerical

approximation methods come in to address the intractabil-

ity. In particular, we focus our attention on sequential

Monte Carlo method which allows on-line estimation in a

Bayesian perspective. The historic root and remarks of

Monte Carlo ﬁltering are traced. Other Bayesian ﬁltering

approaches other than Monte Carlo framework are also re-

viewed. Besides, we extend our discussion from Bayesian

ﬁltering to Bayesian inference, in the latter of which the

well-known hidden Markov model (HMM) (a.k.a. HMM

ﬁlter), dynamic Bayesian networks (DBN) and Bayesian

kernel machines are also brieﬂy discussed.

Nowadays Bayesian ﬁltering has become such a broad

topic involving many scientiﬁc areas that a comprehen-

sive survey and detailed treatment seems crucial to cater

the ever growing demands of understanding this important

ﬁeld for many novices, though it is noticed by the author

that in the literature there exist a number of excellent tuto-

rial papers on particle ﬁlters and Monte Carlo ﬁlters [143],

[144], [19], [438], [443], as well as relevant edited volumes

[141] and books [185], [173], [306], [82]. Unfortunately, as

observed in our comprehensive bibliographies, a lot of pa-

pers were written by statisticians or physicists with some

special terminologies, which might be unfamiliar to many

engineers. Besides, the papers were written with diﬀerent

nomenclatures for diﬀerent purposes (e.g. the convergence

and asymptotic results are rarely cared in engineering but

are important for the statisticians). The author, thus, felt

obligated to write a tutorial paper on this emerging and

promising area for the readership of engineers, and to in-

troduce the reader many techniques developed in statistics

Or the suﬃcient statistics is inﬁnite-dimensional.

MANUSCRIPT 4

and physics. For this purpose again, for a variety of particle

ﬁlter algorithms, the basic ideas instead of mathematical

derivations are emphasized. The further details and exper-

imental results are indicated in the references. Due to the

dual tutorial/review nature of current paper, only few sim-

ple examples and simulation are presented to illustrate the

essential ideas, no comparative results are available at this

stage (see other paper [88]); however, it doesn’t prevent us

presenting the new thoughts. Moreover, many graphical

and tabular illustrations are presented. Since it is also a

survey paper, extensive bibliographies are included in the

references. But there is no claim that the bibliographies

are complete, which is due to the our knowledge limitation

as well as the space allowance.

The rest of this paper is organized as follows: In Section

II, some basic mathematical preliminaries of stochastic ﬁl-

tering theory are given; the stochastic ﬁltering problem is

also mathematically formulated. Section III presents the

essential Bayesian theory, particularly Bayesian statistics

and Bayesian inference. In Section IV, the Bayesian ﬁl-

tering theory is systematically investigated. Following the

simplest LQG case, the celebrated Kalman ﬁlter is brieﬂy

derived, followed by the discussion of optimal nonlinear

ﬁltering. Section V discusses many popular numerical ap-

proximation techniques, with special emphasis on Monte

Carlo sampling methods, which result in various forms of

particle ﬁlters in Section VI. In Section VII, some other

new Bayesian ﬁltering approaches other than Monte Carlo

sampling are also reviewed. Section VIII presents some se-

lected applications and one illustrative example of particle

ﬁlters. We give some discussions and critiques in Section

IX and conclude the paper in Section X.

II. Mathematical Preliminaries and Problem

Formulation

A. Preliminaries

Deﬁnition 1: Let S be a set and F be a family of subsets

of S. F is a σ-algebra if (i) ∅∈F; (ii) A ∈Fimplies

∈F; (iii) A

, ···∈F implies ∪

∞

i=1

∈F.

A σ-algebra is closed under complement and union of

countably inﬁnitely many sets.

Deﬁnition 2: A probability space is deﬁned by the el-

ements {Ω, F,P} where F is a σ-algebra of Ω and P is

a complete, σ-additive probability measure on all F.In

other words, P is a set function whose arguments are ran-

dom events (element of F) such that axioms of probability

hold.

Deﬁnition 3: Let p(x)=

dP (x)

dμ

denote Radon-Nikod´ym

density of probability distribution P (x) w.r.t. a measure μ.

When x ∈ X is discrete and μ is a counting measure, p(x)

is a probability mass function (pmf); when x is continuous

and μ is a Lebesgue measure, p(x) is a probability density

function (pdf).

Intuitively, the true distribution P (x) can be replaced

by the empirical distribution given the simulated samples

(x)

Fig. 1. Empirical probability distribution (density) function con-

structed from the discrete observations {x

(i)

(see Fig. 1 for illustration)

P (x)=



i=1

δ(x − x

(i)

)

where δ(·) is a Radon-Nikod´ym density w.r.t. μ of the

point-mass distribution concentrated at the point x. When

x ∈ X is discrete, δ(x − x

(i)

)is1forx = x

(i)

and 0

elsewhere. When x ∈ X is continuous, δ(x − x

(i)

)isa

Dirac-delta function, δ(x − x

(i)

) = 0 for all x

(i)

= x,and



P (x)=



ˆp(x)dx =1.

B. Notations

Throughout this paper, the bold font is referred to vec-

tor or matrix; the subscript symbol t (t ∈ R

) is referred

to the index in a continuous-time domain; and n (n ∈ N)

is referred to the index in a discrete-time domain. p(x)is

referred to the pdf in a Lebesque measure or the pmf in

a counting measure. E[·]andVar[·] (Cov[·]) are expecta-

tion and variance (covariance) operators, respectively. Un-

less speciﬁed elsewhere, the expectations are taken w.r.t.

the true pdf. Notations x

0:n

and y

0:n

are referred to

the state and observation sets with elements collected from

time step 0 up to n. Gaussian (normal) distribution is de-

noted by N(μ, Σ). x

represents the true state in time

step n, whereas

(or

n|n

)and

n|n−1

represent the ﬁl-

tered state and predicted state of x

, respectively. f and g

are used to represent vector-valued state function and mea-

surement function, respectively. f is denoted as a generic

(vector or scalar valued) nonlinear function. Additional

nomenclatures will be given wherever confusion is neces-

sary to clarify.

For the reader’s convenience, a complete list of notations

used in this paper is summarized in the Appendix G.

C. Stochastic Filtering Problem

Before we run into the mathematical formulation of

stochastic ﬁltering problem, it is necessary to clarify some

basic concepts:

Filtering is an operation that involves the extraction of

information about a quantity of interest at time t by

using data measured up to and including t.

Sometimes it is also denoted by y

1:n

, which diﬀers in the assuming

order of state and measurement equations.

MANUSCRIPT 5

Prediction is an a priori form of estimation. Its aim is to

derive information about what the quantity of interest

will be like at some time t + τ in the future (τ>

0) by using data measured up to and including time

t. Unless speciﬁed otherwise, prediction is referred to

one-step ahead prediction in this paper.

Smoothing is an a posteriori form of estimation in that

data measured after the time of interest are used for

the estimation. Speciﬁcally, the smoothed estimate at

time t



is obtained by using data measured over the

interval [0,t], where t



<t.

Now, let us consider the following generic stochastic ﬁl-

tering problem in a dynamic state-space form [238], [422]:

= f(t, x

, u

, d

), (1a)

= g(t, x

, u

, v

), (1b)

where equations (1a) and (1b) are called state equation and

measurement equation, respectively; x

represents the state

vector, y

is the measurement vector, u

represents the sys-

tem input vector (as driving force) in a controlled environ-

ment; f : R

→ R

and g : R

→ R

are two vector-

valued functions, which are potentially time-varying; d

and v

represent the process (dynamical) noise and mea-

surement noise respectively, with appropriate dimensions.

The above formulation is discussed in the continuous-time

domain, in practice however, we are more concerned about

the discrete-time ﬁltering.

In this context, the following

practical ﬁltering problem is concerned:

n+1

= f(x

, d

), (2a)

= g(x

, v

), (2b)

where d

and v

can be viewed as white noise random

sequences with unknown statistics in the discrete-time do-

main. The state equation (2a) characterizes the state tran-

sition probability p(x

n+1

), whereas the measurement

equation (2b) describes the probability p(y

)whichis

further related to the measurement noise model.

The equations (2a)(2b) reduce to the following special

case where a linear Gaussian dynamic system is consid-

ered:

n+1

= F

n+1,n

+ d

, (3a)

= G

+ v

, (3b)

for which the analytic ﬁltering solution is given by the

Kalman ﬁlter [250], [253], in which the suﬃcient statistics

The continuous-time dynamic system can be always converted

into a discrete-time system by sampling the outputs and using “zero-

order holds” on the inputs. Hence the derivative will be replaced by

the diﬀerence, the operator will become a matrix.

For discussion simplicity, no driving-force in the dynamic system

(which is often referred to the stochastic control problem) is consid-

ered in this paper. However, the extension to the driven system is

straightforward.

An excellent and illuminating review of linear ﬁltering theory is

found in [244] (see also [385], [435], [61]); for a complete treatment of

linear estimation theory, see the classic textbook [247].

Suﬃcient statistics is referred to a collection of quantities which

uniquely determine a probability density in its entirety.

t-1

t+1

t-1

t+1

t-1

t+1

input

state

measurement

t-1

( )

t-1

( )

t+1

( )

Fig. 2. A graphical model of generic state-space model.

of mean and state-error correlation matrix are calculated

and propagated. In equations (3a) and (3b), F

n+1,n

, G

are called transition matrix and measurement matrix, re-

spectively.

Described as a generic state-space model, the stochastic

ﬁltering problem can be illustrated by a graphical model

(Fig. 2). Given initial density p(x

), transition density

p(x

n−1

), and likelihood p(y

), the objective of the

ﬁltering is to estimate the optimal current state at time n

given the observations up to time n, which is in essence

amount to estimating the posterior density p(x

0:n

)or

p(x

0:n

). Although the posterior density provides a

complete solution of the stochastic ﬁltering problem, the

problem still remains intractable since the density is a func-

tion rather than a ﬁnite-dimensional point estimate. We

should also keep in mind that most of physical systems are

not ﬁnite dimensional, thus the inﬁnite-dimensional system

can only be modeled approximately by a ﬁnite-dimensional

ﬁlter, in other words, the ﬁlter can only be suboptimal

in this sense. Nevertheless, in the context of nonlinear

ﬁltering, it is still possible to formulate the exact ﬁnite-

dimensional ﬁltering solution, as we will discuss in Section

IV.

In Table I, a brief and incomplete development history of

stochastic ﬁltering theory (from linear to nonlinear, Gaus-

sian to non-Gaussian, stationary to non-stationary) is sum-

marized. Some detailed reviews are referred to [244], [423],

[247], [205].

D. Nonlinear Stochastic Filtering Is an Ill-posed Inverse

Problem

D.1 Inverse Problem

Stochastic ﬁltering is an inverse problem: Given collected

at discrete time steps (hence y

0:n

), provided f and g are

known, one needs to ﬁnd the optimal or suboptimal

.In

another perspective, this problem can be interpreted as an

inverse mapping learning problem: Find the inputs sequen-

tially with a (composite) mapping function which yields the

output data. In contrast to the forward learning (given in-

puts ﬁnd outputs) which is a many-to-one mapping prob-

lem, the inversion learning problem is one-to-many, in a

sense that the mapping from output to input space is gen-

erally non-unique.

A problem is said to be well-posed if it satisﬁes three con-

剩余68页未读，继续阅读

评论收藏

内容反馈

yripple

粉丝: 18
资源: 20

Bayesian Filtering：From Kalman Filters to Particle Filters, and ...

最新资源

Bayesian Filtering：From Kalman Filters to Particle Filters, and ...

Bayesian filtering and smoothing

Kalman and Bayesian Filters in Python

Kalman_and_Bayesian_Filters_in_Python

Matlab 科技专讲：Understanding Kalman Filters

Kalman_and_Bayesian_Filters_in_Python.pdf

Bayesian-Filtering-and-Smoothing:SimoSärkkä的《贝叶斯滤波和平滑》一书的同伴python代码

贝叶斯滤波与平滑 bayesian filtering and smoothing

matlab代码影响-Kalman-and-Bayesian-Filters-in-Python:有一些修改。来自http://github.

matlab代码影响-Kalman-and-Bayesian-Filters-in-Python:Python中的卡尔曼和贝叶斯过滤器

RaoBlackwellisedParticleFilteringforDynamicBayesia_Bayesian_Rao

Cooperative and Graph Signal Processing

bflib：仅标头的贝叶斯过滤器库

Bayesian Signal Processing Classical,Modern,and Particle Filtering Methods

DAPPER:使用Python进行数据同化

行人与地图匹配：塔哈

国家开放大学计算机应用基础终结性考试（大作业）

离散数学知识点整理（超级全面详细！）

《科研伦理与学术规范》期末考试文档2（40题）

Word2Recite 桌面单词

2021全国及分省市县行政区划矢量图层shp文件.rar

38000词汇思维导图（1-50词根）β版.rar

MCGS组态精品版图库.zip

Revit 各版本官方族库及项目样板下载和安装方法，2016-2021族库离线包下载.rar

博士“申请-考核制”面试——英文提问问题/答案模板

iris-data.csv

最新资源