BundleAdjustment—AModernSynthesis（全部文档）资源-CSDN文库

共1个文件

pdf：1个

捆绑调整

光束法平差

5星 · 超过95%的资源需积分: 50 72 浏览量 2018-06-13 21:03:32 上传评论收藏 521KB RAR 举报

资源推荐

资源详情

资源评论

收起资源包目录

Bundle Adjustment —A Modern Synthesis（全部文档）.rar （1个子文件）

Bundle Adjustment —A Modern Synthesis（全部文档）

Bundle Adjustment —A Modern Synthesis（全部文档）.pdf 570KB

Bundle Adjustment — A Modern Synthesis

Bill Triggs

, Philip McLauchlan

, Richard Hartley

and Andrew Fitzgibbon

INRIA Rhˆone-Alpes, 655 avenue de l’Europe, 38330 Montbonnot, France.

Bill.Triggs@inrialpes.fr  http://www.inrialpes.fr/movi/people/Triggs

School of Electrical Engineering, Information Technology & Mathematics

University of Surrey, Guildford, GU2 5XH, U.K.

P.McLauchlan@ee.surrey.ac.uk  http://www.ee.surrey.ac.uk/Personal/P.McLauchlan

General Electric CRD, Schenectady, NY, 12301

hartley@crd.ge.com

Dept of Engineering Science, University of Oxford, 19 Parks Road, OX1 3PJ, U.K.

awf@robots.ox.ac.uk  http://www.robots.ox.ac.uk/ awf

Abstract

This paper is a survey of the theory and methods of photogrammetric bundle adjustment,

aimed at potential implementors in the computer vision community. Bundle adjustment is the

problem of reﬁning a visual reconstruction to produce jointly optimal structure and viewing pa-

rameter estimates. Topics covered include: the choice of cost function and robustness; numerical

optimization including sparse Newton methods, linearly convergent approximations, updating

and recursive methods; gauge (datum) invariance; and quality control. The theory is developed

for general robust cost functions rather than restricting attention to traditional nonlinear least

squares.

Keywords: Bundle Adjustment, Scene Reconstruction, Gauge Freedom, Sparse Matrices, Opti-

mization.

1 Introduction

This paper is a survey of the theory and methods of bundle adjustment aimed at the computer vision

community, and more especially at potential implementors who already know a little about bundle

methods. Most of the results appeared long ago in the photogrammetry and geodesy literatures, but

many seem to be little known in vision, where they are gradually being reinvented. By providing an

accessible modern synthesis, we hope to forestall some of this duplication of effort, correct some com-

mon misconceptions, and speed progress in visual reconstruction by promoting interaction between

the vision and photogrammetry communities.

Bundle adjustment is the problem of reﬁning a visual reconstruction to produce jointly optimal

3D structure and viewing parameter (camera pose and/or calibration) estimates. Optimal means that

This work was supported in part by the European Commission Esprit LTR project CUMULI (B. Triggs), the UK EPSRC

project GR/L34099 (P. McLauchlan), and the Royal Society (A. Fitzgibbon). We would like to thank A. Zisserman, A. Gr¨un

and W. F¨orstner for valuable comments and references. A version of this paper will appear in Vision Algorithms: Theory &

Practice, B. Triggs, A. Zisserman & R. Szeliski (Eds.), Springer-Verlag LNCS 1883, 2000.

the parameter estimates are found by minimizing some cost function that quantiﬁes the model ﬁtting

error, and jointly that the solution is simultaneously optimal with respect to both structure and camera

variations. The name refers to the ‘bundles’ of light rays leaving each 3D feature and converging on

each camera centre, which are ‘adjusted’ optimally with respect to both feature and camera positions.

Equivalently — unlike independent model methods, which merge partial reconstructions without up-

dating their internal structure — all of the structure and camera parameters are adjusted together ‘in

one bundle’.

Bundle adjustment is really just a large sparse geometric parameter estimation problem, the pa-

rameters being the combined 3D feature coordinates, camera poses and calibrations. Almost every-

thing that we will say can be applied to many similar estimation problems in vision, photogrammetry,

industrial metrology, surveying and geodesy. Adjustment computations are a major common theme

throughout the measurement sciences, and once the basic theory and methods are understood, they

are easy to adapt to a wide variety of problems. Adaptation is largely a matter of choosing a numerical

optimization scheme that exploits the problem structure and sparsity. We will consider several such

schemes below for bundle adjustment.

Classically, bundle adjustment and similar adjustment computations are formulated as nonlinear

least squares problems [19, 46, 100, 21, 22, 69, 5, 73, 109]. The cost function is assumed to be

quadratic in the feature reprojection errors, and robustness is provided by explicit outlier screening.

Although it is already very ﬂexible, this model is not really general enough. Modern systems of-

ten use non-quadratic M-estimator-like distributional models to handle outliers more integrally, and

many include additional penalties related to overﬁtting, model selection and system performance (pri-

ors, MDL). For this reason, we will not assume a least squares / quadratic cost model. Instead, the

cost will be modelled as a sum of opaque contributions from the independent information sources

(individual observations, prior distributions, overﬁtting penalties ... ). The functional forms of these

contributions and their dependence on ﬁxed quantities such as observations will usually be left im-

plicit. This allows many different types of robust and non-robust cost contributions to be incorporated,

without unduly cluttering the notation or hiding essential model structure. It ﬁts well with modern

sparse optimization methods (cost contributions are usually sparse functions of the parameters) and

object-centred software organization, and it avoids many tedious displays of chain-rule results. Im-

plementors are assumed to be capable of choosing appropriate functions and calculating derivatives

themselves.

One aim of this paper is to correct a number of misconceptions that seem to be common in the

vision literature:

• “Optimization / bundle adjustment is slow”: Such statements often appear in papers introducing

yet another heuristic Structure from Motion (SFM) iteration. The claimed slowness is almost

always due to the unthinking use of a general-purpose optimization routine that completely ignores

the problem structure and sparseness. Real bundle routines are much more efﬁcient than this, and

usually considerably more efﬁcient and ﬂexible than the newly suggested method (§6, 7). That is

why bundle adjustment remains the dominant structure reﬁnement technique for real applications,

after 40 years of research.

• “Only linear algebra is required”: This is a recent variant of the above, presumably meant to

imply that the new technique is especially simple. Virtually all iterative reﬁnement techniques

use only linear algebra, and bundle adjustment is simpler than many in that it only solves linear

systems: it makes no use of eigen-decomposition or SVD, which are themselves complex iterative

methods.

• “Any sequence can be used”: Many vision workers seem to be very resistant to the idea that

reconstruction problems should be planned in advance (§11), and results checked afterwards to

verify their reliability (§10). System builders should at least be aware of the basic techniques

for this, even if application constraints make it difﬁcult to use them. The extraordinary extent to

which weak geometry and lack of redundancy can mask gross errors is too seldom appreciated, c.f .

[34, 50, 30, 33].

• “Point P is reconstructed accurately”: In reconstruction, just as there are no absolute references

for position, there are none for uncertainty. The 3D coordinate frame is itself uncertain, as it

can only be located relative to uncertain reconstructed features or cameras. All other feature and

camera uncertainties are expressed relative to the frame and inherit its uncertainty, so statements

about them are meaningless until the frame and its uncertainty are speciﬁed. Covariances can look

completely different in different frames, particularly in object-centred versus camera-centred ones.

See §9.

There is a tendency in vision to develop a profusion of ad hoc adjustment iterations. Why should you

use bundle adjustment rather than one of these methods? :

• Flexibility: Bundle adjustment gracefully handles a very wide variety of different 3D feature and

camera types (points, lines, curves, surfaces, exotic cameras), scene types (including dynamic and

articulated models, scene constraints), information sources (2D features, intensities, 3D informa-

tion, priors) and error models (including robust ones). It has no problems with missing data.

• Accuracy: Bundle adjustment gives precise and easily interpreted results because it uses accurate

statistical error models and supports a sound, well-developed quality control methodology.

• Efﬁciency: Mature bundle algorithms are comparatively efﬁcient even on very large problems.

They use economical and rapidly convergent numerical methods and make near-optimal use of

problem sparseness.

In general, as computer vision reconstruction technology matures, we expect that bundle adjustment

will predominate over alternative adjustment methods in much the same way as it has in photogram-

metry. We see this as an inevitable consequence of a greater appreciation of optimization (notably,

more effective use of problem structure and sparseness), and of systems issues such as quality control

and network design.

Coverage: We will touch on a good many aspects of bundle methods. We start by considering the

camera projection model and the parametrization of the bundle problem §2, and the choice of er-

ror metric or cost function §3. §4 gives a rapid sketch of the optimization theory we will use. §5

discusses the network structure (parameter interactions and characteristic sparseness) of the bundle

problem. The following three sections consider three types of implementation strategies for adjust-

ment computations: §6 covers second order Newton-like methods, which are still the most often

used adjustment algorithms; §7 covers methods with only ﬁrst order convergence (most of the ad

hoc methods are in this class); and §8 discusses solution updating strategies and recursive ﬁltering

bundle methods. §9 returns to the theoretical issue of gauge freedom (datum deﬁciency), including

the theory of inner constraints. §10 goes into some detail on quality control methods for monitoring

the accuracy and reliability of the parameter estimates. §11 gives some brief hints on network design,

i.e. how to place your shots to ensure accurate, reliable reconstruction. §12 completes the body of

the paper by summarizing the main conclusions and giving some provisional recommendations for

methods. There are also several appendices. §A gives a brief historical overview of the development

of bundle methods, with literature references. §B gives some technical details of matrix factorization,

updating and covariance calculation methods. §C gives some hints on designing bundle software, and

pointers to useful resources on the Internet. The paper ends with a glossary and references.

General references: Cultural differences sometimes make it difﬁcult for vision workers to read

the photogrammetry literature. The collection edited by Atkinson [5] and the manual by Karara

[69] are both relatively accessible introductions to close-range (rather than aerial) photogrammetry.

Other accessible tutorial papers include [46, 21, 22]. Kraus [73] is probably the most widely used

photogrammetry textbook. Brown’s early survey of bundle methods [19] is well worth reading. The

often-cited manual edited by Slama [100] is now quite dated, although its presentation of bundle

adjustment is still relevant. Wolf & Ghiliani [109] is a text devoted to adjustment computations, with

an emphasis on surveying. Hartley & Zisserman [62] is an excellent recent textbook covering vision

geometry from a computer vision viewpoint. For nonlinear optimization, Fletcher [29] and Gill et al

[42] are the traditional texts, and Nocedal & Wright [93] is a good modern introduction. For linear

least squares, Bj¨orck [11] is superlative, and Lawson & Hanson is a good older text. For more general

numerical linear algebra, Golub & Van Loan [44] is the standard. Duff et al [26] and George & Liu

[40] are the standard texts on sparse matrix techniques. We will not discuss initialization methods for

bundle adjustment in detail, but appropriate reconstruction methods are plentiful and well-known in

the vision community. See, e.g., [62] for references.

Notation: The structure, cameras, etc., being estimated will be parametrized by a single large state

vector x. In general the state belongs to a nonlinear manifold, but we linearize this locally and work

with small linear state displacements denoted δx. Observations (e.g. measured image features) are

denoted z

. The corresponding predicted values at parameter value x are denoted z = z(x), with

residual prediction error 4z(x) ≡ z

− z(x). However, observations and prediction errors usually

only appear implicitly, through their inﬂuence on the cost function f(x)=f(predz(x)). The cost

function’s gradient is g ≡

, and its Hessian is H ≡

.Theobservation-state Jacobian is

J ≡

. The dimensions of δx, δz are n

2 Projection Model and Problem Parametrization

2.1 The Projection Model

We begin the development of bundle adjustment by considering the basic image projection model and

the issue of problem parametrization. Visual reconstruction attempts to recover a model of a 3D scene

from multiple images. As part of this, it usually also recovers the poses (positions and orientations)

of the cameras that took the images, and information about their internal parameters. A simple scene

model might be a collection of isolated 3D features, e.g., points, lines, planes, curves, or surface

patches. However, far more complicated scene models are possible, involving, e.g., complex objects

linked by constraints or articulations, photometry as well as geometry, dynamics, etc. One of the great

strengths of adjustment computations — and one reason for thinking that they have a considerable

future in vision — is their ability to take such complex and heterogeneous models in their stride.

Almost any predictive parametric model can be handled, i.e. any model that predicts the values of

some known measurements or descriptors on the basis of some continuous parametric representation

of the world, which is to be estimated from the measurements.

Similarly, many possible camera models exist. Perspective projection is the standard, but the

afﬁne and orthographic projections are sometimes useful for distant cameras, and more exotic models

such as push-broom and rational polynomial cameras are needed for certain applications [56, 63]. In

addition to pose (position and orientation), and simple internal parameters such as focal length and

principal point, real cameras also require various types of additional parameters to model internal

aberrations such as radial distortion [17, 18, 19, 100, 69, 5].

For simplicity, suppose that the scene is modelled by individual static 3D features X

, p =1...n,

imaged in m shots with camera pose and internal calibration parameters P

, i =1...m.Theremay

also be further calibration parameters C

, c =1...k, constant across several images (e.g., depending

on which of several cameras was used). We are given uncertain measurements x

of some subset of

the possible image features x

(the true image of feature X

in image i). For each observation x

we assume that we have a predictive model x

= x(C

, P

, X

) based on the parameters, that can

be used to derive a feature prediction error:

, P

, X

) ≡ x

− x(C

, P

, X

) (1)

In the case of image observations the predictive model is image projection, but other observation

types such as 3D measurements can also be included.

To estimate the unknown 3D feature and camera parameters from the observations, and hence

reconstruct the scene, we minimize some measure (discussed in §3) of their total prediction error.

Bundle adjustment is the model reﬁnement part of this, starting from given initial parameter estimates

(e.g., from some approximate reconstruction method). Hence, it is essentially a matter of optimizing

a complicated nonlinear cost function (the total prediction error) over a large nonlinear parameter

space (the scene and camera parameters).

We will not go into the analytical forms of the various possible feature and image projection

models, as these do not affect the general structure of the adjustment network, and only tend to

obscure its central simplicity. We simply stress that the bundle framework is ﬂexible enough to

handle almost any desired model. Indeed, there are so many different combinations of features,

image projections and measurements, that it is best to regard them as black boxes, capable of giving

measurement predictions based on their current parameters. (For optimization, ﬁrst, and possibly

second, derivatives with respect to the parameters are also needed).

For much of the paper we will take quite an abstract view of this situation, collecting the scene and

camera parameters to be estimated into a large state vector x, and representing the cost (total ﬁtting

error) as an abstract function f(x). The cost is really a function of the feature prediction errors 4x

− x(C

, P

, X

). But as the observations x

are constants during an adjustment calculation, we

leave the cost’s dependence on them and on the projection model x(·) implicit, and display only its

dependence on the parameters x actually being adjusted.

2.2 Bundle Parametrization

The bundle adjustment parameter space is generally a high-dimensional nonlinear manifold — a large

Cartesian product of projective 3D feature, 3D rotation, and camera calibration manifolds, perhaps

with nonlinear constraints, etc. The state x is not strictly speaking a vector, but rather a point in this

space. Depending on how the entities that it contains are represented, x can be subject to various

types of complications including singularities, internal constraints, and unwanted internal degrees of

freedom. These arise because geometric entities like rotations, 3D lines and even projective points and

planes, do not have simple global parametrizations. Their local parametrizations are nonlinear, with

singularities that prevent them from covering the whole parameter space uniformly (e.g.themany

variants on Euler angles for rotations, the singularity of afﬁne point coordinates at inﬁnity). And their

global parametrizations either have constraints (e.g. quaternions with kqk

=1), or unwanted internal

degrees of freedom (e.g. homogeneous projective quantities have a scale factor freedom, two points

deﬁning a line can slide along the line). For more complicated compound entities such as matching

tensors and assemblies of 3D features linked by coincidence, parallelism or orthogonality constraints,

parametrization becomes even more delicate.

评论收藏

内容反馈

u010873777

2020-05-18

下载了，内容可以。
qq_30236345

2018-07-01

very good,good ,good

姚永祥

粉丝: 139
资源: 5

Bundle Adjustment —A Modern Synthesis（全部文档）

Bundle Adjustment —A Modern Synthesis

Bundle Adjustment：A Modern Synthesis

matlab开发-BundleAdjustment

BUNDLE ADJUSTMENT RULES

光束平差（Bundle Adjustment）源代码

sparse bundle adjustment

SBA: A Software Package for Generic Sparse Bundle Adjustment

bundle adjustment 工具SBA

Generic Sparse Bundle Adjustment for C++

Bundle Adjustment Method using Sparse BFGS Solution

Bundle Adjustment Constrained Smoothing For Multi-View Point Cloud Data

SLAM专题（10）- 最小化重投影误差与Bundle Adjustment （BA）

sparse bundle adjustment( sba)编译及使用

Bundle Adjustment for Matlab.zip

BundleAdjustment

SBA（Sparse Bundle Adjustment）的VC2010版本

基于python实现在线外参估计功能源码(基于Bundle Adjustment算法和李群-李代数工具)+运行说明.zip

Bundle Adjustment using lsqnonlin：此代码使用Matlab函数“lsqnonlin”实现Bundle Adjustment-matlab开发

Qt 5实现串口调试助手 （源工程文件、0积分下载）

【SystemVerilog】路科验证V2学习笔记（全600页）.pdf

AutoSAR标准协议4.2.2

光伏-储能并网系统仿真.rar

NPPJSONViewer.zip

GD32替换STM32注意事项.pdf

XCP协议的规范文档

VS2015安装证书，JavaScript_ProjectSystem.msi，JavaScript_LanguageService.msi

CANoe通过CAPL脚本实现自动测试

蓝牙BLE协议中文版.pdf

BaiduOCR.zip

最新资源

Qt 5实现串口调试助手（源工程文件、0积分下载）