相机姿态校准-基于C++实现的用于校准相机姿态估计的最小求解器-附项目源码-优质项目实战.zip

# PoseLib This library provides a collection of minimal solvers for camera pose estimation. The focus is on calibrated absolute pose estimation problems from different types of correspondences (e.g. point-point, point-line, line-point, line-line). The goals of this project are to provide * Fast and robust implementation of the current state-of-the-art solvers. * Consistent calling interface between different solvers. * Minimize dependencies, both external (currently only [Eigen](http://eigen.tuxfamily.org/)) and internal. Each solver is (mostly) stand-alone, making it easy to extract only a specific solver to integrate into other frameworks. * Robust estimators (based on LO-RANSAC) that just works out-of-the-box for most cases. # Robust Estimation and Non-linear Refinement We provide robust estimators for the most common problems * Absolute pose from points (and lines) * Essential / Fundamental matrix * Homography * Generalized relative pose It is fairly straight-forward to implement robust estimators for other problems. See for example [absolute_pose.h](PoseLib/robust/estimators/absolute_pose.h). If you implement estimators for other problems, please consider submitting a pull-request. In [robust.h](PoseLib/robust.h) we provide interfaces which normalizes the data, calls the RANSAC and runs a post-RANSAC non-linear refinement. It is also possible to directly call the individual components as well (see e.g. [ransac.h](PoseLib/robust/ransac.h), [bundle.h](PoseLib/robust/bundle.h), etc.). The RANSAC is straight-forward implementation of LO-RANSAC which generate hypothesis with minimal solvers and relies on non-linear refinement for refitting. The robust estimator takes the following options ```c++ struct RansacOptions { size_t max_iterations = 100000; size_t min_iterations = 1000; double dyn_num_trials_mult = 3.0; double success_prob = 0.9999; double max_reproj_error = 12.0; // used for 2D-3D matches double max_epipolar_error = 1.0; // used for 2D-2D matches unsigned long seed = 0; // If we should use PROSAC sampling. Assumes data is sorted bool progressive_sampling = false; size_t max_prosac_iterations = 100000; // Whether to use real focal length checking for F estimation: https://arxiv.org/abs/2311.16304 // Assumes that principal points of both cameras are at origin. bool real_focal_check = false; }; ``` and the non-linear refinement ```c++ struct BundleOptions { size_t max_iterations = 100; enum LossType { TRIVIAL, TRUNCATED, HUBER, CAUCHY, TRUNCATED_LE_ZACH } loss_type = LossType::CAUCHY; double loss_scale = 1.0; double gradient_tol = 1e-8; double step_tol = 1e-8; double initial_lambda = 1e-3; double min_lambda = 1e-10; double max_lambda = 1e10; bool verbose = false; }; ``` Note that in [robust.h](PoseLib/robust.h) this is only used for the post-RANSAC refinement. In [bundle.h](PoseLib/robust/bundle.h) we provide non-linear refinement for different problems. Mainly minimizing reprojection error and Sampson error as these performed best in our internal evaluations. These are used in the LO-RANSAC to perform non-linear refitting. Most estimators directly minimize the MSAC score (using `loss_type = TRUNCATED` and `loss_scale = threshold`) over all input correspondences. In practice we found that this works quite well and avoids recursive LO where inliers are added in steps. ## Camera models PoseLib use [COLMAP](https://colmap.github.io/cameras.html)-compatible camera models. These are defined in [colmap_models.h](PoseLib/misc/colmap_models.h). Currently we only support * SIMPLE_PINHOLE * PINHOLE * SIMPLE_RADIAL * RADIAL * OPENCV but it is relatively straight-forward to add other models. If you do so please consider opening a pull-request. In contrast to COLMAP, we require analytical jacobians for the distortion mappings which make it a bit more work to port them. The `Camera` struct currently contains `width`/`height` fields, however these are not used anywhere in the code-base and are provided simply to be consistent with COLMAP. The `Camera` class also provides the helper function `initialize_from_txt(str)` which initializes the camera from a line given by the `cameras.txt` file of a COLMAP reconstruction. The python bindings also expose the `poselib.Camera` class with `focal(), focal_x(), focal_y(), model_name(), prinicipal_point()` read-only methods and a read-write `params` property, but currently this is only used as a return type for some methods. To supply camera information to robust estimators you should use python `dicts` as shown below. ## Python bindings The python bindings can be installed by running `pip install .`. The python bindings expose all minimal solvers, e.g. `poselib.p3p(x,X)`, as well as all robust estimators from [robust.h](PoseLib/robust.h). Examples of how the robust estimators can be called are ```python camera = {'model': 'SIMPLE_PINHOLE', 'width': 1200, 'height': 800, 'params': [960, 600, 400]} pose, info = poselib.estimate_absolute_pose(p2d, p3d, camera, {'max_reproj_error': 16.0}, {}) ``` or ```python F, info = poselib.estimate_fundamental_matrix(p2d_1, p2d_2, {'max_epipolar_error': 0.75, 'progressive_sampling': True}, {}) ``` The return value `info` is a dict containing information about the robust estimation (inliers, iterations, etc). The last two options are dicts which describe the `RansacOptions` and `BundleOptions`. Ommited values are set to their default (see above), except for the `loss_scale` used for the Cauchy loss which is set to half of the threshold used in RANSAC (which seems to be a good heuristic). Dicts with the default options can be obtained as `opt = poselib.RansacOptions()` or `poselib.BundleOptions()`. Some of the available estimators are listed below, check [pyposelib.cpp](pybind/pyposelib.cpp) and [robust.h](PoseLib/robust.h) for more details. The table also shows which error threshold is used in the estimation (`RansacOptions.max_reproj_error` or `RansacOptions.max_epipolar_error`). All thresholds are given in pixels. | Method | Arguments | (RansacOptions) Threshold | | --- | --- | --- | | _{`estimate_absolute_pose`} | _{`(p2d, p3d, camera, ransac_opt,bundle_opt)`} | _{`max_reproj_error`} | | _{`estimate_absolute_pose_pnpl`} | _{`(p2d, p3d, l2d_1, l2d_2, l3d_1, l3d_2, camera, ransac_opt, bundle_opt)`} | <sub>`max_reproj_error` (points), `max_epipolar_error` (lines) | | <sub>`estimate_generalized_absolute_pose` | _{`(p2ds, p3ds, camera_ext, cameras, ransac_opt, bundle_opt)`} | _{`max_reproj_error`} | | _{`estimate_relative_pose`} | _{`(x1, x2, camera1, camera2, ransac_opt, bundle_opt)`} | _{`max_epipolar_error`}| | _{`estimate_shared_focal_relative_pose`} | _{`(x1, x2, pp, ransac_opt, bundle_opt)`} | _{`max_epipolar_error`}| | _{`estimate_fundamental`} | _{`(x1, x2, ransac_opt, bundle_opt)`} | _{`max_epipolar_error`} | | _{`estimate_homography`} | _{`(x1, x2, ransac_opt, bundle_opt)`} | _{`max_reproj_error`} | | _{`estimate_generalized_relative_pose`} | _{`(matches, camera1_ext, cameras1, camera2_ext, cameras2, ransac_opt, bundle_opt)`} | _{`max_epipolar_error`} | ### Storing poses and estimated camera parameters To handle poses and cameras we provide the following classes: - `CameraPose`: This class is the return type for the most of the methods. While the class internally represent the pose with `q` and `t`, it also exposes `R` (3x3) and `Rt` (3x4) which are read/write, i.e. you can do `pose.R = Rnew` and it will update the underlying quaternion `q`. - `Image`: Following COLMAP, this class stores information about the camera (`image.camera`) and its pose (`image.pose`) used to take an image. - `ImagePair`: This class holds information about two cameras (`image_pair.camera1`, `image_pair.