3-D Point Cloud Generation from Rigid and Flexible Stereo
Vision Systems
Nathaniel J. Short
Thesis submitted to the faculty of the Virginia Polytechnic Institute and State University in
partial fulfillment of the requirements for the degree of
MASTER OF SCIENCE
in
Computer Engineering
A. Lynn Abbott
Kevin Kochersberger
Robert Broadwater
December 4
th
, 2009
Blacksburg, VA
Keywords: Stereo Vision, UAV, VTOL, Camera Calibration, Terrain Mapping
Copyright 2009
3-D Point Cloud Generation from Rigid and Flexible Stereo Vision Systems
Nathaniel Short
ABSTRACT
When considering the operation of an Unmanned Aerial Vehicle (UAV) or an Unmanned
Ground Vehicle (UGV), such problems as landing site estimation or robot path planning become
a concern. Deciding if an area of terrain has a level enough slope and a wide enough area to land
a Vertical Take Off and Landing (VTOL) UAV or if an area of terrain is traversable by a ground
robot is reliant on data gathered from sensors, such as cameras. 3-D models, which can be built
from data extracted from digital cameras, can help facilitate decision making for such tasks by
providing a virtual model of the surrounding environment the system is in. A stereo vision
system utilizes two or more cameras, which capture images of a scene from two or more
viewpoints, to create 3-D point clouds. A point cloud is a set of un-gridded 3-D points
corresponding to a 2-D image, and is used to build gridded surface models.
Designing a stereo system for distant terrain modeling requires an extended baseline, or
distance between the two cameras, in order to obtain a reasonable depth resolution. As the width
of the baseline increases, so does the flexibility of the system, causing the orientation of the
cameras to deviate from their original state. A set of tools have been developed to generate 3-D
point clouds from rigid and flexible stereo systems, along with a method for applying corrections
to a flexible system to regain distance accuracy in a flexible system.
iii
ACKNOWLEDGEMENTS
I would like to thank my committee members for their help in writing this thesis; Dr.
Abbott, Dr. Broadwater, and Dr. Kochersberger. Dr. Abbott first pointed me towards the
Unmanned Systems Lab and his help with the stereo vision has been extremely helpful. I would
especially like to thank Dr. K for providing me the opportunity to work at the Unmanned
Systems Lab, where I was able to do research on some exciting projects in computer vision and
without which I would not have been able to produce this thesis. In addition, this research would
not have been possible without the support from Pacific Northwest National Laboratory.
I would also like to thank all the fellow research assistants at the lab who have been a
pleasure to work with over the past year. I would especially like to thank Prather Lanier, Brian
McCabe, and Jason Gassaway for their direct help on various aspects of research conducted in
this thesis.
My close friends and family have always been there for every achievement in my life,
and for that I am thankful. Their love, support, and direction have gotten me to and past many
milestones in life. I‟d especially like to thank my dad for his help and willingness to explain
things to me that have helped with my coursework and particularly with some of the research for
this thesis.
iv
CONTENTS
ABSTRACT .................................................................................................................................... ii
ACKNOWLEDGEMENTS ........................................................................................................... iii
CONTENTS ................................................................................................................................... iv
LIST OF FIGURES ....................................................................................................................... vi
LIST OF TABLES .......................................................................................................................... x
Chapter One: Introduction ............................................................................................................. 1
1.1 Background ...................................................................................................................... 1
1.2 Problem Statement ........................................................................................................... 3
1.3 Contribution ..................................................................................................................... 5
1.4 Organization of Thesis ..................................................................................................... 5
Chapter Two: Binocular Vision ..................................................................................................... 6
2.1 Stereo Vision .................................................................................................................... 6
2.2 Camera Calibration .......................................................................................................... 8
2.2.1 Overview ................................................................................................................... 8
2.2.2 Camera Model ......................................................................................................... 10
2.2.3 Rectification ............................................................................................................ 12
2.3 Correlation ...................................................................................................................... 16
2.4 3-D Re-projection ........................................................................................................... 19
2.5 Stereo vision considerations ........................................................................................... 20
2.5.1 Camera Selection and System Design .................................................................... 20
2.5.2 Stereo Procesre 14sing Latency .............................................................................. 24
Chapter Three: Stereo Vision System Tools ................................................................................ 27
3.1 Stereo Vision System Tools Overview .......................................................................... 27
v
3.2 Stereo Vision System Tools Process .............................................................................. 29
Chapter Four: Custom Stereo Vision Systems............................................................................. 34
4.1 Stereo Systems ............................................................................................................... 34
4.2 Stereo Vision Intermediate Results ................................................................................ 38
4.3 Terrain Mapping Results ................................................................................................ 40
4.4 Object Detection and Distance Measurement Performance ........................................... 45
Chapter Five: Camera Pose Correction for Large Baseline Stereo System ................................. 50
5.1 Overview ........................................................................................................................ 50
5.2 Instrumentation............................................................................................................... 51
5.3 Experimental Setup ........................................................................................................ 53
5.4 Test Procedures .............................................................................................................. 55
5.5 Results of Camera Motion Testing................................................................................. 59
5.6 Real-time Camera Trigger .............................................................................................. 61
Chapter Six: Conclusions and Future Work ................................................................................ 65
WORKS CITED ........................................................................................................................... 67
Appendix A: Stereo Vision System Tools C++ source code ....................................................... 69
Appendix B: Object extraction source code using Matlab ........................................................ 109
Appendix C: Camera Calibration Guide using Matlab .............................................................. 114
