【免费】斯坦福大学《机器人学》课程讲义资源-CSDN文库

共8个文件

pdf：8个

课程资源

需积分: 0 140 浏览量 2023-05-10 14:06:14 上传评论 1 收藏 3.82MB ZIP 举报

资源推荐

资源详情

资源评论

收起资源包目录

机器人学讲义.zip （8个子文件）

机器人学讲义

9讲讲义.pdf 2.8MB

13-15讲讲义.pdf 432KB

11-12讲讲义.pdf 248KB

1讲讲义.pdf 40KB

6-8讲讲义.pdf 336KB

1-3讲讲义.pdf 193KB

10讲讲义.pdf 72KB

4-5讲讲义.pdf 161KB

Gregory D. Hager

Laboratory for Computation, Sensing, and Control

Department of Computer Science

Johns Hopkins University

Perception & Sensing

in Robotic Mobility and Manipulation

!"# $% & '(# )% "*+%*,%-% #)(.

The Role of Perception in RMM

• Where am I relative to the world?

– sensors: vision, stereo, range sensors, acoustics

– problems: scene modeling/classification/recognition

– integration: localization/mapping algorithms (e.g. SLAM)

• What is around me?

– sensors: vision, stereo, range sensors, acoustics, sounds,

smell

– problems: object recognition, structure from x, qualitative

modeling

– integration: collision avoidance/navigation, learning

!"# $% & '(# )% "*+%*,%-% #)(.

The Role of Perception in RMM

• How can I safely interact with environment (including

people!)?

– sensors: vision, range, haptics (force+tactile)

– problems: structure/range estimation, modeling, tracking,

materials, size, weight, inference

– integration: navigation, manipulation, control, learning

• How can I solve “new” problems (generalization)?

– sensors: vision, range, haptics, undefined new sensor

– problems: categorization by function/shape/context/??

– integrate: inference, navigation, manipulation, control,

learning

• Obstacle detection, environment interaction

•Mapping, registration, localization, recognition

• Manipulation

Topics Today

• Computational Stereo

• Feature detection and matching

• Motion tracking and visual feedback

Techniques

Applications in Robotics:

!"# $% & '(# )% "*+%*,%-% #)(.

What is Computational Stereo?

Viewing the same physical point from

two different viewpoints allows depth

from triangulation

!"# $% & '(# )% "*+%*,%-% #)(.

Computational Stereo

• Much of geometric vision is based on information from 2 (or

more) camera locations

– hard to recover 3D information from a single 2D image without

extra knowledge

– motion and stereo (multiple cameras) are both common in the

world

• Stereo vision is ubiquitous in nature

– (oddly, nearly 10% of people are stereo blind)

• Stereo involves the following three problems:

1. calibration

2. matching (correspondence problem)

3. reconstruction (reconstruction problem)

!"# $% & '(# )% "*+%*,%-% #)(.

Binocular Stereo System: Geometry

• GOAL: Passive 2-camera system

using triangulation to generate a

depth map of a world scene.

• Depth map: z=f(x,y) where x,y are

coordinates one of the image

planes and z is the height above

the respective image plane.

– Note that for stereo systems which

differ only by an offset in x, the v

coordinates (projection of y) is the

same in both images!

– Note we must convert from image

(pixel) coordinates to external

coordinates -- requires calibration

(0,0,f)

4 intrinsic parameters convert

from pixel to metric values

!"# $% & '(# )% "*+%*,%-% #)(.

Non-verged Binocular Stereo System

(0,0) (b,0)

Z=f

Define Disparity:

D = (x

- x

)

Z =

b s

Assume: image are scan-line aligned

From perspective projection:

= s

X/Z

= s

(X - b)/Z

= y

= s

Y/Z

!"# $% & '(# )% "*+%*,%-% #)(.

To increase resolution:

• Increase of the baseline

(B) - size of the system

• Increase of the focal

length (f) - field of view

• Decrease of the pixel-size

(1/s

) - resolution of the

camera

75cm

Stereo-System Accuracy

Z =

b s

!"# $% & '(# )% "*+%*,%-% #)(.

Two-Camera Geometry

It is not hard to show that when we rotate the

cameras inward, corresponding points no longer lie

on a scan line

!"# $% & '(# )% "*+%*,%-% #)(.

How to Change Epipolar Geometry

Image rectification is the computation of

an image as seen by a rotated camera

Original image plane

New image plane

!"# $% & '(# )% "*+%*,%-% #)(.

= R(P

– T)

E p

= 0

Note that E is invariant to the scale

of the points, therefore we also have

where p denotes the (metric) image

projection of P

Now if K denotes the internal

calibration, converting from metric

to pixel coordinates, we have further

that

-t

E K

-1

= r

F r

= 0

where r denotes the pixel coordinates

of p. F is called the fundamental matrix

Fundamental Matrix Derivation

!"# $% & '(# )% "*+%*,%-% #)(.

Correspondence Problem:

How to find corresponding areas of two camera

images (points, line segments, curves, regions)

Stereo-Based Reconstruction

!"# $% & '(# )% "*+%*,%-% #)(.

MATCHING AND CORRESPONDENCE

• Two major approaches

– feature-based

– region based

In feature-based matching, the idea is

to pick a feature type (e.g. edges),

define a matching criteria (e.g.

orientation and contrast sign), and

then look for matches within a

disparity range

!"# $% & '(# )% "*+%*,%-% #)(.

Results - Reconstruction

!"# $% & '(# )% "*+%*,%-% #)(.

MATCHING AND CORRESPONDENCE

• Two major approaches

– feature-based

– region based

In region-based matching, the

idea is to pick a region in the image

and attempt to find the matching

region in the second image by

maximizing the some measure:

1. normalized SSD

2. SAD

3. normalized cross-correlation

!"# $% & '(# )% "*+%*,%-% #)(.

Match Metric Summary

u, v

( )

! I

( )

" I

u + d, v

( )

! I

( )

u, v

( )

! I

( )

" I

u + d, v

( )

! I

( )

u, v

( ) ( )( )

vduIvuI

( )( )

(

vuvu

IvduI

IvuI

( ) ( )

vduIvuI

( ) ( )

( )

vduIvuI

( ) ( ) ( )

kkk

vuInmIvuI

,,,

( ) ( )

( )

vduIvuIHAMMING

,,,

( ) ( ) ( )( )

vuInmIBITSTRINGvuI

kknmk

,,,

MATCH METRIC DEFINITION

Normalized Cross-Correlation

(NCC)

Sum of Squared Differences

(SSD)

Normalized SSD

Sum of Absolute Differences

(SAD)

Zero Mean SAD

Rank

Census

Remember, these

two are actually

the same

( ) ( )

"+""

IvduIIvuI

),(),(

!"# $% & '(# )% "*+%*,%-% #)(.

Correspondence Search Algorithm

For i = 1:nrows

for j=1:ncols

best(i,j) = -1

for k = mindisparity:maxdisparity

c = ComputeMatchMetric(I

(i,j),I

(i,j+k),winsize)

if (c > best(i,j))

best(i,j) = c

disparities(i,j) = k

end

end O(nrows * ncols * disparities * winx * winy)

end

!"# $% & '(# )% "*+%*,%-% #)(.

Correspondence Search Algorithm V2

best = -ones(size(im))

disp = zeros(size(im))

for k = mindisparity:maxdisparity

prod = I

(:,overlap) .* I

(:,k+overlap)

CC = conv2(prod,fspecial(‘average’,winsize))

better = CC > best;

disp = better .* k + (1-better).*disp;

best = better .*CC + (1-better).*best;

end

Typically saves O(winx*winy) operations for most any match

metric

!"# $% & '(# )% "*+%*,%-% #)(.

An Additional Twist

• Note that searching from left to right is not the same as searching from

right to left.

• As a result, we can obtain a somewhat independent disparity map by

flipping the images around.

• The results should be the same map up to sign.

• LRCheck: disp

(i,j) = - disp

(i,j+disp

(i,j))

!"# $% & '(# )% "*+%*,%-% #)(.

Example Disparity Maps

SSD

ZNNC

!"# $% & '(# )% "*+%*,%-% #)(.

Real-Time Stereo

REAL-TIME

STEREO SYSTEM

IMAGE

SIZE

FRAME

RATE

RANGE

BINS

METHOD PROCESSOR CAMERAS

INRIA 1993 256x256 3.6 fps 32 Normalized

Correlation

PeRLe-1 3

CMU iWarp 1993 256x240 15 fps 16 SSAD 64 Processor

iWarp Computer

Teleos 1995 320x240 0.5 fps 32 Sign

Correlation

Pentium 166 MHz 2

JPL 1995 256x240 1.7 fps 32 SSD Datacube &

68040

CMU Stereo

Machine 1995

256x240 30 fps 30 SSAD Custom HW &

C40 DSP Array

Point Grey Triclops

1997

320x240 6 fps 32 SAD Pentium II 450

MHz

SRI SVS 1997 320x240 12 fps 32 SAD Pentium II 233

MHz

SRI SVM II 1997 320x240 30+ fps 32 SAD TMS320C60x

200MHz DSP

Interval PARTS

Engine 1997

320x240 42 fps 24 Census

Matching

Custom FPGA 2

CSIRO 1997 256x256 30 fps 32 Census

Matching

Custom FPGA 2

SAZAN 1999 320x240 20 fps 25 SSAD FPGA &

Convolvers

Point Grey

Triclops 2001

320x240 20 fps

13 fps

32 SAD Pentium IV

1.4 GHz

SRI SVS 2001 320x240 30 fps 32 SAD Pentium III

700 MHZ

!"# $% & '(# )% "*+%*,%-% #)(.

Applications of Real-Time Stereo

• Mobile robotics

– Detect the structure of ground; detect obstacles; convoying

• Graphics/video

– Detect foreground objects and matte in other objects (super-

matrix effect)

• Surveillance

– Detect and classify vehicles on a street or in a parking

garage

• Medical

– Measurement (e.g. sizing tumors)

– Visualization (e.g. register with pre-operative CT)

!"# $% & '(# )% "*+%*,%-% #)(.

Stereo Example: Obstacle Detection

/$%- 012 *# % *.% 0314

5).#)"6 ').7 *- 1#8 11"*$10139"#

%- .# 9(01.*: ;<5= *9"&

)$$10139"#*: > < ?= *% - .#9 (01.

!"# $% & '(# )% "*+%*,%-% #)(.

Obstacle Detection (cont’d)

Observation: Removing the ground plane immediately exposes obstacles

!"# $% & '(# )% "*+%*,%-% #)(.

Applications of Real-Time Stereo

!"# $% & '(# )% "*+%*,%-% #)(.

斯坦福大学《机器人学》课程讲义

斯坦福大学机器人学讲义全集

斯坦福机器人教程 讲义和课件

斯坦福大学机器人学讲义Handout of CS223A

[斯坦福大学2009网络视频课程之机器人学]IntroductionToRobotics讲义

斯坦福机器人学讲义全集（含作业和解答）英文版

斯坦福大学机器学习课程讲义

斯坦福大学机器学习课程学习讲义

斯坦福开放课程 讲义7一：机器人卡雷尔 答案

斯坦福公开课机器人学PDF讲义打包.rar

斯坦福大学表示将提供CS与机器人课程教学

斯坦福大学开放课程机器人卡雷尔作业

斯坦福大学机器学习课程讲义cs229-andrew Ng. lecture notes

斯坦福大学机器学习课程原始讲义

嵌入式系统/ARM技术中的斯坦福大学表示将免费提供CS与机器人课程教学

斯坦福大学编程方法公开课与Karel机器人

stanford《机器人学》资料.rar

《斯坦福大学 卡雷尔机器人.pdf

斯坦福大学网络视频课程之机器人学 Lecture15

斯坦福大学机器学习学习课程讲义.rar_斯坦福大学机器学习_机器学习

斯坦福大学开放课程-编程方法-讲义

《斯坦福大学开放课程：编程方法》讲义

斯坦福大学CS106A课程讲义及教材

最新资源

斯坦福机器人教程讲义和课件

斯坦福开放课程讲义7一：机器人卡雷尔答案

《斯坦福大学卡雷尔机器人.pdf