Multi-SensorFusionSystemforMovingObjectDetection&Tracking资源-CSDN文库

需积分: 10 98 浏览量 2021-09-11 11:58:45 上传评论收藏 1.53MB PDF 举报

A Multi-Sensor Fusion System for Moving Object Detection and Tracking in Urban Driving Environments - icra-14-sensor-fusion.pdf 在城市驾驶环境中，移动物体的检测与跟踪是自动驾驶汽车技术中的关键组成部分。本文"Multi-Sensor Fusion System for Moving Object Detection&Tracking"详细介绍了如何通过多传感器融合系统来提高这一过程的可靠性和效率。该系统是针对2007年DARPA城市挑战赛的早期系统的扩展和改进。【摘要】中提到，为了在现实世界驾驶环境中安全有效地部署自动驾驶汽车，车辆必须能够准确检测并追踪附近的移动物体。作者Hyunggi Cho、Young-Woo Seo、B.V.K. Vijaya Kumar和Ragunathan (Raj) Rajkumar提出了一种新的移动物体检测和跟踪系统，增加了视觉传感器，以提升之前仅依赖雷达和激光雷达（LIDAR）的主动传感器系统的性能。在【部分内容】中，新系统引入了视觉模块，用于识别行人、骑自行车的人和车辆，生成对应的视觉目标。这一视觉识别信息被用来优化早期系统的跟踪模型选择、数据关联和运动分类。通过实际驾驶数据记录的测试，展示了新系统在跟踪性能上的提升。 I. 引言部分指出，2005年和2007年的DARPA挑战赛推动了自动驾驶技术的发展，并引起了公众对自动驾驶汽车的关注。在此之后，业界和学术界取得了显著的进步，包括在日常驾驶环境中实现超过30万英里的自主驾驶，跨国自动驾驶，以及具有普通轿车外观的自动驾驶汽车等。这些发展为近未来的自动驾驶汽车铺平了道路。 Carnegie Mellon大学在城市挑战赛后开始新的研究，以进一步发展其成果，并开发出新的自动驾驶车辆。这个新的多传感器融合系统正是这种努力的体现，它结合了不同传感器的优势，以增强对复杂城市环境的理解和适应能力。通过融合不同的传感器数据，如雷达的远距离探测能力，激光雷达的高精度三维空间信息，以及视觉传感器的场景理解能力，系统能更好地处理遮挡、光照变化和传感器噪声等问题。这种融合不仅提高了目标检测的准确性，还增强了对目标运动状态的估计，这对于避免碰撞和做出安全决策至关重要。此外，系统改进了运动模型和观察模型，使数据关联更加精确，这意味着可以更准确地跟踪同一目标的运动轨迹，即使目标在视野中消失后也能重新捕获。同时，系统还优化了运动分类，区分不同类型的移动对象，例如行人、自行车和汽车，这对于自动驾驶汽车的决策制定至关重要。这篇论文提出的多传感器融合系统对于移动物体的检测和跟踪提供了更全面、更可靠的解决方案，对于未来自动驾驶汽车的安全性和实用性具有重要意义。随着技术的不断发展，我们可以期待这样的系统将为自动驾驶汽车带来更高的性能和更广泛的应用。

资源推荐

资源详情

资源评论

A Multi-Sensor Fusion System for Moving Object Detection and

Tracking in Urban Driving Environments

Hyunggi Cho, Young-Woo Seo, B.V.K. Vijaya Kumar, and Ragunathan (Raj) Rajkumar

Abstract— A self-driving car, to be deployed in real-world

driving environments, must be capable of reliably detecting

and effectively tracking of nearby moving objects. This paper

presents our new, moving object detection and tracking system

that extends and improves our earlier system used for the 2007

DARPA Urban Challenge. We revised our earlier motion and

observation models for active sensors (i.e., radars and LIDARs)

and introduced a vision sensor. In the new system, the vision

module detects pedestrians, bicyclists, and vehicles to generate

corresponding vision targets. Our system utilizes this visual

recognition information to improve a tracking model selection,

data association, and movement classiﬁcation of our earlier

system. Through the test using the data log of actual driving,

we demonstrate the improvement and performance gain of our

new tracking system.

I. INTRODUCTION

The 2005 DARPA Grand Challenge and the 2007 Urban

Challenge offered researchers with unique opportunities to

demonstrate the state of the art in the autonomous driving

technologies. These events were milestones in that they pro-

vided opportunities of reevaluating the status of the relevant

technologies and of regaining the public attention on self-

driving car development. Since then, the related technolo-

gies have been drastically advanced. Industry and academia

have reported notable achievements including: autonomous

driving more than 300,000 miles in daily driving contexts

[19], intercontinental autonomous driving [3], a self-driving

car with a stock-car appearance [20], and many more. Such

developments and demonstrations increased possibility of

self-driving cars in near future.

After the Urban Challenge, Carnegie Mellon University

started a new effort to advance the ﬁndings of the Urban

Challenge and developed a new autonomous vehicle [20]

to ﬁll the gap between the experimental robotic vehicles

and consumer cars. Among these efforts, this paper details

our perception system, particularly, a new moving objects

detection and tracking system. The Urban Challenge was

held in a simpliﬁed, urban driving setup where restricted

vehicle interactions occurred and no pedestrians, bicyclists,

motorcyclists, trafﬁc lights, GPS dropouts appeared. How-

ever, as shown in Figure 1, to be deployed in real-world

driving environments, autonomous driving vehicles must be

capable of safely interacting with nearby pedestrians and

vehicles. The prerequisite to safe interactions with nearby

objects is reliable detection and tracking of moving objects.

H. Cho, B.V.K Vijaya Kumar, and Ragunathan (Raj) Rajkumar are

with the ECE Department and Young-Woo Seo is with the Robotics

Institute, Carnegie Mellon University, 5000, Forbes Ave., Pittsburgh,

PA 15213, USA. {hyunggic, kumar, raj}@ece.cmu.edu,

young-woo.seo@ri.cmu.edu

Fig. 1. Sample images show urban driving environments and screen-

captures of our tracking system’s results. The images in the ﬁrst row

show detection and tracking results from an arriving area of Pittsburgh

international airport. The other two images in the second row show those

of an urban street.

To develop such a reliable perception capability for au-

tonomous urban driving, we redesigned our sensing system,

extended our earlier moving obstacle tracking system and

introduced new sensors in different modalities. Section III

and Section IV detail the conﬁguration of multiple sensors in

different modalities. Knowledge of moving objects’ classes

(e.g., car, pedestrian, bicyclists, etc.) is greatly helpful to

reliably track them and derive a better inference about driving

contexts. To acquire such a knowledge, we exploit vision

sensors to identify the classes of moving objects and to en-

hance measurements from automotive-grade active sensors,

such as LIDARs and radars. Section V describes interactions

between our vision sensor based object detection system

and active sensor based object tracking system. Section VI

discusses the experimental results and the ﬁndings. Section

VII summarizes our work and discusses future work.

II. RELATED WORK

Detection and tracking of moving objects is a core task

in mobile robotics and as well as in the ﬁeld of intelligent

vehicles. Due to such a critical role, this subject has been

extensively studied for the past decades. Since a compre-

hensive literature survey of this topic is beyond the scope

of this paper (we refer to [12], [16] for such surveys), here

we review only the earlier work on multi-sensor fusion for

2014 IEEE International Conference on Robotics & Automation (ICRA)

Hong Kong Convention and Exhibition Center

May 31 - June 7, 2014. Hong Kong, China

Camera

LIDAR

RADAR

LIDAR

RADAR

b) c)

Wheel Speed Sensor

RADAR

LIDAR

Fig. 2. CMU’s new autonomous vehicle and its sensor conﬁguration. (a) CMU’s new autonomous vehicle is designed to minimize alterations of a stock-car

appearance while installing multiple sensors to maximize the sensing coverage. (b) Visualization of LIDAR measurement. LIDAR scans acquired from

individual sensors are depicted in different color. (c) A horizontal ﬁeld of view (HFOV) of sensing coverage, emphasizing the coverage around the vehicle.

moving object detection and tracking, which are relevant to

our work.

The Navlab group at Carnegie Mellon University has a

long history of development of autonomous vehicles and

advanced driver assistance systems. For the Navlab 11, one

of the latest developments, they proposed a high-level fusion

approach for object tracking using cameras and LIDARs

[1]. In fact, our feature extraction algorithm for LIDAR is

motivated by their method [12]. Another interesting effort

was the work of Stiller et al. [18], where they used radar,

LIDAR, and stereo vision for obstacle detection and tracking.

Although they did not provide quantitative results of the

system, it brought researchers’ attention to the multi-sensor

fusion approach.

Since then, an approach of fusing LIDAR measurements

with vision sensors’ outputs has gained popularity for vehicle

tracking [11], [14] and pedestrian tracking [16], [17]. Mon-

teiro et al. [14] used a single-layer LIDAR and a monocular

camera to detect, track, and classify objects. For the fast

detection and tracking, a LIDAR was used and generated

regions of interest (ROIs) to a vision module. For the

classiﬁcation of objects, two classiﬁers, a Gaussian Mixture

Model (GMM) classiﬁer for a LIDAR and an Adaboost

classiﬁer for a camera, are applied. A sum decision rule

was used to combine both outputs. M

ahlisch et al. [11]

focused on a ‘cross-calibration’ method between these two

sensors while showing vehicle tracking. Premebida et al.

[16] used a multi-layer LIDAR and a monocular camera

for pedestrian detection. They exploited several features for

each sensor measurements and classiﬁcation algorithms for

better accuracy. Spinello and Siegwart [17] also utilized a

multi-layer LIDAR for detecting hypotheses for pedestrians

and then a vision-based pedestrian detector was applied for

veriﬁcation. A Bayesian decomposed expression was used as

a reasoning fusion rule.

The 2007 DARPA Urban Challenge provided researchers

with a unique opportunity to develop and test the multi-

sensor based systems [10], [15], [13]. Due to the practical

nature of the competition, high-level fusion approaches were

widely exploited. In particular, the Stanford [15] and MIT

TABLE I

INSTALLED SENSORS SPECIFICATIONS AND THEIR PRIMARY USAGES.

Sensor HFOV (°) MaxRange (m) Update Tracking

Type /Resolution Rate (Hz) Features

LIDAR 85∼110 200 50 edge target

RADAR1 30 250 12.5 point target

RADAR2 20 (near) 60 (near) 20 point target

18 (far) 175 (far)

Video 45 640×480 8 vision

Camera target

FLIR 36 640×480 24 vision

Camera target

[10] teams developed a similar object tracking system which

utilized a set of LIDAR sensors as primary sensors. Their

systems ﬁrst removed irrelevant measurements, such as laser

scans from the ground and from vertical structures, and

then ﬁtted geometric primitives (e.g., 2D rectangles) to the

remaining measurements to, using Bayesian ﬁlters, estimate

objects’ position, velocity, and size. Similarly, the Cornell

team [13] used a Rao-Blackwellized particle ﬁlter for moving

object tracking, where a data association problem is solved

by a particle ﬁlter and a state estimation problem is solved

by an extended Kalman ﬁlter. Most of the teams developed

their own tracking systems to effectively fuse sensor mea-

surements in different modalities. However, due to reliability

issue and computational cost, a tracking system based on

vision-sensor was not extensively studied for the competition.

III. A MULTI-SENSOR SETUP FOR ROBUST

PERCEPTION

The underlying ideas of our sensor conﬁguration are to

1) minimize any potential alterations of a vehicle’s original

appearance, 2) completely cover the area around the vehicle

within a certain range, and 3) utilize existing, off-the-shelf

and stock-car grade sensors. Based on these guidelines and

prior experiences, we built a new sensing system as shown

[20] in Figure 2(a). All sensors are seamlessly integrated into

the vehicle chassis and their appearance is indistinguishable

1837

剩余7页未读，继续阅读

评论收藏

内容反馈

TracelessLe

粉丝: 6w+
资源: 468

Multi-Sensor Fusion System for Moving Object Detection&Tracking

Multi-Sensor Fusion and Tracking.zip

SFND_3D_Object_Tracking-master：有关3D空间中对象检测的Sensor Fusion ND课程

A Contour-Based Moving Object Detection and Tracking

Sensor-Fusion-3D-Object-Tracking:该项目是Udacity的Sensor Fusion Nanodegree中的第三个项目。 该项目涵盖了以下TTC构件中显示的所有部分

Multi-Model Estimation Based Moving Object Detection for Aerial Video

some resource of image fusion for object tracking and detection

Multi-Sensor Data Fusion

End-to-end Learning of Multi-sensor 3D Tracking by Detection.pdf

讲稿_Robust Multi-Modality Multi-Object Tracking.docx

演示-Robust Multi-Modality Multi-Object Tracking.pptx

Sensor-Fusion-Lidar-Object-Detection-master(1).zip

Center-based 3D Object Detection and Tracking译文

End-to-end Learning of Multi-sensor 3D Tracking by Detection.zip

a contour-based moving object detection and tracking

Multi-Pose Multi-Target Tracking for Activity Understanding

Discriminatively Trained Particle Filters for Complex Multi-Object Tracking

Moving Object Detection and Tracking in Videos

Improving Context Modeling for Video Object Detection and Tracking

YOLO_&_RCNN_Object_Detection_and_Multi-

Hough Forests for Object Detection,Tracking, and Action Recognition.pdf

几篇CVPR关于multi-task的整理

data-fusion-for-indoor-tracking-by-RFID-matlab画图资源

下一代雷达的传感Sensor Fusion and Tracking for Next Generation Radar.pdf

kalman-filter-in-single-object-tracking-main.zip

dctracking.zip_Discrete-Continuous_multi object_multi-target_obj

Multi-parameter fusion algorithm for auto focus

Tracking Multiple People with a Multi-Camera System

A Twofold Siamese Network for Real-Time Object Tracking

MeMOTR: Long-Term Memory-Augmented Transformer for Multi-Object

最新资源

Sensor-Fusion-3D-Object-Tracking:该项目是Udacity的Sensor Fusion Nanodegree中的第三个项目。该项目涵盖了以下TTC构件中显示的所有部分