PAMI'14-Fast-Feature-Pyramids-for-Object-Detection

Fast Feature Pyramids for Object Detection

Piotr Doll



ar, Ron Appel, Serge Belongie, and Pietro Perona

Abstract—Multi-resolution image features may be approximated via extrapolation from nearby scales, rather than being computed

explicitly. This fundamental insight allows us to design object detection algorithms that are as accurate, and considerably faster, than

the state-of-the-art. The computational bottleneck of many modern detectors is the computation of features at every scale of a finely-

sampled image pyramid. Our key insight is that one may compute finely sampled feature pyramids at a fraction of the cost, without

sacrificing performance: for a broad family of features we find that features computed at octave-spaced scale intervals are sufficient to

approximate features on a finely-sampled pyramid. Extrapolation is inexpensive as compared to direct feature computation. As a result,

Ç

1INTRODUCTION

M

ULTI-RESOLUTION multi-orientation decompositions

are one of the foundational techniques of image anal-

ysis. The idea of analyzing image structure separately at

every scale and orientation originated from a number of

sources: measurements of the physiology of mammalian

visual systems [1], [2], [3], principled reasoning about the

statistics and coding of visual information [4], [5], [6], [7]

[18] and recognition [19], [20], [21].

It has become clear that such representations are best at

extracting visual information when they are overcomplete,

i.e., when one oversamples scale, orientation and other ker-

nel properties. This was suggested by the architecture of

the primate visual system [22], where striate cortical cells

(roughly equivalent to a wavelet expansion of an image)

outnumber retinal ganglion cells (a representation close to

image structure across scales. Our analysis and experiments

show that this makes it possible to inexpensively estimate

features at a dense set of scales by extrapolating computa-

tions carried out expensively, but infrequently, at a coarsely

three distinct detection frameworks including integral chan-

nel features (ICF) [29], aggregate channel features (a novel

variant of integral channel features), and deformable part

models (DPM) [35]. We show results for both pedestrian

detection (measured on the Caltech [31], INRIA [21], TUD-

Brussels [36] and ETH [37] data sets) and general object

detection (measured on the PASCAL VOC [38]). Demon-

strated speedups are significant and impact on accuracy is

relatively minor.

Building on our work on fast feature pyramids (first pre-

achieve speeds of 35-65 fps. Benenson et al. [30] imple-

mented fast feature pyramids on a GPU, and with addi-

tional innovations achieved detection rates of over 100 fps.

In this work we examine and analyze feature scaling and its

effect on object detection in far more detail than in our pre-

vious work [39].

The remainder of this paper is organized as follows.

We review rela ted work in Section 2. In Section 3 we

show that it is possible to create high fidelity approxima-

tions of multiscale gradient histograms using gradients

conclude in Section 7.

2RELATED WORK

Significant research has been devoted to scale space theory

[41], including real time implementations of octave and

half-octave image pyramids [42], [43]. Sparse image pyra-

mids often suffice for certain approximations, e.g., [42]

shows how to recover a disk’s characteristic scale using

half-octave pyramids. Although only loosely related, these

ideas provide the intuition that finely sampled feature pyra-

mids can perhaps be approximated.

k

tude in an upsampled discrete image. Then:

kn

kn

km

Thus, the sum of gradient magnitudes in the original and

upsampled image should be related by about a factor of k.

tionship between h

q

(computed over I)andh

dient histograms in an upsampled image using gradients

images of natural scenes, upsampled using bilinear interpo-

lation, the approximation h

q

is reasonable. We use

two sets of images for these experiments, one class specific

q

q

q

derived in Section 3.1 should apply.

Fig. 1. B ehav ior of gradient h is togr ams in images res amp led by a factor of two. (a) U psampling grad ients. G iven images I and I

where I

denotes I upsampled by two, and corre sponding gradient magnitude images M and M

M

M

to derive an alytical expressions governing the scaling properties of various feature types under different resa mpling facto rs, in Section 4 we

describe a general law gover ning feature scaling.