VI Preface
function, and then train an algorithm that outputs decisions that directly mini-
mize the expected loss. In a realistic setting, however, the loss function might be
unknown, or depend on additional factors only determined at a later stage. A
system that predicts the presence of calcification from a mammography should
also provide information about its uncertainty. Whether to operate or not will
depend on the particular patient, as well as on the context in general. If the loss
function is unknown, expressing uncertainties becomes crucial. Failing to do so
implies throwing information away.
There does not seem to be a universal way of producing good estimates of
predictive uncertainty in the machine learning community, nor a consensus on
the ways of evaluating them. In part this is caused by deep fundamental differ-
ences in methodology (classical statistics, Bayesian inference, statistical learning
theory). We decided to organize the Evaluating Predictive Uncertainty Challenge
(http://predict.kyb.tuebingen.mpg.de/) to allow the different philosophies
to compete directly on the empirical battleground. This required us to define
losses for probabilistic predictions. Twenty groups of participants competed on
two classification and three regression datasets before the submission deadline of
December 11, 2004, and a few more after the deadline. We present six contributed
chapters to this volume, by all the winners plus authors of other outstanding
entries.
Visual Objects Classes
The PASCAL Visual Object Classes Challenge ran from February to March
2005 (http://www.pascal-network.org/challenges/VOC/). The goal of the
challenge was to recognize objects from a number of visual object classes in
realistic scenes (i.e., not pre-segmented objects). Although there already exist
benchmarks such as the so-called ‘Caltech 5’ (faces, airplanes, motorbikes, cars
rear, spotted cats) and UIUC car side images, largely used by the community
of image recognition, it appears now that the developed methods are achieving
such good performance that they have effectively saturated on these datasets,
and thus the datasets are failing to challenge the next generation of algorithms.
Such saturation can arise because the images used do not explore the full range
of variability of the imaged visual class. Some dimensions of variability include:
clean vs. cluttered background; stereotypical views vs. multiple views (e.g., side
views of cars vs. cars from all angles); degree of scale change, amount of occlusion;
the presence of multiple objects (of one or multiple classes) in the images.
Given this problem of saturation of performance, the Visual Object Classes
Challenge was designed to be more demanding by enhancing some of the di-
mensions of variability listed above compared to the databases that had been
available previously, so as to explore the failure modes of different algorithms.
Four object classes were selected: motorbikes, bicycles, cars and people. Twelve
teams entered the challenge. This book includes a contributed review chapter
about the methods and the results achieved by the participants.
评论0
最新资源