张弥老师推荐- An Analysis for DNN for Practical Applications1
需积分: 0 187 浏览量
2022-08-04
14:04:22
上传
评论
收藏 278KB PDF 举报
AN ANALYSIS OF DEEP NEURAL NETWORK MODELS
FOR PRACTICAL APPLICATIONS
Alfredo Canziani & Eugenio Culurciello
Weldon School of Biomedical Engineering
Purdue University
{canziani,euge}@purdue.edu
Adam Paszke
Faculty of Mathematics, Informatics and Mechanics
University of Warsaw
ABSTRACT
Since the emergence of Deep Neural Networks (DNNs) as a prominent technique
in the field of computer vision, the ImageNet classification challenge has played a
major role in advancing the state-of-the-art. While accuracy figures have steadily
increased, the resource utilisation of winning models has not been properly taken
into account. In this work, we present a comprehensive analysis of important met-
rics in practical applications: accuracy, memory footprint, parameters, operations
count, inference time and power consumption. Key findings are: (1) power con-
sumption is independent of batch size and architecture; (2) accuracy and inference
time are in a hyperbolic relationship; (3) energy constraint is an upper bound on
the maximum achievable accuracy and model complexity; (4) the number of oper-
ations is a reliable estimate of the inference time. We believe our analysis provides
a compelling set of information that helps design and engineer efficient DNNs.
1 INTRODUCTION
Since the breakthrough in 2012 ImageNet competition (Russakovsky et al., 2015) achieved by
AlexNet (Krizhevsky et al., 2012) — the first entry that used a Deep Neural Network (DNN) —
several other DNNs with increasing complexity have been submitted to the challenge in order to
achieve better performance.
In the ImageNet classification challenge, the ultimate goal is to obtain the highest accuracy in a
multi-class classification problem framework, regardless of the actual inference time. We believe
that this has given rise to several problems. Firstly, it is now normal practice to run several trained
instances of a given model over multiple similar instances of each validation image. This practice,
also know as model averaging or ensemble of DNNs, dramatically increases the amount of com-
putation required at inference time to achieve the published accuracy. Secondly, model selection is
hindered by the fact that different submissions are evaluating their (ensemble of) models a different
number of times on the validation images, and therefore the reported accuracy is biased on the spe-
cific sampling technique (and ensemble size). Thirdly, there is currently no incentive in speeding up
inference time, which is a key element in practical applications of these models, and affects resource
utilisation, power-consumption, and latency.
This article aims to compare state-of-the-art DNN architectures, submitted for the ImageNet chal-
lenge over the last 4 years, in terms of computational requirements and accuracy. We compare these
architectures on multiple metrics related to resource utilisation in actual deployments: accuracy,
memory footprint, parameters, operations count, inference time and power consumption. The pur-
pose of this paper is to stress the importance of these figures, which are essential hard constraints
for the optimisation of these networks in practical deployments and applications.
2 METHODS
In order to compare the quality of different models, we collected and analysed the accuracy values
reported in the literature. We immediately found that different sampling techniques do not allow for
a direct comparison of resource utilisation. For example, central-crop (top-5 validation) errors of a
1
arXiv:1605.07678v4 [cs.CV] 14 Apr 2017
评论0