BWN doc E 预生产1

An Energy-efficient Speech Classification Convolution Neural

Network Accelerator Based on FPGA and Quantization

Abstract

Deep convolution neural networks (CNN) have been shown to own unique advantages in acoustic tasks

over recurrent neural networks (RNN). However, activation data in convolution neural networks is often

indicated in floating format, which is both time-consuming and power-consuming when be computed.

Quantization method can turn activation into fix-point, replacing floating computing into faster and more

energy-saving fix-point computing. Based on this method, this article provides a design space searching

method to quantize a binary weight neural network with least accuracy loss. We also design a specific

and speech controlling. Since Deng, Yu et al

introduced RNN and LSTM (Long-Short Stage

Memory) acoustic model into speech recognition

and sound classification, LSTM has reached

series of excellent performance in this area [1][2].

However, deep neural networks based on RNN

are hard to be trained and parallelized due to

complex structure and recurrent computation.

When applied in actual missions, RNN models

usually demand high performance GPU and CPU

files can be transferred into feature maps or

feature matrixes by wave-filtering algorithms

(such as Mel Frequency Cepstral Coefficients

algorithm) [6], acoustic CNN models then run on

these maps just like input images for computer

vision models. With tiny 3x3 or 5x5 convolution

kernels, CNNs can be trained and forward faster

than RNN for convoluting operation is easier to

accelerate an acoustic CNN model on specific

power-efficiency hardware platforms.

longer word-length to store and more circuit parts

to compute, leading to more energy consumption

and bigger circuit designing area [9]. Also, the

computing complexity of floating-point data

makes it difficult to reduce computing cycle

counts. Floating computation, although can keep

precision well, has become the bottleneck of

power-efficiency high performance computing.

Fortunately, some works [10][11] have

proven that floating-point data is unnecessary to

models, there comes BNN (Binary Neural

Network) accelerators [12][13][14] and GPUs

supporting 8 bits integer data etc. [15][16] These

designs reduce power consumption greatly and

have up to hundreds of speedup ratios compared

with CPU platforms. It turns out that hardware

with corresponding quantized CNN models can

achieve excellent computing performance as well

as high energy efficiency.

Compared with CPU and GPU, ASIC and

FPGA are more suitable to accelerate a specific

task for their reconfigurable feature. These

reconfigurable platforms can be customized by

FPGA keeps a good balance between

performance, power, flexibility and expense due

to programmable feature and mature industry

design. Now, FPGA has been widely used in

cloud computing and intelligence computing by

Microsoft, Amazon and Alibaba [15][16],

becoming an important part of high- performance

computing.

The sound classification model which

focuses on specific speech instructions or

acoustics signal, is a basic component of

edge end. Such applying circumstance needs a

reduce computing platform’s energy

consumption by quantization and customized

hardware design. To implement this power-

efficient sound classification computation

classification model where weight value is +1 or

data format [17]. We design an accelerator based

on Xilinx XCKU-115 FPGA platform and run

this BWN (Binary Weight Network) model.

Compared with state-of-art CPU platform, our

accelerator achieves 18-300x throughput speed

up ratio and high energy efficiency. The main

contributions of this work are as follows:

1. We turn float-type feature data into fix-

point data each level by design space

exploration method. Model’s loss on

accuracy will be covered by the

performance of fix-point computing on

FPGA platforms.

2. We design a multi-PE BWN accelerator

balanced pipeline structure and low-delay

pipeline between CNN’s levels. Also, the

performance, power consumption and

discussed.

advantage on performance per watt and

2. Neural Network Forwarding Quantization

When training a deep neural network,

researchers usually choose full-precision data

format to ensure best model accuracy. However,

in inference task, these parameters will not be

changed and therefore we can prune them in an

offline method. [17] raises an algorithm to

compress floating-point DNN parameters into

Compared with common DNN with floating-

point weight and activation, this compression

method not only sharply reduces parameter

storage, but also replaces multiplication and

division with add and minus. Less storage space

and multiplication mean less memory-consuming

energy and less computing cycles, leading to

faster lower power consumption and faster

working speed.

[18] brings out a method to turn activation

into binary format. Unlike parameter in neural

networks, activation data fluctuates numerically

with different input (such like input image or

input audio feature map). Although [18] still

keeps a good model accuracy on very deep CNNs

like VGGNet, great numerical precision loss

would be brought out binary activation data,

which may cause vital influence on some small-

size CNNs [19][20]. In this situation, turning

floating data into fix-point format can keep a

good balance between computing performance

needs less computing cycles compared with

floating data, and, fix-point can adapt to data’s

numerical distribution by flexible allocation of

integer bitwise and decimal bitwise.

In Fig 1, when integer part is allocated with

more bitwise, it can indicate bigger data; and

when we give decimal part longer word-length, it

can improve numerical precision correspondingly.

However, increasing the length of fix-point data

format will add cycle counts to computation or

excess hardware’s limitation, so taking speed and

accuracy into account, it is important to find the

Fig.1 How Bitwise Influences Numerical Precision

3.1 Model Architecture and Weight

Binarization

This CNN-based speech recognition model

is trained on Tensorflow speech command dataset.

It can recognize the six sort of short speech

segment “up”, “down”, “yes”, “right”, “left” and

“unknown words”. This model first uses MFCC

algorithm turning an audio file into a float-type

tensor, whose dimension is 20x49x1. Then this

tensor will be sent into a convolution neural

function, this model outputs the possibility of six

type of labels.

primitive parameter is normal distribution, the