Hierarchical Convolutional Neural Networks for EEG-Based Emotion...

Hierarchical Convolutional Neural Networks for EEG-Based

Emotion Recognition

Jinpeng Li

1,2

& Zhaoxiang Zhang

1,2,3

& Huiguang He

1,2,3

Received: 10 April 2017 /Accepted: 30 November 2017

#

Springer Science+Business Media, LLC, part of Springer Nature 2017

Abstract

Traditional machine learning methods suffer from severe overfitting in EEG-based emotion reading. In this paper, we use

chical structure. Benefiting from the strong representational learning capacity in the two-dimensional space, HCNN is efficient in

emotion recognition especially on Beta and Gamma waves.

Keywords Affective brain-computer interface

.

Emotion recognition

Emotion is important when people interact with each other. In

some cases, we also want the machines to interact with us

according to our emotion states. The affective brain-computer

interface (aBCI) [1] is a technique to make it a reality. In

aBCIs, the key problem is to deduce their human user’semo-

also help psychologists understand the internal mechanism of

human emotion processing.

speech, and other multimodal information together and discrim-

atively economical. However, signals of this kind are highly

related to the personal habit (culture) of subjects, and thus could

not be universally applied. Several studies have been conducted

able. The former has relatively higher signal-to-noise ratio

and wearable, but the SNR is lower, which is a challenge to

Cognitive Computation

https://doi.org/10.1007/s12559-017-9533-x

EEG signals are time-domain series, and we could analyze

them either in the time domain [11] or in the frequency do-

main [12], or the combination of them. There are five bands of

former three bands are associated with sleep or relaxation,

but the remaining two bands are different. Beta waves (14–

30 Hz) [16] are the Bwaking consciousness and reasoning

waves,^ which are tightly related to o ur thinking process.

Beta waves are significant in the effective functioning

throughout the day. Researchers proved that they may trans-

late into stress, anxiety, and restlessness. This band has been

intensively investigated. Gamma waves (31–50 Hz) [17]have

the highest frequency, and we know little about them. Initial

research shows Gamma waves are associated with bursts of

solve this problem, researchers introduced window functions

to analyze signals within a short span of time, in which the

process could be thought to be stationary. Hadjidimitriou et al.

employed spectrogram, Hilbert-Huang spectrum, and Zhao-

Atlas-Marks transform to classify ratings of liking and famil-

iarity [21]. Li et al. applied wavelet energy to compensate for

the influence of the non-stationary of EEG series [22]. Wang

et al. compared power spectrum, wavelet, and nonlinear dy-

namic feature to classify emotion states and indicated that

power spectrum was superior to the other two features [23].

autoregressive parameters, fractal dimension, and sample en-

tropy [25, 26].

Based on the aforementioned features, researchers have

constructed numerous models for emotion reading, e.g., [27]

used SVM to classify happiness, relaxation, sadness, and fear,

and [28] introduced autoregression modeling to judge whether

the subject was in the positive or negative emotion state on

band. However, the models they used were shallow, and the

low SNR was still the major frustration for classification. In

order to reduce the influence of the uncertainness, EEG signal

to

the number of features considered [29]. Therefore, we need

more powerful models to learn efficient representations for the

EEG-based brain decoding.

deep learning (DL) has dominated the machine learning field

and showed advantage in some complex tasks [31, 32]. The

most common structures include HCNN [30], SAE [33], and

deep belief networks (DBN) [34]. In the traditional models,

the feature configuration is fixed, and the classifiers spared no

ing together, and thus yields the optimum matching of them.

In practice, deep networks need a great deal of training cases

as they have plenty of parameters. As EEG brings no harm to

human body during long-term experiment, the data amount

requirement is not a barrier.

signals. Provost et al. proposed an automatic emotion recog-

nition system on the basis of DBN in an audiovisual task and

offered the performance comparison with SVM [35]. They

find that the learned high-order nonlinear relationships provid-