HierarchicalConvolutionalNeuralNetworksforEEG-BasedEmotionRecognition

研究论文

93 浏览量 2021-02-11 18:30:37 上传评论收藏 1.39MB PDF 举报

资源推荐

资源详情

资源评论

Hierarchical Convolutional Neural Networks for EEG-Based

Emotion Recognition

Jinpeng Li

1,2

& Zhaoxiang Zhang

1,2,3

& Huiguang He

1,2,3

Received: 10 April 2017 /Accepted: 30 November 2017

Springer Science+Business Media, LLC, part of Springer Nature 2017

Abstract

Traditional machine learning methods suffer from severe overfitting in EEG-based emotion reading. In this paper, we use

hierarchical convolutional neural network (HCNN) to classify the positive, neutral, and negative emotion states. We organize

differential entropy features from different channels as two-dimensional maps to train the HCNNs. This approach maintains

information in the spatial topology of electrodes. We use stacked autoencoder (SAE), SVM, and KNN as competing methods.

HCNN yields the highest accuracy, and SAE is slightly inferior. Both of them show absolute advantage over traditional shallow

models including SVM and KNN. We confirm that the high-frequency wave bands Beta and Gamma are the most suitable bands

for emotion reading. We visualize the hidden layers of HCNNs to investigate the feature transformation flow along the hierar-

chical structure. Benefiting from the strong representational learning capacity in the two-dimensional space, HCNN is efficient in

emotion recognition especially on Beta and Gamma waves.

Keywords Affective brain-computer interface

Emotion recognition

Brain wave

Deep learning

EEG

Introduction

Emotion is important when people interact with each other. In

some cases, we also want the machines to interact with us

according to our emotion states. The affective brain-computer

interface (aBCI) [1] is a technique to make it a reality. In

aBCIs, the key problem is to deduce their human user’semo-

tion state, which is the foundation of other functional modules,

e.g., emotion feedback. Engineers hold strong interest in emo-

tion recognition techniques as they are valuable in multiple

applications, e.g., workload estimation [2], driving fatigue

detection [3], and mental state monitoring for pilots [4]. At

the same time, the computational models of emotion might

also help psychologists understand the internal mechanism of

human emotion processing.

There are many clues that contain emotion information. The

first kind is called the Bnon-physiological clue,^ e.g., facial ex-

pression and gesture [5, 6]. For example, [7] fused expression,

speech, and other multimodal information together and discrim-

inated four emotion states. The non-physiological clues are rel-

atively economical. However, signals of this kind are highly

related to the personal habit (culture) of subjects, and thus could

not be universally applied. Several studies have been conducted

concerning this topic [8–10].Thesecondkindiscalledthe

Bphysiological clue,^ e.g., the electric activities of neural cell

clusters across the human cerebral cortex. EEG is used to record

such activities. EEG is reliable in emotion recognition, because

it has relatively objective evaluation on emotion in comparison

with the non-physiological ones [5]. EEG devices often have

multiple channels (electrodes) to collect electric potentials from

different positions, which are either implantable or non-implant-

able. The former has relatively higher signal-to-noise ratio

(SNR), but not fit to the daily use. The latter is noninvasive

and wearable, but the SNR is lower, which is a challenge to

classification. For the convenience in practice, the non-

implantable EEG is favorable to commercial aBCI [10].

* Huiguang He

huiguang.he@ia.ac.cn

Jinpeng Li

lijinpeng2015@ia.ac.cn

Zhaoxiang Zhang

zhaoxiang.zhang@ia.ac.cn

Research Center for Brain-inspired Intelligence, Institute of

Automation, Chinese Academy of Sciences, Beijing, China

University of Chinese Academy of Sciences (UCAS), Beijing, China

Center for Excellence in Brain Science and Intelligence Technology,

Chinese Academy of Sciences, Beijing, China

Cognitive Computation

https://doi.org/10.1007/s12559-017-9533-x

EEG signals are time-domain series, and we could analyze

them either in the time domain [11] or in the frequency do-

main [12], or the combination of them. There are five bands of

brain waves that interest researchers: Delta, Theta, Alpha,

Beta, and Gamma. Delta waves (1–3Hz) [13]aretheslowest

Bsleep waves,^ which are often used to characterize the depth

of sleep. Theta waves (4–7Hz)[14] are believed to be active

in light meditation and sleeping. Alpha waves (8–13 Hz) [15]

are linked with relaxation, happiness, and well-being. The

former three bands are associated with sleep or relaxation,

but the remaining two bands are different. Beta waves (14–

30 Hz) [16] are the Bwaking consciousness and reasoning

waves,^ which are tightly related to o ur thinking process.

Beta waves are significant in the effective functioning

throughout the day. Researchers proved that they may trans-

late into stress, anxiety, and restlessness. This band has been

intensively investigated. Gamma waves (31–50 Hz) [17]have

the highest frequency, and we know little about them. Initial

research shows Gamma waves are associated with bursts of

insight and high-level information processing. Li and Lu [18]

found Gamma waves were suitable for emotion reading with

emotional still images as stimuli. There are many feature ex-

traction techniques to characterize the EEG signals on each

band. Fourier energy [19] is a choice, but the electric activity

at each electrode is not a stationary process over time [20]. To

solve this problem, researchers introduced window functions

to analyze signals within a short span of time, in which the

process could be thought to be stationary. Hadjidimitriou et al.

employed spectrogram, Hilbert-Huang spectrum, and Zhao-

Atlas-Marks transform to classify ratings of liking and famil-

iarity [21]. Li et al. applied wavelet energy to compensate for

the influence of the non-stationary of EEG series [22]. Wang

et al. compared power spectrum, wavelet, and nonlinear dy-

namic feature to classify emotion states and indicated that

power spectrum was superior to the other two features [23].

Sabzpoushan et al. used six types of features and then selected

the top-ranked subset of them to characterize EEG signal for

classification [24]. After a detailed comparison test, Lu et al.

proved that differential entropy (DE) was the most accurate

and stable fea ture for emotion recognition than traditional

features, including power spectrum density (PSD),

autoregressive parameters, fractal dimension, and sample en-

tropy [25, 26].

Based on the aforementioned features, researchers have

constructed numerous models for emotion reading, e.g., [27]

used SVM to classify happiness, relaxation, sadness, and fear,

and [28] introduced autoregression modeling to judge whether

the subject was in the positive or negative emotion state on

band. However, the models they used were shallow, and the

low SNR was still the major frustration for classification. In

order to reduce the influence of the uncertainness, EEG signal

analysis often inv olve s in f eature selection procedures or

hand-crafted signal-representing techniques, e.g., PCA and

Fisher Projection. However, they are insufficient to excavate

patterns in EEG signals as they ignore the original information

such as channels. On the other hand, the cost of traditional

feature selection methods increases quadratically with respect

the number of features considered [29]. Therefore, we need

more powerful models to learn efficient representations for the

EEG-based brain decoding.

Since the work of Hinton and Krizhevsky in 2012 [30],

deep learning (DL) has dominated the machine learning field

and showed advantage in some complex tasks [31, 32]. The

most common structures include HCNN [30], SAE [33], and

deep belief networks (DBN) [34]. In the traditional models,

the feature configuration is fixed, and the classifiers spared no

effort to fit the task according to the features. As a contrast, DL

transforms the features from layer to layer dynamically in the

hierarchical structure and automatically learn the optimized

combination of them. In DL, the feature transformation and

classifier training are no longer independent procedures, be-

cause it binds the classifier training and representational learn-

ing together, and thus yields the optimum matching of them.

In practice, deep networks need a great deal of training cases

as they have plenty of parameters. As EEG brings no harm to

human body during long-term experiment, the data amount

requirement is not a barrier.

Recently, some researchers have used DL to analyze EEG

signals. Provost et al. proposed an automatic emotion recog-

nition system on the basis of DBN in an audiovisual task and

offered the performance comparison with SVM [35]. They

find that the learned high-order nonlinear relationships provid-

ed by deep networks were more effective for emotion recog-

nition. Israsena et al. calculated the PSD features of 32-

channel EEG signals and applied PCA to extract the most

important components of initial input featur es. They used

SAE to learn the nonlinear representations and classified the

level of valence and arousal. Their results indicate that SAE is

better than SVM and naive Bayes classifiers [36]. In our work,

we also use SAE as a comparison method with HCNN, and

the PCA operation is removed to maintain the channel infor-

mation. Graeser et al. proposed to discriminate different men-

tal states according to EEG signals from six electrodes via a

shallow HCNN [37]. They organized the EEG signals as 2-D

maps where each column corresponded to one electrode, and

each row was one sample recorded at an interval of time res-

olution. Although their electrode number was quite limited,

they showed the advantage of DL in EEG-based recognition.

The two dimensions were electrode and time; whereas our

work, both dimensions refer to electrode. We will discuss it

in detail in BMethods.^

Some researchers have shown that the individual difference

of EEG signals across subjects is remarkable [20, 38, 39].

From person to person, the EEG signal differs by a large scale.

Rothkrantz et al. showed that the subject-specific EEG data

contained enough information to recognize emotion state, but

Cogn Comput

剩余12页未读，继续阅读

评论收藏

内容反馈

weixin_38693476

粉丝: 1
资源: 949

Hierarchical Convolutional Neural Networks for EEG-Based Emotion...

最新资源

Hierarchical Convolutional Neural Networks for EEG-Based Emotion...

Hierarchical Neural Networks for Image Interpretation

Facial Expression Recognition with Convolutional Neural Networks

斯坦福CS231n Convolutional Neural Networks for Visual Recognition lecture 1-5

[斯坦福CS231n课程整理] Convolutional Neural Networks for Visual Recognition.rar

Convolutional Neural Networks for Visual Recognition 6

2019-MINCUT POOLING IN GRAPH NEURAL NETWORKS-网文-rrrr1

A Survey of the Recent Architectures of Deep Convolutional Neural Networks.pdf

Pattern Recognition with Neural Networks in C++

Hierarchical-Double-Attention-Neural-Networks-for-Sentiment-Classification:分层双注意力神经网络在情感分类中的应用

Pattern Recognition using Neural and Functional Networks(Springer2009新书)

Convolutional Neural Networks for Visual Recognition.pdf

Convolutional Neural Networks for Visual Recognition 7

Convolutional Neural Networks for Visual Recognition 8

Convolutional Neural Networks for Visual Recognition 2

Convolutional Neural Networks for Visual Recognition 4

code for Holistically-Nested Edge Detection

2019-IJCAI-自动所-Hierarchical Graph Convolutional Networks for Sem

multi-label-classification.pdf

Hierarchical Deep Click Feature Prediction for Fine-grained Image Recognition

Convolutional Neural Networks for Visual Recognition 5

斯坦福大学公开课 CS231n_Convolutional_Neural_Networks_for_Visual_Recognition PPT

Convolutional Neural Networks for Visual Recognition 3

Convolutional Neural Networks for Visual Recognition 1

Tomato Leaf Disease Detection System Based on Deep Convolutional Neural Networks

Video-based Sign Language Recognition without Temporal Segmentation.pdf

Hierarchical-Recurrent-Neural-Networks-for-Speech-Bandwidth-Extension:论文编号

A hierarchical ontology context model for work-based learning

ExtremeLearningMachine资源共享-New-delay-variation-dependent-stability-for-neural-networks-_2013_Neurocompu.pdf

最新资源