openEAR - Introducing the Munich Open-Source
Emotion and Affect Recognition Toolkit
Florian Eyben, Martin W
¨
ollmer, and Bj
¨
orn Schuller
Technische Universit
¨
at M
¨
unchen, Institute for Human-Machine Communication
Theresienstrasse 90, 80333 M
¨
unchen
{eyben|woellmer|schuller}@tum.de
Abstract
Various open-source toolkits exist for speech recognition
and speech processing. These toolkits have brought a great
benefit to the research community, i.e. speeding up research.
Yet, no such freely available toolkit exists for automatic af-
fect recognition from speech. We herein introduce a novel
open-source affect and emotion recognition engine, which
integrates all necessary components in one highly efficient
software package. The components include audio recording
and audio file reading, state-of-the-art paralinguistic fea-
ture extraction and plugable classification modules. In this
paper we introduce the engine and extensive baseline re-
sults. Pre-trained models for four affect recognition tasks
are included in the openEAR distribution. The engine is tai-
lored for multi-threaded, incremental on-line processing of
live input in real-time, however it can also be used for batch
processing of databases.
1. Introduction
Affective Computing has become a popular area of re-
search in recent times [17]. Many achievements have been
made towards making machines detect and understand hu-
man affective states, such as emotion, interest or dialogue
role. Yet, in contrast to the field of speech recognition, only
very few software toolkits exist, which are tailored specif-
ically for affect recognition from audio or video. In this
paper, we introduce and describe the Munich open Affect
Recognition Toolkit (openEAR), the first such tool, which
runs on multiple platforms and is publicly available
1
.
OpenEAR in it’s initial version is introduced as an affect
and emotion recognition toolkit for audio and speech affect
recognition. However, openEAR’s architecture is modular
and by principle modality independent. Thus, also vision
features such as facial points or optical flow measures can
1
http://sourceforge.net/projects/openear
be added and fused with audio features. Moreover, phys-
iological features such as heart rate, ECG, or EEG signals
from devices such as the Neural Impulse Actuator (NIA),
can be analysed using the same methods and algorithms
as for speech signals and thus can also be processed us-
ing openEAR – provided suitable capture interfaces and
databases.
2. Existing work
A few free toolkits exist, that provide various compo-
nents usable for emotion recognition. Most toolkits that in-
clude feature extraction algorithms are targeted at speech
recognition and speech processing, such as the Hidden
Markov Toolkit (HTK) [16], the PRAAT Software [1], the
Speech Filling System (SFS) from UCL, and the SNACK
package for the Tcl scripting language. These can all be
used to extract state-of-the-art features for emotion recog-
nition. However, only PRAAT and HTK include certain
classifiers. For further classifiers WEKA and RapidMiner,
for example, can be used. Moreover, only few of the listed
toolkits are available under a permissive Open-Source li-
cense, e. g. WEKA, PRAAT, and RapidMiner.
The most complete and task specific framework for
Emotion Recognition currently is EmoVoice [13]. How-
ever, the main design objective is to provide an emotion
recognition system for the non-expert. Thus it is a great
framework for demonstrator applications and making emo-
tion recognition available to the non-expert. openEAR, in
contrast, aims at being a stable and efficient set of tools
for researchers and those developing emotional aware ap-
plications, providing the elementary functionality for emo-
tion recognition, i. e. the Swiss Army Knife for research and
development of affect aware applications. openEAR com-
bines everything from audio recording, feature extraction,
and classification to evaluation of results, and pre-trained
models while being very fast and highly efficient. All fea-
ture extractor components are written in C++ and can be
used as a library, facilitating integration into custom appli-
978-1-4244-4799-2/09/$25.00
c
2009 IEEE
