Attention-based LSTM-CNNs for Uncertainty
Identification on Chinese Social Media Texts
Binyang Li
School of Information
Science and Technology
University of
International Relations
Beijing, China
byli@uir.edu.cn
Kaiming Zhou
School of Information
Science and Technology
University of
International Relations
Beijing, China
kmzhou@uir.edu.cn
Wei Gao
School of Information
Management
Victoria University of
Wellington
Wellington, New Zealand
wei.gao@vuw.ac.nz
Xu Han
School of Information
Science and Technology
Capital Normal
University
Beijing, China
csxhan@cnu.edu.cn
Linna Zhou
School of Information
Science and Technology
University of
International Relations
Beijing, China
lnzhou@uir.edu.cn
Abstract—Uncertainty identification is an important semantic
processing task, which is crucial to the quality of information in
terms of factuality in many techniques, e.g. topic detection,
question answering. Especially in social media, the texts are
written informally which are widely used in many applications, so
the factuality has become a premier concern. However, existing
approaches that still rely on lexical cues suffer greatly from the
casual or word-of-mouth peculiarity of social media, in which the
cue phrases are often expressed in sub-standard form or even
omitted from sentences. To tackle these problems, this paper
proposes the attention-based LSTM-CNNs for the uncertainty
identification on social media texts, named ALUNI. ALUNI
incorporates attention-based LSTM networks to represent the
semantics of words, and convolutional neural networks to capture
the most important semantics of uncertainty for identification.
Experiments are conducted on both Chinese Weibo and news
datasets, and 78.19% and 73.95% of F1-measure scores are
achieved with 11% and 3% improvement over the baseline,
respectively.
Keywords—LSTM, CNN, uncertainty identification, social
media
I. INTRODUCTION
“Uncertainty - in its most general sense - can be interpreted as
lack of information: the receiver of the information (i.e., the
hearer or the reader) cannot be certain about some pieces of
information” [1]. The identification of uncertainty is significant
to the trustworthiness of many natural language processing
techniques and applications, such as question answering,
information extraction, and so on [2].
The CoNLL-2010 Shared Task aimed at identifying
uncertainty in biological papers and Wikipedia articles written
in English [3] [4]. Most participants utilized linguistics features,
e.g., lexical cues, Part-Of-Speech (POS), to detect the uncertain
sentences from the texts.
Recently, with the growing popularity of social media, there
exist more and more texts consisting of casual or word-of-mouth
expressions. The quality of information in social media in terms
of factuality has become a premier concern [5]. The generation
and propagation of uncertain information will lead to rumor
flooding among social media and even influence the real world.
For example, the 2011 London Riots occurred owing to the
spread of uncertain in-formation among social media, such as
Corresponding author
Twitter or Facebook. Therefore, uncertainty identification, i.e.,
identifying uncertain sentences is becoming increasingly critical
for users to synthesize information to derive reliable
interpretation.
However, unlike the biological papers and Wikipedia
articles, the texts in social media are usually short and informal.
Due to the word count limit and casual expression, many cue
phrases are expressed in substandard shape or even omitted from
sentences. In this case, the uncertain semantics will be implicitly
conveyed by the whole sentence rather than explicitly by cue
phrases. Existing approaches based on cue phrases for
uncertainty identification are ineffective for social media texts,
and they are also not good enough for formal text uncertain
identification. It is noteworthy that in the CoNLL-2010 Shared
Task, the participants all achieved better results on biological
dataset than wiki dataset. It indicated the more formal the article
is, the easier it is to judge the sentence uncertainty. As a result,
uncertainty identification on Chinese social media texts has
become a big challenge which needs more semantics
information to solve.
We tried to judge the uncertainty of the Chinese text of social
media based on semantics, so we turned to deep learning which
could express the semantics of words and sentences well.
Bahdanau et al. apply the RNN with attention mechanism to
machine translation [6], their model makes the words’ semantics
and the relation between words in both languages clearer. Kim
utilizes CNNs to classify sentences and achieves good results [7],
which shows CNNs have a unique advantage in both image and
text classifying issues. Considering these above researches we
decided to combine the two model structures to solve
uncertainty identification problem.
This paper proposes Attention-based Long Short-Term
Memory-Convolutional Neural Networks (LSTM-CNNs) for
Uncertainty Identification on social media texts, named ALUNI.
ALUNI incorporates attention mechanisms into LSTM
networks to represent the semantics of the context in a sentence,
and uses CNNs for the uncertainty identification. Benefitted
from the attention mechanisms, the key elements of sentences
can be highlighted and the hidden semantics can be captured,
which will enable us to detect uncertainty based on the context
of the whole sentence instead of depending on the cue-phrases.
资源评论
