How Universal is Genre in Universal Dependencies?
Max Müller-Eberstein and Rob van der Goot and Barbara Plank
Department of Computer Science
IT University of Copenhagen, Denmark
mamy@itu.dk, robv@itu.dk, bapl@itu.dk
Abstract
This work provides the first in-depth analysis of genre in Universal Dependencies (UD). In con-
trast to prior work on genre identification which uses small sets of well-defined labels in mono-
/bilingual setups, UD contains 18 genres with varying degrees of specificity spread across 114
languages. As most treebanks are labeled with multiple genres while lacking annotations about
which instances belong to which genre, we propose four methods for predicting instance-level
genre using weak supervision from treebank metadata. The proposed methods recover instance-
level genre better than competitive baselines as measured on a subset of UD with labeled in-
stances and adhere better to the global expected distribution. Our analysis sheds light on prior
work using UD genre metadata for treebank selection, finding that metadata alone are a noisy
signal and must be disentangled within treebanks before it can be universally applied.
1 Introduction
Identifying document genre automatically has long been of interest to the NLP community due to its
immediate applications both in document grouping (Petrenz, 2012) as well as task-specific data selec-
tion (Ruder and Plank, 2017; Sato et al., 2017).
Cross-lingual genre identification has however remained a challenge, mainly due to the lack of stable
cross-lingual representations (Petrenz, 2012). Recent work has shown that pre-trained masked language
models (MLMs) capture monolingual genre (Aharoni and Goldberg, 2020). Do such distinctions man-
ifest in highly multilingual spaces as well? In this work, we investigate whether this property holds for
the genre distribution in the 114 language Universal Dependencies corpus (UD version 2.8; Zeman et al.,
2021) using the multilingual mBERT MLM (Devlin et al., 2019).
In absence of an exact definition of textual genre (Kessler et al., 1997; Webber, 2009; Plank, 2016), this
work will focus on the information specifically denoted by the genres metadata tag in UD. We hope
that an in-depth, cross-lingual analysis of what this label represents will enable practitioners to better
control for the effects of domain shift in their experiments. Previous work using these UD metadata
for proxy training data selection have produced mixed results (Stymne, 2020). We investigate possible
reasons and identify inconsistencies in genre annotation. The fact that genre labels are only available at
the level of treebanks makes it difficult to gather a clear picture of the sentence-level genre distribution
— especially with some treebanks having up to 10 genre labels. We therefore investigate the degree to
which instance-level genre is recoverable using only the treebank-level metadata as weak supervision.
Our contributions entail the, to our knowledge, first detailed definition of all UD metadata genre labels
(Section 3), four weakly supervised methods for extracting instance-level genre across 114 languages
(Section 4) as well as genre identification experiments which show that our proposed two-step procedure
allows for effective genre recovery in multilingual setups where language relatedness typically outweighs
genre similarities (Section 5).
1
This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http://
creativecommons.org/licenses/by/4.0/.
1
Code available at https://personads.me/x/syntaxfest-2021-code.
arXiv:2112.04971v1 [cs.CL] 9 Dec 2021
剩余16页未读，继续阅读
资源评论
