arXiv:1607.01759v3 [cs.CL] 9 Aug 2016
Bag of Tricks for Efficient Text Classification
Armand Joulin Edouard Grave Piotr Bojanowski Tomas Mikolov
Facebook AI Research
{ajoulin,egrave,bojanowski,tmikolov}@fb.com
Abstract
This paper explores a simple and efficient
baseline for text classification. Our ex-
periments show that our fast text classi-
fier fastText is often on par with deep
learning classifiers in terms of accuracy, and
many orders of magnitude faster for training
and evaluation. We can train fastText on
more than one billion words in less than ten
minutes using a standard multicore CPU, and
classify half a million sentences among 312K
classes in less than a minute.
1 Introduction
Text classification is an important task in Natural
Language Processing with many applications, such
as web search, information retrieval, ranking and
document classification (Deerwester et al., 1990;
Pang and Lee, 2008). Recently, models based
on neural networks have become increasingly
popular (Kim, 2014; Zhang and LeCun, 2015;
Conneau et al., 2016). While these models achieve
very good performance in practice, they tend to be
relatively slow both at train and test time, limiting
their use on very large datasets.
Meanwhile, linear classifiers are of-
ten considered as strong baselines for text
classification problems (Joachims, 1998;
McCallum and Nigam, 1998; Fan et al., 2008).
Despite their simplicity, they often obtain state-
of-the-art performances if the right features are
used (Wang and Manning, 2012). They also
have the potential to scale to very large cor-
pus (Agarwal et al., 2014).
In this work, we explore ways to scale these
baselines to very large corpus with a large output
space, in the context of text classification. Inspired
by the recent work in efficient word representation
learning (Mikolov et al., 2013; Levy et al., 2015),
we show that linear models with a rank constraint
and a fast loss approximation can train on a billion
words within ten minutes, while achieving perfor-
mance on par with the state-of-the-art. We evalu-
ate the quality of our approach fastText
1
on two
different tasks, namely tag prediction and sentiment
analysis.
2 Model architecture
A simple and efficient baseline for sentence
classification is to represent sentences as bag of
words (BoW) and train a linear classifier, e.g., a
logistic regression or an SVM (Joachims, 1998;
Fan et al., 2008). However, linear classifiers do
not share parameters among features and classes.
This possibly limits their generalization in the
context of large output space where some classes
have very few examples. Common solutions
to this problem are to factorize the linear clas-
sifier into low rank matrices (Schutze, 1992;
Mikolov et al., 2013) or to use multilayer
neural networks (Collobert and Weston, 2008;
Zhang et al., 2015).
Figure 1 shows a simple linear model with rank
constraint. The first weight matrix A is a look-up
table over the words. The word representations are
then averaged into a text representation, which is in
turn fed to a linear classifier. The text representa-
1
https://github.com/facebookresearch/fastText
资源评论
java的世界
- 粉丝: 0
- 资源: 6
最新资源
- 永磁同步电机旋转高频信号注入法零低速无位置控制仿真，相比高频方波信号注入法，旋转高频信号注入法噪声更小损耗更低，该模型注入1000Hz旋转高频电压信号到电机中用于产生激励电流，在低速100rpm下无感
- jQuery实现的带左右切换按钮鼠标滑过图片向上弹出放大动态特效源码.zip
- 电子电气架构中整车网络管理与休眠唤醒机制详解
- jQuery实现点击按钮爆炸式弹出不同颗粒动画特效源码.zip
- 管材切割机sw18全套技术资料100%好用.zip
- 基于STM32环境温湿度监测系统设计（附项目代码zip）
- 钢板折边机sw18全套技术资料100%好用.zip
- IMG_20250104_163933.jpg
- win32汇编环境,理解BeginPaint函数与GetDC函数的区别
- SMTP(2).zip
- 过滤器清洗机step全套技术资料100%好用.zip
- CCNA-200-301.pdf
- 计算机视觉中圆与椭圆超最小二乘拟合算法（HyperLS）的理论与实验研究
- COMSOL 一维电化学模型和热模型 很重要，P2D模型计算 可以得到准确的热源 同时也可以仿真电压得到准确的数据 拟合即可使用 不用担心审稿人问你数据来源
- 轨道式可调铁板分张器sw19可编辑全套技术资料100%好用.zip
- 环形柔性链输送机sw16可编辑全套技术资料100%好用.zip
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
