Chapter 1
Introduction
The Word Vector Tool (WVTool) is a flexible Java library for statistical language
modeling. In particular it is used to create word vector representations of text
documents in the vector space model [1]. In the vector space model, a document
is represented by a vector that denotes the relevance of a given set of terms for
this document. Terms are usually natural language words, but they can also be
more general entities, as words that are reduced to some linguistic base form or
abstract concept as “<number>” denoting any occurrence of a number in the
text.
agent Java <number> ...
doc1.txt 1.0 0.3 0.0 ...
doc2.txt 0.9 0.0 0.6 ...
... ... ... ... ...
From the early days of automatic text processing and information retrieval, the
vector space model has played a very important role. It is the point of depar-
ture for many automatic text processing tasks, as text classification, clustering,
characterization and summarization as well as information retrieval [2].
The aim of the Java WVTool is to provide a simple to use, simple to extend
pure Java library for creating word vectors. It can be easily invoked from any
Java application. Furthermore, the tool is tightly integrated with the Rapid-
Miner data mining environment [3], allowing to perform diverse experiments
using textual data directly. In this way, the WVTool bridges a gap between
highly sophisticated linguistic packages as the GATE system [11] on the one side
and many partial solutions that are part of diverse text and information retrieval
applications on the other side. Closest related to the Word Vector Tool is the
Bow package [10], which is a C library, for the creation of word vectors and
clustering/classifying text.
5
shuimoxianglan2014-05-11中文就好了
jiexinglongtao2013-11-18需要中文哦
