云计算-文本分类中词语权重计算方法的改进及应用.pdf
版权申诉
52 浏览量
2022-07-06
15:38:24
上传
评论
收藏 619KB PDF 举报
II
Abstract
People can gain more and more knowledge along with the fast development of the
network and information technology. However, in the face of specific knowledge, it is
difficult to obtain it quickly in the vast world of information. Although it has been
solved for some degree by great many kinds of search engines, which only simply
include some key words and the results are vast. So it’s unbenefit for people to find their
specific information. Document automatic classification which is an efficiency
method,has become a valuable technology.In recent years,great many statistics theories
and methods of machine learning have been used for document automatic
classification,which becomes hot research.
One of the most difficulties in document automatic classification is the high
dimension of feature space and the sparseness of text representation vector. In order to
lower the dimension of feature space and improve the efficiency and precision of
classification, it is the first problem of document automatic classification to find an
effective calculation algorithm of words weight. In the research process of Chinese text
classification, this paper focuses on the improvement of calculation algorithm of words
weight, and have completed the following works:
① The traditional calculation algorithm of words weight is mainly studied in this
paper and is found that it has three limitations: 1）it does not take into account the
distribution of feature terms among categories; 2） it does not take into account the
inner-category distribution of feature terms; 3）it does not take into account the
partial-classification of feature terms. From the views of frequency degree,integration
degree, and distribution degree of terms, this paper gives the calculation algorithm of
words weight: TF-IDF-DI-WFDB.
② The proposed measure is introduced which describes the inter-category and
inner-category distribution information of the feature terms by using the inter-category
and inner-category distribution degree of the feature terms in this paper，which forms
the improved algorithm of words weight: TF-IDF-DI. And the traditional calculation
algorithm of words weight does not take into account the partial-classification of feature
terms, this paper introduced word frequency differentia based(WFDB) to make up the
剩余66页未读，继续阅读
资源评论
