A survey of Chinese word segmentation development and
techniques
Abstract
In Chinese natural language processing tasks, Chinese word segmentation is the most
basic task, and has long been the premise and cornerstone of other tasks. Reviewing the
development of Chinese word segmentation technology and making a summary and
conclusion can provide guidance for future research. This article sorts out the necessity and
key issues of Chinese word segmentation, gives a brief introduction to the typical algorithms
and models of mechanical word segmentation, machine learning and deep learning, and
gives an analysis of the advantages and disadvantages. At the same time, 31 Chinese and
English papers from various journals and conferences were referred to support my opinions.
The results show that the current deep learning-based model is the most prominent, and it
has further improved the accuracy of Chinese word segmentation compared with the
existing results. This article hasn’t yet conducted in-depth discussion and comparison of the
excellent models in recent years. In the field of Chinese word segmentation, there are still
areas worthy of research and development in the future, such as integrated algorithms, joint
models, transfer learning, and multi-modality.
Key Words: Chinese Word Segmentation; Machine Learing; Deep Learning;
Semantic Understanding
评论0