论文名称：Chinese POS tagging employing maximum entropy and word clustering
论文类型：期刊论文
发表刊物：Journal of Information and Computational Science
收录刊物：EI、Scopus
卷号：7
期号：12
页面范围：2420-2428
ISSN号：15487741
摘要：Chinese Part-Of-Speech Tagging is a basic task in the field of Chinese information processing. This paper builds a Chinese POS tagger by combining Maximum Entropy Model with Chinese Word Clustering, solving the problem of data sparseness especially. First, we have a tagging by Maximum Entropy model as a baseline. Second, we have a bottom-to-up hierarchical Chinese Word Clustering, which clusters all the words in the corpus into 1024 clusters automatically. Then the word clusters act as features, which serves to relieve overfitting caused by data sparseness. According to our experiments, the method achieves a promising result of an accuracy of 93.35%, using 3M Tsinghua Chinese Tree Bank corpus for training, which outperforms the previous method solely based on Maximum Entropy model with the same training size. ? 2010 Binary Information Press.
发表时间：2010-12-01