论文成果
Chinese POS tagging employing maximum entropy and word clustering
- 点击次数：
- 论文类型：期刊论文
- 发表时间：2010-12-01
- 发表刊物：Journal of Information and Computational Science
- 收录刊物：EI、Scopus
- 文献类型：J
- 卷号：7
- 期号：12
- 页面范围：2420-2428
- ISSN号：15487741
- 摘要：Chinese Part-Of-Speech Tagging is a basic task in the field of Chinese information processing. This paper builds a Chinese POS tagger by combining Maximum Entropy Model with Chinese Word Clustering, solving the problem of data sparseness especially. First, we have a tagging by Maximum Entropy model as a baseline. Second, we have a bottom-to-up hierarchical Chinese Word Clustering, which clusters all the words in the corpus into 1024 clusters automatically. Then the word clusters act as features, which serves to relieve overfitting caused by data sparseness. According to our experiments, the method achieves a promising result of an accuracy of 93.35%, using 3M Tsinghua Chinese Tree Bank corpus for training, which outperforms the previous method solely based on Maximum Entropy model with the same training size. ? 2010 Binary Information Press.
