Hits:

Indexed by:期刊论文

Date of Publication:2010-12-01

Journal:Journal of Information and Computational Science

Included Journals:EI、Scopus

Volume:7

Issue:12

Page Number:2420-2428

ISSN No.:15487741

Abstract:Chinese Part-Of-Speech Tagging is a basic task in the field of Chinese information processing. This paper builds a Chinese POS tagger by combining Maximum Entropy Model with Chinese Word Clustering, solving the problem of data sparseness especially. First, we have a tagging by Maximum Entropy model as a baseline. Second, we have a bottom-to-up hierarchical Chinese Word Clustering, which clusters all the words in the corpus into 1024 clusters automatically. Then the word clusters act as features, which serves to relieve overfitting caused by data sparseness. According to our experiments, the method achieves a promising result of an accuracy of 93.35%, using 3M Tsinghua Chinese Tree Bank corpus for training, which outperforms the previous method solely based on Maximum Entropy model with the same training size. ? 2010 Binary Information Press.