Hits:

Indexed by:期刊论文

Date of Publication:2009-06-01

Journal:ICIC Express Letters

Included Journals:EI、Scopus

Volume:3

Issue:2

Page Number:177-182

ISSN No.:1881803X

Abstract:In order to solve problems of the Chinese word segmentation and POS tagging which are still existing in Chinese lexical analysis, a Hidden semi-CRF model, which has two chains of states with unequal number of states, is proposed for the Chinese lexical analysis. The Hidden semi-CRF detects the words together with their part-of-speech regardless whether the words are in the system dictionary or not. A new-words-generating framework is also built for training and testing, under which the definition and distribution of the new words conforms to the characteristic of the ones in real text. The proposed framework enhances the performance ofnew words detecting and POS tagging, so that the overall precision of the system for Chinese lexical analysis could be further increased. The experiment results show that the proposed method is capable of detecting even low frequency new words, which in return increases the overall precision of Chinese word segmentation and POS tagging in Chinese lexical analysis. ? 2009 ISSN.