点击次数：

论文类型：期刊论文

发表时间：2014-12-01

发表刊物：JOURNAL OF SUPERCOMPUTING

收录刊物：SCIE、EI

卷号：70

期号：3

页面范围：1249-1259

ISSN号：0920-8542

关键字：K-means; MapReduce; Sampling; Performance

摘要：Clustering analysis is one of the most commonly used data processing algorithms. Over half a century, K-means remains the most popular clustering algorithm because of its simplicity. Recently, as data volume continues to rise, some researchers turn to MapReduce to get high performance. However, MapReduce is unsuitable for iterated algorithms owing to repeated times of restarting jobs, big data reading and shuffling. In this paper, we address the problems of processing large-scale data using K-means clustering algorithm and propose a novel processing model in MapReduce to eliminate the iteration dependence and obtain high performance. We analyze and implement our idea. Extensive experiments on our cluster demonstrate that our proposed methods are efficient, robust and scalable.