Hits:

Indexed by:期刊论文

Date of Publication:2018-10-01

Journal:IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS

Included Journals:ESI高被引论文、SCIE、Scopus

Volume:48

Issue:10

Page Number:1657-1666

ISSN No.:2168-2216

Key Words:Back-propagation strategy; big data feature learning; canonical polyadic decomposition (CP-DCM); deep computation model

Abstract:Deep computation models achieve super performance for big data feature learning. However, training a deep computation model poses a significant challenge since a deep computation model typically involves a large number of parameters. Specially, it needs a high-performance computing server with a large-scale memory and a powerful computing unit to train a deep computation model, making it difficult to increase the size of a deep computation model further for big data feature learning on low-end devices such as conventional desktops and portable CPUs. In this paper, we propose an improved deep computation model based on the canonical polyadic decomposition scheme to compress the parameters and to improve the training efficiency. Furthermore, we devise a learning algorithm based on the back-propagation strategy to train the parameters of the proposed model. The learning algorithm can be directly performed on the compressed parameters to improve the training efficiency. Finally, we carry on the experiments on three representative datasets, i.e., CUAVE, SNAE2, and STL-10, to evaluate the performance of the proposed model by comparing with the conventional deep computation model and other two improved deep computation models based on the Tucker decomposition and the tensor-train network. Results demonstrate that the proposed model can compress parameters greatly and improve the training efficiency significantly with a low classification accuracy drop.