Hits:

Indexed by:期刊论文

Date of Publication:2014-02-01

Journal:NEURAL NETWORKS

Included Journals:SCIE、EI、PubMed、Scopus

Volume:50

Page Number:72-78

ISSN No.:0893-6080

Key Words:Feedforward neural networks; Batch gradient method; Smoothing L-1/2 regularization; Convergence

Abstract:The aim of this paper is to develop a novel method to prune feedforward neural networks by introducing an L-1/2 regularization term into the error function. This procedure forces weights to become smaller during the training and can eventually removed after the training. The usual L-1/2 regularization term involves absolute values and is not differentiable at the origin, which typically causes oscillation of the gradient of the error function during the training. A key point of this paper is to modify the usual L-1/2 regularization term by smoothing it at the origin. This approach offers the following three advantages: First, it removes the oscillation of the gradient value. Secondly, it gives better pruning, namely the final weights to be removed are smaller than those produced through the usual L-1/2 regularization. Thirdly, it makes it possible to prove the convergence of the training. Supporting numerical examples are also provided. (C) 2013 Elsevier Ltd. All rights reserved.