Hits:

Indexed by:期刊论文

Date of Publication:2019-01-01

Journal:IEEE ACCESS

Included Journals:SCIE

Volume:7

Page Number:9540-9557

ISSN No.:2169-3536

Key Words:Feedforward neural networks; pruning hidden layer nodes and weights; group L-1(/2); smooth group L-1/2; group lasso; convergence

Abstract:A group L-1(/2) regularization term is defined and introduced into the conventional error function for pruning the hidden layer nodes of feedforward neural networks. This group L-1(/2) regularization method (GL(1/2)) can prune not only the redundant hidden nodes but also the redundant weights of the surviving hidden nodes of the neural networks. As a comparison, the popular group lasso regularization (GL(2)) can prune the redundant hidden nodes, but cannot prune any redundant weights of the surviving hidden nodes, of the neural networks. A disadvantage of the GL(1/2) is that it involves a non-smooth absolute value function, which causes oscillation in the numerical computation and difficulty in the convergence analysis. As a remedy, the absolute value function is approximated by a smooth function, resulting in a smooth group L-1(/2) regularization method (SGL(1/2)). Numerical simulations on a few benchmark data sets show that, compared with GL(2), SGL(1/2) can achieve better accuracy and remove more redundant nodes and weights of the surviving hidden nodes. A convergence theorem is also proved for SGL(1/2).