Hits:

Indexed by:期刊论文

Date of Publication:2009-06-01

Journal:IEEE TRANSACTIONS ON NEURAL NETWORKS

Included Journals:SCIE、EI、PubMed

Volume:20

Issue:6

Page Number:1050-1054

ISSN No.:1045-9227

Key Words:Boundedness; convergence; feedforward neural networks; online gradient method; penalty

Abstract:In this brief, we consider an online gradient method with penalty for training feedforward neural networks. Specifically, the penalty is a term proportional to the norm of the weights. Its roles in the method are to control the magnitude of the weights and to improve the generalization performance of the network. By proving that the weights are automatically bounded in the network training with penalty, we simplify the conditions that are required for convergence of online gradient method in literature. A numerical example is given to support the theoretical analysis.