Title of Paper:A STOCHASTIC SEMISMOOTH NEWTON METHOD FOR NONSMOOTH NONCONVEX OPTIMIZATION

Hits:

Date of Publication:2019-01-01

Journal:SIAM JOURNAL ON OPTIMIZATION

Included Journals:EI、SCIE

Volume:29

Issue:4

Page Number:2916-2948

ISSN No.:1052-6234

Key Words:nonsmooth stochastic optimization; stochastic approximation; semismooth Newton method; stochastic second order information; global convergence

Abstract:In this work, we present a globalized stochastic semismooth Newton method for solving stochastic optimization problems involving smooth nonconvex and nonsmooth convex terms in the objective function. We assume that only noisy gradient and Hessian information of the smooth part of the objective function is available via calling stochastic first and second order oracles. The proposed method can be seen as a hybrid approach combining stochastic semismooth Newton steps and stochastic proximal gradient steps. Two inexact growth conditions are incorporated to monitor the convergence and the acceptance of the semismooth Newton steps and it is shown that the algorithm converges globally to stationary points in expectation and almost surely. We present numerical results and comparisons on l1-regularized logistic regression and nonconvex binary classification that demonstrate the efficiency of the algorithm.