04/20/2023
By Joris Roos
The talk will be delivered by Duy Nhat Phan, Post Doc Research Associate in the Department of Mathematics & Statistics, UMass Lowell.
Title: Stochastic Variance-Reduced Majorization-Minimization Algorithms
Date: Wednesday, April 26
Time: 12:30-1:30 p.m.
Room: Shah Hall 308 or via Zoom
Abstract: In this talk, we focus on a class of nonconvex nonsmooth optimization problems in which the objective is a sum of two functions; one function is the average of a large number of differentiable functions, while the other function is proper, lower semicontinuous and has a surrogate function that satisfies standard assumptions. Such problems arise in machine learning and regularized empirical risk minimization applications. However, nonconvexity and the large-sum structure make such problems challenging for designing new algorithms. Consequently, algorithms which can be effectively applied in such scenarios are scarce. We introduce and study three stochastic variance-reduced majorization-minimization (MM) algorithms, combining the general MM principle with new variance-reduced techniques. We study the almost surely subsequential convergence of the generated sequence to a stationary point and prove that our algorithms achieve the best-known complexity bounds in terms of the number of gradient evaluations. We demonstrate the effectiveness of our algorithms on sparse binary classification problems, sparse multi-class logistic regressions, and neural networks using several widely-used and publicly available data sets.