Time: 2023/07/05 13:30 - 14:20

Title: A Theory of the NEPv Approach for Optimization On the Stiefel Manifold Part I: NPDo

 

【Talk II】

Abstract

The NEPv approach has been increasingly used lately for optimization on the Stiefel manifold arising from machine learning. General speaking, the approach first turns the first order optimality condition into a nonlinear eigenvalue problem with eigenvector dependency (NEPv) and then solve the nonlinear problem via some variations of the self-consistent-field (SCF) iteration. The difficulty, however, lies in designing a proper SCF iteration so that a maximizer is found at the end. Currently, each use of the approach is very much individualized, especially in its convergence analysis to show whether the approach works or not. Related, the NPDo approach is recently proposed for the sum of coupled traces and it seeks to turn the first order optimality condition into a nonlinear polar decomposition with orthogonal factor dependency (NPDo). 

 

This talk consists of two parts, approximately one hour each. Part I is about a unifying framework for the NPDo approach, while Part II is about a unifying framework for the NEPV approach. The frameworks are built upon some basic assumptions, with which globally convergence to a stationary point is guaranteed and during the SCF iterative process that leads to the stationary point, the objective function increases monotonically. Also notion of atomic function is proposed, which includes commonly used matrix traces of linear and quadratic forms as special ones. It is shown that the basic assumptions are satisfied by atomic functions and, more importantly, by convex compositions of the atomic functions. Together they provide a large collection of objectives for which the NEPv approach and the NPDo approach are guaranteed to work.

 

Ren-Cang Li is a Professor of Mathematics in University of Texas at Arlington. Most recently, he was a Chair Professor at Hong Kong Baptist University. He received his Ph.D. from UC Berkeley in 1995. He was awarded the 1995 Householder Fellowship in Scientific Computing by ORNL, a Friedman memorial prize in Applied Mathematics from UC Berkeley in 1996, a CAREER award from NSF in 1999, distinguished paper awards from ICCM in 2018, 2019, and 2020, and most recently the 2019 INFORMS’ SAS Data Mining Best Paper award. His secular equation solver and codes sit at the kernel of MATLAB’s eig and svd (through LAPACK) that are being used around the o’clock. He helped HP in developing its libm library for HP Itanium computers in 2001. His research interest includes floating-point support for scientific computing, numerical linear algebra, reduced order modeling, large scale eigenvalue computations, nonlinear manifold learning, optimizations on manifolds, data sciences, and unconventional schemes for ordinary differential equations. He is on the editorial boards of several internal journals, and a co-EiC of NACO. Previously, he served as an associate editor of SIMAX.