Publications related to Saddle point

Photothermal spectroscopy on-chip sensor for the measurement of a PMMA film using a silicon nitride micro-ring resonator and an external cavity quantum cascade laser

Laser-based mid-infrared (mid-IR) photothermal spectroscopy (PTS) represents a selective, fast, and sensitive analytical technique. Recent developments in laser design permits the coverage of wider spectral regions in combination with higher power, enablin ...

Walter De Gruyter Gmbh2024

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

A Theory of Finite-Width Neural Networks: Generalization, Scaling Laws, and the Loss Landscape

Berfin Simsek

Deep learning has achieved remarkable success in various challenging tasks such as generating images from natural language or engaging in lengthy conversations with humans. The success in practice stems from the ability to successfully train massive neural ...

EPFL2023

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy

Nicolas Boumal

We consider the problem of provably finding a stationary point of a smooth function to be minimized on the variety of bounded-rank matrices. This turns out to be unexpectedly delicate. We trace the difficulty back to a geometric obstacle: On a nonsmooth se ...

SPRINGER HEIDELBERG2022

Preconditioners for robust optimal control problems under uncertainty

Fabio Nobile, Tommaso Vanzan

The discretization of robust quadratic optimal control problems under uncertainty using the finite element method and the stochastic collocation method leads to large saddle-point systems, which are fully coupled across the random realizations. Despite its ...

2023

Development of a new method to determine the axial void velocity profile in BWRs from measurements of the in-core neutron noise

Mathieu Hursin

Determination of the local void fraction in BWRs from in-core neutron noise measurements requires the knowledge of the axial velocity of the void. The purpose of this paper is to revisit the problem of determining the axial void velocity profile from the t ...

PERGAMON-ELSEVIER SCIENCE LTD2021

Distributed Learning in Non-Convex Environments-Part II: Polynomial Escape From Saddle-Points

Ali H. Sayed, Stefan Vlaski

The diffusion strategy for distributed learning from streaming data employs local stochastic gradient updates along with exchange of iterates over neighborhoods. In Part I [3] of this work we established that agents cluster around a network centroid and pr ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

Volkan Cevher, Ali Kavis

In this work we investigate stochastic non-convex optimization problems wherethe objective is an expectation over smooth loss functions, and the goal is to find an approximate stationary point. The most popular approach to handling such problems is varianc ...

2021

Linear Speedup In Saddle-Point Escape For Decentralized Non-Convex Optimization

Ali H. Sayed, Stefan Vlaski

Under appropriate cooperation protocols and parameter choices, fully decentralized solutions for stochastic optimization have been shown to match the performance of centralized solutions and result in linear speedup (in the number of agents) relative to no ...

IEEE2020

A Catalyst Framework for Minimax Optimization

Negar Kiyavash

We introduce a generic two-loop scheme for smooth minimax optimization with strongly-convex-concave objectives. Our approach applies the accelerated proximal point framework (or Catalyst) to the associated dual problem and takes full advantage of existing ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2020

Second-Order Guarantees In Federated Learning

Ali H. Sayed, Stefan Vlaski, Elsa Rizk

Federated learning is a useful framework for centralized learning from distributed data under practical considerations of heterogeneity, asynchrony, and privacy. Federated architectures are frequently deployed in deep learning settings, which generally giv ...

IEEE2020

Bulk and Surface Electronic Structure of the Dual-Topology Semimetal Pt2HgSe3

Nicola Marzari, Marco Gibertini, Antimo Marrazzo

We report high-resolution angle-resolved photoemission measurements on single crystals of Pt2HgSe3 grown by high-pressure synthesis. Our data reveal a gapped Dirac nodal line whose (001) projection separates the surface Brillouin zone in topological and tr ...

AMER PHYSICAL SOC2020

An Inexact Augmented Lagrangian Framework for Nonconvex Optimization with Nonlinear Constraints

Volkan Cevher, Mehmet Fatih Sahin, Ahmet Alacaoglu, Armin Eftekhari, Fabian Ricardo Latorre Gomez

We propose a practical inexact augmented Lagrangian method (iALM) for nonconvex problems with nonlinear constraints. We characterize the total computational complexity of our method subject to a verifiable geometric condition, which is closely related to t ...

2019

Polynomial Escape-Time From Saddle Points In Distributed Non-Convex Optimization

Ali H. Sayed, Stefan Vlaski

The diffusion strategy for distributed learning from streaming data employs local stochastic gradient updates along with exchange of iterates over neighborhoods. In this work we establish that agents cluster around a network centroid in the mean-fourth sen ...

IEEE2019

Non-perturbative probability distribution function for cosmological counts in cells

Sergey Sibiryakov, Mikhail Ivanov

We present a non-perturbative calculation of the 1-point probability distribution function (PDF) for the spherically-averaged matter density field. The PDF is represented as a path integral and is evaluated using the saddle-point method. It factorizes into ...

IOP PUBLISHING LTD2019

A Data-driven Approach to Robust Control of Multivariable Systems by Convex Optimization

Alireza Karimi, Christoph Manuel Kammer

The frequency-domain data of a multivariable system in different operating points is used to design a robust controller with respect to the measurement noise and multimodel uncertainty. The controller is fully parametrized in terms of matrix polynomial fun ...

2017

Higgs-Dilaton cosmology: Universality versus criticality

Mikhail Chapochnikov, Javier Rubio

The Higgs-Dilaton model is able to produce an early inflationary expansion followed by a dark energy dominated era responsible for the late time acceleration of the Universe. At tree level, the model predicts a small tensor-to-scalar ratio (0.0021

American Physical Society2014

Reduced basis approximation and a posteriori error estimation for Stokes flows in parametrized geometries: roles of the inf-sup stability constants

Gianluigi Rozza, Andrea Manzoni

In this paper we review and we extend the reduced basis approximation and a posteriori error estimation for steady Stokes flows in affinely parametrized geometries, focusing on the role played by the Brezzi's and Babuska's stability constants. The crucial ...

Springer2013