Publications related to Topologies on spaces of linear maps

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Volkan Cevher, Grigorios Chrysos, Fanghui Liu

Despite the widespread empirical success of ResNet, the generalization properties of deep ResNet are rarely explored beyond the lazy training regime. In this work, we investigate scaled ResNet in the limit of infinitely deep and wide neural networks, of wh ...

2024

Mapping orthorhombic domains with geometrical phase analysis in rare-earth nickelate heterostructures

Duncan Thomas Lindsay Alexander, Bernat Mundet, Jean-Marc Triscone

Most perovskite oxides belong to the Pbnm space group, composed of an anisotropic unit cell, A-site antipolar displacements, and oxygen octahedral tilts. Mapping the orientation of the orthorhombic unit cell in epitaxial heterostructures that consist of at ...

Aip Publishing2024

Seeding and Emergence of Composite Skyrmions in a van der Waals Magnet

Klaus Kern, Marko Burghard, Lukas Powalla

Topological charge plays a significant role in a range of physical systems. In particular, observations of real-space topological objects in magnetic materials have been largely limited to skyrmions - states with a unitary topological charge. Recently, mor ...

Wiley-V C H Verlag Gmbh2023

Decentralized learning made easy with DecentralizePy

Anne-Marie Kermarrec, Rafael Pereira Pires, Akash Balasaheb Dhasade, Rishi Sharma, Milos Vujasinovic

Decentralized learning (DL) has gained prominence for its potential benefits in terms of scalability, privacy, and fault tolerance. It consists of many nodes that coordinate without a central server and exchange millions of parameters in the inherently ite ...

2023

Discrete Voss surfaces: Designing geodesic gridshells with planar cladding panels

Corentin Jean Dominique Fivet, Nicolas Robin Montagne, Olivier Baverel

The design and construction of doubly-curved structures often reveals to be challenging and can result in complex manufacturing and assembly. A recent strategy to tackle this difficulty consists in exploiting the connection between discrete differential ge ...

2022

Preconditioners for robust optimal control problems under uncertainty

Fabio Nobile, Tommaso Vanzan

The discretization of robust quadratic optimal control problems under uncertainty using the finite element method and the stochastic collocation method leads to large saddle-point systems, which are fully coupled across the random realizations. Despite its ...

2023

On The Convergence Of Stochastic Primal-Dual Hybrid Gradient

Volkan Cevher, Ahmet Alacaoglu

In this paper, we analyze the recently proposed stochastic primal-dual hybrid gradient (SPDHG) algorithm and provide new theoretical results. In particular, we prove almost sure convergence of the iterates to a solution with convexity and linear convergenc ...

SIAM PUBLICATIONS2022

Decentralized Proximal Gradient Algorithms With Linear Convergence Rates

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

This article studies a class of nonsmooth decentralized multiagent optimization problems where the agents aim at minimizing a sum of local strongly-convex smooth components plus a common nonsmooth term. We propose a general primal-dual algorithmic framewor ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Variance-Reduced Stochastic Learning Under Random Reshuffling

Ali H. Sayed, Bicheng Ying, Kun Yuan

Several useful variance-reduced stochastic gradient algorithms, such as SVRG, SAGA, Finito, and SAG, have been proposed to minimize empirical risks with linear convergence properties to the exact minimizer. The existing convergence results assume uniform d ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features

Ali H. Sayed, Sulaiman A S A E Alghunaim, Ming Yan

This work studies multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) function coupling all agents. This scenario arises in many machine learning and engineering ...

IEEE2020

Dictionary Learning for Two-Dimensional Kendall Shapes

Michaël Unser, Julien René Pierre Fageot, Virginie Sophie Uhlmann, Anna You-Lai Song

We propose a novel sparse dictionary learning method for planar shapes in the sense of Kendall, namely configurations of landmarks in the plane considered up to similitudes. Our shape dictionary method provides a good trade-off between algorithmic simplici ...

SIAM PUBLICATIONS2020

Quenched convergence and strong local equilibrium for asymmetric zero-range process with site disorder

Thomas Mountford

We study asymmetric zero-range processes on Z with nearest-neighbour jumps and site disorder. The jump rate of particles is an arbitrary but bounded nondecreasing function of the number of particles. We prove quenched strong local equilibrium at subcritica ...

SPRINGER HEIDELBERG2020

A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!

Martin Jaggi, Anastasiia Koloskova

Decentralized optimization methods enable on-device training of machine learning models without a central coordinator. In many scenarios communication between devices is energy demanding and time consuming and forms the bottleneck of the entire system. We ...

MICROTOME PUBLISHING2020

On the convergence of stochastic primal-dual hybrid gradient

Volkan Cevher, Ahmet Alacaoglu

In this paper, we analyze the recently proposed stochastic primal-dual hybrid gradient (SPDHG) algorithm and provide new theoretical results. In particular, we prove almost sure convergence of the iterates to a solution and linear convergence with standard ...

2019

A Linearly Convergent Proximal Gradient Algorithm for Decentralized Optimization

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

Decentralized optimization is a powerful paradigm that finds applications in engineering and learning design. This work studies decentralized composite optimization problems with non-smooth regularization terms. Most existing gradient-based proximal decent ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2019

Convergence Of Variance-Reduced Learning Under Random Reshuffling

Ali H. Sayed, Bicheng Ying, Kun Yuan

Several useful variance-reduced stochastic gradient algorithms, such as SVRG, SAGA, Finito, and SAG, have been proposed to minimize empirical risks with linear convergence properties to the exact minimizers. The existing convergence results assume uniform ...

IEEE2018

On Matching Pursuit and Coordinate Descent

Martin Jaggi, Sebastian Urban Stich, Sai Praneeth Reddy Karimireddy, Anant Raj

Two popular examples of first-order optimization methods over linear spaces are coordinate descent and matching pursuit algorithms, with their randomized variants. While the former targets the optimization by moving along coordinates, the latter considers ...

2018

On the linear convergence of the stochastic gradient method with constant step-size

Volkan Cevher, Cong Bang Vu

The strong growth condition (SGC) is known to be a sufficient condition for linear convergence of the stochastic gradient method using a constant step-size γ (SGM-CS). In this paper, we provide a necessary condition, for the linear convergence of SGM-CS, t ...

2018

Non-convex, non-local functionals converging to the total variation

Hoài-Minh Nguyên

We present new results concerning the approximation of the total variation, integral(Omega)vertical bar del u vertical bar, of a function u by non-local, non-convex functionals of the form Lambda delta(u) = integral(Omega)integral(Omega)delta phi(vertical ...

Elsevier2017

Hypomonotonicity of the Normal Cone and Proximal Smoothness

Grigory Ivanov

We study the properties of the normal cone to a proximally smooth set. We give a complete characterization of a proximally smooth set through the monotonicity properties of its normal cone in an arbitrary uniformly convex and uniformly smooth Banach space. ...

Heldermann Verlag2017