Publications associées à Méthode du gradient conjugué

SPEEDING UP KRYLOV SUBSPACE METHODS FOR COMPUTING f(A)b VIA RANDOMIZATION

This work is concerned with the computation of the action of a matrix function f(A), such as the matrix exponential or the matrix square root, on a vector b. For a general matrix A, this can be done by computing the compression of A onto a suitable Krylov ...

Siam Publications2024

A Combination Technique for Optimal Control Problems Constrained by Random PDEs

Fabio Nobile, Tommaso Vanzan

We present a combination technique based on mixed differences of both spatial approximations and quadrature formulae for the stochastic variables to solve efficiently a class of optimal control problems (OCPs) constrained by random partial differential equ ...

2024

Randomized flexible GMRES with deflated restarting

Laura Grigori, Emeric Martin

For a high-dimensional problem, a randomized Gram-Schmidt (RGS) algorithm is beneficial in terms of both computational cost and numerical stability. We apply this dimension reduction technique by random sketching to Krylov subspace methods, e.g., to the ge ...

Springer2024

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Fast Adversarial Training With Adaptive Step Size

Mathieu Salzmann, Sabine Süsstrunk, Chen Liu, Zhuoyi Huang, Yong Zhang, Jue Wang

While adversarial training and its variants have shown to be the most effective algorithms to defend against adversarial attacks, their extremely slow training process makes it hard to scale to large datasets like ImageNet. The key idea of recent works to ...

Ieee-Inst Electrical Electronics Engineers Inc2023

Efficient and Accurate Handling of Periodic Flows in Time-Sensitive Networks

Jean-Yves Le Boudec, Seyed Mohammadhossein Tabatabaee

Total Flow Analysis (TFA) is a method for the worst-case analysis of time-sensitive networks. It uses service curve characterizations of the network nodes and arrival curves of flows at their sources; for tractability, the latter are often taken to be line ...

2023

Stable Nonconvex-Nonconcave Training via Linear Interpolation

Volkan Cevher, Thomas Michaelsen Pethick, Wanyun Xie

This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss lan ...

2023

An optimal preconditioned FFT-accelerated finite element solver for homogenization

Till Junge, Ali Falsafi, Martin Ladecký

We generalize and provide a linear algebra-based perspective on a finite element (FE) ho-mogenization scheme, pioneered by Schneider et al. (2017)[1] and Leuschner and Fritzen (2018)[2]. The efficiency of the scheme is based on a preconditioned, well-scale ...

ELSEVIER SCIENCE INC2023

Hamiltonian Deep Neural Networks Guaranteeing Non-Vanishing Gradients by Design

Giancarlo Ferrari Trecate, Luca Furieri, Clara Lucía Galimberti, Liang Xu

Deep Neural Networks (DNNs) training can be difficult due to vanishing and exploding gradients during weight optimization through backpropagation. To address this problem, we propose a general class of Hamiltonian DNNs (H-DNNs) that stem from the discretiz ...

2023

Implicit Distance Functions: Learning and Applications in Robotics

Mikhail Koptev

In this thesis, we address the complex issue of collision avoidance in the joint space of robots. Avoiding collisions with both the robot's own body parts and obstacles in the environment is a critical constraint in motion planning and is crucial for ensur ...

EPFL2023

Regularization Techniques for Low-Resource Machine Translation

Alejandro Ramírez Atrio

Neural machine translation (MT) and text generation have recently reached very high levels of quality. However, both areas share a problem: in order to reach these levels, they require massive amounts of data. When this is not present, they lack generaliza ...

EPFL2023

Training Efficient Controllers via Analytic Policy Gradient

Dario Floreano, Valentin Wüest, Davide Scaramuzza

Control design for robotic systems is complex and often requires solving an optimization to follow a trajectory accurately. Online optimization approaches like Model Predictive Control (MPC) have been shown to achieve great tracking performance, but requir ...

New York2023

Computational design and additive manufacturing of electromagnetic coils

Adrien Jean-Michel Thabuis

The modern world is heavily reliant on electromagnetic devices to convert mechanical energy into electrical energy and vice versa. These devices are fundamental to powering our society, and the growing need for automated production lines and electrified tr ...

EPFL2023

Predicting nonlinear optical scattering with physics-driven neural networks

Demetri Psaltis, Carlo Gigli, Ahmed Ayoub

Deep neural networks trained on physical losses are emerging as promising surrogates for nonlinear numerical solvers. These tools can predict solutions to Maxwell's equations and compute gradients of output fields with respect to the material and geometric ...

AIP Publishing2023

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Practical considerations of diffusion-weighted MRS with ultra-strong diffusion gradients

André Döring

IntroductionDiffusion-weighted magnetic resonance spectroscopy (DW-MRS) offers improved cellular specificity to microstructure-compared to water-based methods alone-but spatial resolution and SNR is severely reduced and slow-diffusing metabolites necessita ...

Frontiers Media Sa2023

Rank-Adaptive Time Integration Of Tree Tensor Networks

Gianluca Ceruti

A rank-adaptive integrator for the approximate solution of high-order tensor differential equations by tree tensor networks is proposed and analyzed. In a recursion from the leaves to the root, the integrator updates bases and then evolves connection tenso ...

SIAM PUBLICATIONS2023

Gait Phase Estimation in Steady Walking: A Comparative Study of Methods Based on the Phase Portrait of the Hip Angle

Auke Ijspeert, Mohamed Bouri, Ali Reza Manzoori, Coline Lugaz, Tian Ye, Davide Malatesta

Accurate real-time estimation of the gait phase (GP) is crucial for many control methods in exoskeletons and prostheses. A class of approaches to GP estimation construct the phase portrait of a segment or joint angle, and use the normalized polar angle of ...

IEEE Xplore2023

Preconditioners for robust optimal control problems under uncertainty

Fabio Nobile, Tommaso Vanzan

The discretization of robust quadratic optimal control problems under uncertainty using the finite element method and the stochastic collocation method leads to large saddle-point systems, which are fully coupled across the random realizations. Despite its ...

2023

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Amirkeivan Mohtashami, Sebastian Urban Stich

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022