Publications associées à On Sparsity Averaging

Efficient local linearity regularization to overcome catastrophic overfitting

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Elias Abad Rocamora

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

2024

Towards Trustworthy Deep Learning for Image Reconstruction

Alexis Marie Frederic Goujon

The remarkable ability of deep learning (DL) models to approximate high-dimensional functions from samples has sparked a revolution across numerous scientific and industrial domains that cannot be overemphasized. In sensitive applications, the good perform ...

EPFL2024

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization

Volkan Cevher, Fanghui Liu

This paper studies kernel ridge regression in high dimensions under covariate shifts and analyzes the role of importance re-weighting. We first derive the asymptotic expansion of high dimensional kernels under covariate shifts. By a bias-variance decomposi ...

2024

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

Pakshal Narendra Bohra

In inverse problems, the task is to reconstruct an unknown signal from its possibly noise-corrupted measurements. Penalized-likelihood-based estimation and Bayesian estimation are two powerful statistical paradigms for the resolution of such problems. They ...

EPFL2024

Quantum radio astronomy: Data encodings and quantum image processing

Jean-Paul Richard Kneib, Emma Elizabeth Tolley, Stefano Corda

We explore applications of quantum computing for radio interferometry and astronomy using recent developments in quantum image processing. We evaluate the suitability of different quantum image representations using a toy quantum computing image reconstruc ...

Elsevier2024

(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability

Scott William Pesme, Nicolas Henri Bernard Flammarion, Mathieu Even

In this paper, we investigate the impact of stochasticity and large stepsizes on the implicit regularisation of gradient descent (GD) and stochastic gradient descent (SGD) over diagonal linear networks. We prove the convergence of GD and SGD with macroscop ...

2023

Regularization of polynomial networks for image recognition

Volkan Cevher, Bohan Wang, Grigorios Chrysos

Deep Neural Networks (DNNs) have obtained impressive performance across tasks, however they still remain as black boxes, e.g., hard to theoretically analyze. At the same time, Polynomial Networks (PNs) have emerged as an alternative method with a promising ...

2023

Regularization of polynomial networks for image recognition

Volkan Cevher, Bohan Wang, Grigorios Chrysos

Deep Neural Networks (DNNs) have obtained impressive performance across tasks, however they still remain as black boxes, e.g., hard to theoretically analyze. At the same time, Polynomial Networks (PNs) have emerged as an alternative method with a promising ...

Ieee Computer Soc2023

Regularization Techniques for Low-Resource Machine Translation

Alejandro Ramírez Atrio

Neural machine translation (MT) and text generation have recently reached very high levels of quality. However, both areas share a problem: in order to reach these levels, they require massive amounts of data. When this is not present, they lack generaliza ...

EPFL2023

A Neural-Network-Based Convex Regularizer for Inverse Problems

Michaël Unser, Stanislas Ducotterd, Pakshal Narendra Bohra, Alexis Marie Frederic Goujon, Sebastian Jonas Neumayer

The emergence of deep-learning-based methods to solve image-reconstruction problems has enabled a significant increase in quality. Unfortunately, these new methods often lack reliability and explainability, and there is a growing interest to address these ...

2023

Stability of Image-Reconstruction Algorithms

Michaël Unser, Pol del Aguila Pla, Sebastian Jonas Neumayer

Robustness and stability of image-reconstruction algorithms have recently come under scrutiny. Their importance to medical imaging cannot be overstated. We review the known results for the topical variational regularization strategies ( ℓ2 and ℓ1 regulariz ...

2023

Bayes-optimal Learning of Deep Random Networks of Extensive-width

Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width ...

2023

Saddle-to-Saddle Dynamics in Diagonal Linear Networks

Scott William Pesme, Nicolas Henri Bernard Flammarion

In this paper we fully describe the trajectory of gradient flow over diagonal linear networks in the limit of vanishing initialisation. We show that the limiting flow successively jumps from a saddle of the training loss to another until reaching the minim ...

2023

Controlled physics-informed data generation for deep learning-based remaining useful life prediction under unseen operation conditions

Jian Zhou, Olga Fink

Limited availability of representative time-to-failure (TTF) trajectories either limits the performance of deep learning (DL)-based approaches on remaining useful life (RUL) prediction in practice or even precludes their application. Generating synthetic d ...

2023

Maximum likelihood bolometry for ASDEX upgrade experiments

Olivier Claude Martin Février, Riccardo Rossi

Bolometry is an essential diagnostic for calculating the power balances and for the understanding of different physical aspects of tokamak experiments. The reconstruction method based on the Maximum Likelihood (ML) principle, developed initially for JET, h ...

Iop Publishing Ltd2023

LenslessPiCam: A Hardware and Software Platform for Lensless Computational Imaging with a Raspberry Pi

Martin Vetterli, Sepand Kashani, Matthieu Martin Jean-André Simeoni, Eric Bezzam

This toolkit has everything you need to perform imaging with a lensless camera. We make use of a low-cost implementation of DiffuserCam, namely replacing the lens with a piece of tape and using the Raspberry Pi HQ camera sensor. Similar principles and meth ...

Zenodo2023

Prediction of post-stroke motor recovery benefits from measures of sub-acute widespread network damages

Dimitri Nestor Alice Van De Ville, Elvira Pirondini, Maria Giulia Preti, Cyprien Alban Félicien Rivier

Rivier et al. report that global measures of brain connectivity estimated by introducing virtual lesions in healthy structural connectomes significantly increase the accuracy of motor recovery prediction in stroke patients when added to focal and clinical ...

OXFORD UNIV PRESS2023

Variational Methods For Continuous-Domain Inverse Problems: the Quest for the Sparsest Solution

Thomas Jean Debarre

The goal of this thesis is to study continuous-domain inverse problems for the reconstruction of sparse signals and to develop efficient algorithms to solve such problems computationally. The task is to recover a signal of interest as a continuous function ...

EPFL2022

Regularized Diffusion Adaptation via Conjugate Smoothing

Ali H. Sayed, Stefan Vlaski

The purpose of this article is to develop and study a decentralized strategy for Pareto optimization of an aggregate cost consisting of regularized risks. Each risk is modeled as the expectation of some loss function with unknown probability distribution, ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

System and method for transcoding spectral data with task-based optimization via symmetric non-negative matrix factorization

Forrest Simon Webler

A computer-implemented method for reconstructing/recovering high-resolution visible light spectral data at a target resolution d, that comprises obtaining a configuration of a low- resolution multi-channel imaging sensor of resolution p, the configuration ...

2022