Publications related to Sparse approximation

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

In inverse problems, the task is to reconstruct an unknown signal from its possibly noise-corrupted measurements. Penalized-likelihood-based estimation and Bayesian estimation are two powerful statistical paradigms for the resolution of such problems. They ...

EPFL2024

Multi-Frequency Tracking via Group-Sparse Optimal Transport

Isabel Haasler

In this letter, we introduce an optimal transport framework for inferring power distributions over both spatial location and temporal frequency. Recently, it has been shown that optimal transport is a powerful tool for estimating spatial spectra that chang ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov, Scott William Pesme

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

The inductive bias of deep learning: Connecting weights and functions

Guillermo Ortiz Jimenez

Years of a fierce competition have naturally selected the fittest deep learning algorithms. Yet, although these models work well in practice, we still lack a proper characterization of why they do so. This poses serious questions about the robustness, trus ...

EPFL2023

(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability

Nicolas Henri Bernard Flammarion, Scott William Pesme, Mathieu Even

In this paper, we investigate the impact of stochasticity and large stepsizes on the implicit regularisation of gradient descent (GD) and stochastic gradient descent (SGD) over diagonal linear networks. We prove the convergence of GD and SGD with macroscop ...

2023

Saddle-to-Saddle Dynamics in Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Scott William Pesme

In this paper we fully describe the trajectory of gradient flow over diagonal linear networks in the limit of vanishing initialisation. We show that the limiting flow successively jumps from a saddle of the training loss to another until reaching the minim ...

2023

Variational Methods For Continuous-Domain Inverse Problems: the Quest for the Sparsest Solution

Thomas Jean Debarre

The goal of this thesis is to study continuous-domain inverse problems for the reconstruction of sparse signals and to develop efficient algorithms to solve such problems computationally. The task is to recover a signal of interest as a continuous function ...

EPFL2022

A Structured Dictionary Perspective on Implicit Neural Representations

Pascal Frossard, Guillermo Ortiz Jimenez, Beril Besbinar, Gizem Yüce

Implicit neural representations (INRs) have recently emerged as a promising alternative to classical discretized representations of signals. Nevertheless, despite their practical success, we still do not understand how INRs represent signals. We propose a ...

IEEE COMPUTER SOC2022

From Undercomplete to Sparse Overcomplete Autoencoders to Improve LF-MMI Speech Recognition

Hervé Bourlard, Selen Hande Kabil

Starting from a strong Lattice-Free Maximum Mutual Information (LF-MMI) baseline system, we explore different autoencoder configurations to enhance Mel-Frequency Cepstral Coefficients (MFCC) features. Autoencoders are expected to generate new MFCC features ...

ISCA-INT SPEECH COMMUNICATION ASSOC2022

One- and two-step semi-Lagrangian integrators for arbitrary Lagrangian-Eulerian-finite element two-phase flow simulations

John Richard Thome, Gustavo Rabello Dos Anjos, Gustavo Charles Peixoto de Oliveira

Semi-Lagrangian (SL) schemes are of utmost relevance to simulate two-phase flows where advection dominates. The combination of SL schemes with the finite element (FE) method and arbitrary Lagrangian-Eulerian (ALE) dynamic meshes yields a strong ingredient ...

WILEY2022

Adaptive Path Interpolation Method for Sparse Systems: Application to a Censored Block Model

Nicolas Macris, Jean François Emmanuel Barbier, Chun Lam Chan

Recently, a new adaptive path interpolation method has been developed as a simple and versatile scheme to calculate exactly the asymptotic mutual information of Bayesian inference problems defined on dense factor graphs. These include random linear and gen ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Sparsest piecewise-linear regression of one-dimensional data

Michaël Unser, Julien René Pierre Fageot, Thomas Jean Debarre, Quentin Alain Denoyelle

We study the problem of one-dimensional regression of data points with total-variation (TV) regularization (in the sense of measures) on the second derivative, which is known to promote piecewise-linear solutions with few knots. While there are efficient a ...

Elsevier2021

Multiple Line Outage Detection in Power Systems by Sparse Recovery Using Transient Data

Ping Hu, Feng Liu, Li Ding

Fast and accurate transmission line outage detection can help the central control unit to respond rapidly to better maintain the security and reliability of power systems. It is especially critical in the situation of multiple line outages which is more li ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Weak-lensing mass reconstruction using sparsity and a Gaussian random field

Austin Chandler Peel

Aims. We introduce a novel approach to reconstructing dark matter mass maps from weak gravitational lensing measurements. The cornerstone of the proposed method lies in a new modelling of the matter density field in the Universe as a mixture of two compone ...

2021

Multikernel Regression with Sparsity Constraint

Michaël Unser, Shayan Aziznejad

In this paper, we provide a Banach-space formulation of supervised learning with generalized total-variation (gTV) regularization. We identify the class of kernel functions that are admissible in this framework. Then, we propose a variation of supervised l ...

2021

Convex optimization in sums of Banach spaces

Michaël Unser, Shayan Aziznejad

We characterize the solution of a broad class of convex optimization problems that address the reconstruction of a function from a finite number of linear measurements. The underlying hypothesis is that the solution is decomposable as a finite sum of compo ...

2021

Efficient algorithms for wave problems

Boris Bonev

Wave phenomena manifest in nature as electromagnetic waves, acoustic waves, and gravitational waves among others. Their descriptions as partial differential equations in electromagnetics, acoustics, and fluid dynamics are ubiquitous in science and engineer ...

EPFL2021

Sound Field Reconstruction in a room through Sparse Recovery and its application in Room Modal Equalization

Vu Thach Pham

The acoustics of a room often refer to the quality of the room in terms of sound recording as well as sound reproduction. At low frequencies, where the wavelengths are of the same order of magnitude with the dimensions of the room, due to the standing wave ...

EPFL2021

Regularity and sparse approximation of the recursive first moment equations for the lognormal Darcy problem

Fabio Nobile, Francesca Bonizzoni

We study the Darcy boundary value problem with lognormal permeability field. We adopt a perturbation approach, expanding the solution in Taylor series around the nominal value of the coefficient, and approximating the expected value of the stochastic solut ...

2020

Dynamic Model Pruning with Feedback

Martin Jaggi, Luis Felipe Barba Flores, Sebastian Urban Stich, Tao Lin, Daniil Dmitriev

Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased latency at inference. We propose a novel model compression method that gener ...

2020