Publications related to Model-based Sketching and Recovery with Expanders

Statistical Inference for Inverse Problems: From Sparsity-Based Methods to Neural Networks

In inverse problems, the task is to reconstruct an unknown signal from its possibly noise-corrupted measurements. Penalized-likelihood-based estimation and Bayesian estimation are two powerful statistical paradigms for the resolution of such problems. They ...

EPFL2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Scott William Pesme, Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability

Scott William Pesme, Nicolas Henri Bernard Flammarion, Mathieu Even

In this paper, we investigate the impact of stochasticity and large stepsizes on the implicit regularisation of gradient descent (GD) and stochastic gradient descent (SGD) over diagonal linear networks. We prove the convergence of GD and SGD with macroscop ...

2023

Principles of Network Plasticity in Neocortical Microcircuits

András Ecker

Synaptic plasticity underlies our ability to learn and adapt to the constantly changing environment. The phenomenon of synapses changing their efficacy in an activity-dependent manner is often studied in small groups of neurons in vitro or indirectly throu ...

EPFL2023

Augmented Lagrangian Methods for Provable and Scalable Machine Learning

Mehmet Fatih Sahin

Non-convex constrained optimization problems have become a powerful framework for modeling a wide range of machine learning problems, with applications in k-means clustering, large- scale semidefinite programs (SDPs), and various other tasks. As the perfor ...

EPFL2023

Variational Methods For Continuous-Domain Inverse Problems: the Quest for the Sparsest Solution

Thomas Jean Debarre

The goal of this thesis is to study continuous-domain inverse problems for the reconstruction of sparse signals and to develop efficient algorithms to solve such problems computationally. The task is to recover a signal of interest as a continuous function ...

EPFL2022

Approximating Relu Networks By Single-Spike Computation

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Angeliki Pantazi, Ana Stanojevic, Evangelos Eleftheriou, Giovanni Cherubini

Developing energy-saving neural network models is a topic of rapidly increasing interest in the artificial intelligence community. Spiking neural networks (SNNs) are biologically inspired models that strive to leverage the energy efficiency stemming from a ...

IEEE2022

Space-Efficient Representations of Graphs

Jakab Tardos

With the increasing prevalence of massive datasets, it becomes important to design algorithmic techniques for dealing with scenarios where the input to be processed does not fit in the memory of a single machine. Many highly successful approaches have emer ...

EPFL2022

From Undercomplete to Sparse Overcomplete Autoencoders to Improve LF-MMI Speech Recognition

Hervé Bourlard, Selen Hande Kabil

Starting from a strong Lattice-Free Maximum Mutual Information (LF-MMI) baseline system, we explore different autoencoder configurations to enhance Mel-Frequency Cepstral Coefficients (MFCC) features. Autoencoders are expected to generate new MFCC features ...

ISCA-INT SPEECH COMMUNICATION ASSOC2022

Multiple Line Outage Detection in Power Systems by Sparse Recovery Using Transient Data

Li Ding, Feng Liu, Ping Hu

Fast and accurate transmission line outage detection can help the central control unit to respond rapidly to better maintain the security and reliability of power systems. It is especially critical in the situation of multiple line outages which is more li ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Sparsest piecewise-linear regression of one-dimensional data

Michaël Unser, Julien René Pierre Fageot, Thomas Jean Debarre, Quentin Alain Denoyelle

We study the problem of one-dimensional regression of data points with total-variation (TV) regularization (in the sense of measures) on the second derivative, which is known to promote piecewise-linear solutions with few knots. While there are efficient a ...

Elsevier2021

Sublinear Algorithms for Spectral Graph Clustering

Aidasadat Mousavifar

This thesis focuses on designing spectral tools for graph clustering in sublinear time. With the emergence of big data, many traditional polynomial time, and even linear time algorithms have become prohibitively expensive. Processing modern datasets requir ...

EPFL2021

Adaptive Path Interpolation Method for Sparse Systems: Application to a Censored Block Model

Nicolas Macris, Chun Lam Chan, Jean François Emmanuel Barbier

Recently, a new adaptive path interpolation method has been developed as a simple and versatile scheme to calculate exactly the asymptotic mutual information of Bayesian inference problems defined on dense factor graphs. These include random linear and gen ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Sparsely Observed Functional Time Series: Theory and Applications

Tomas Rubin

Functional time series is a temporally ordered sequence of not necessarily independent random curves. While the statistical analysis of such data has been traditionally carried out under the assumption of completely observed functional data, it may well ha ...

EPFL2021

Multikernel Regression with Sparsity Constraint

Michaël Unser, Shayan Aziznejad

In this paper, we provide a Banach-space formulation of supervised learning with generalized total-variation (gTV) regularization. We identify the class of kernel functions that are admissible in this framework. Then, we propose a variation of supervised l ...

2021

Convex optimization in sums of Banach spaces

Michaël Unser, Shayan Aziznejad

We characterize the solution of a broad class of convex optimization problems that address the reconstruction of a function from a finite number of linear measurements. The underlying hypothesis is that the solution is decomposable as a finite sum of compo ...

2021

Faster Monotone Min-Plus Product, Range Mode, and Single Source Replacement Paths

Adam Teodor Polak

One of the most basic graph problems, All-Pairs Shortest Paths (APSP) is known to be solvable in n^{3-o(1)} time, and it is widely open whether it has an O(n^{3-ε}) time algorithm for ε > 0. To better understand APSP, one often strives to obtain subcubic t ...

Schloss Dagstuhl -- Leibniz-Zentrum für Informatik2021

Clustering and control for adaptation uncovers time-warped spike time patterns in cortical networks in vivo

James Bryden Isbister

How information in the nervous system is encoded by patterns of action potentials (i.e. spikes) remains an open question. Multi-neuron patterns of single spikes are a prime candidate for spike time encoding but their temporal variability requires further c ...

Nature Research2021

Online Distributed Learning Over Graphs With Multitask Graph-Filter Models

Ali H. Sayed, Roula Nassif

In this article, we are interested in adaptive and distributed estimation of graph filters from streaming data. We formulate this problem as a consensus estimation problem over graphs, which can be addressed with diffusion LMS strategies. Most popular grap ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Dynamic Model Pruning with Feedback

Tao Lin, Martin Jaggi, Sebastian Urban Stich, Daniil Dmitriev, Luis Felipe Barba Flores

Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased latency at inference. We propose a novel model compression method that gener ...

2020