Publications associées à High-dimensional statistics

Spectral Estimators for High-Dimensional Matrix Inference

A key challenge across many disciplines is to extract meaningful information from data which is often obscured by noise. These datasets are typically represented as large matrices. Given the current trend of ever-increasing data volumes, with datasets grow ...

EPFL2024

A Combination Technique for Optimal Control Problems Constrained by Random PDEs

Fabio Nobile, Tommaso Vanzan

We present a combination technique based on mixed differences of both spatial approximations and quadrature formulae for the stochastic variables to solve efficiently a class of optimal control problems (OCPs) constrained by random partial differential equ ...

2024

Random matrix methods for high-dimensional machine learning models

Antoine Philippe Michel Bodin

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

EPFL2024

Stable parameterization of continuous and piecewise-linear functions

Michaël Unser, Alexis Marie Frederic Goujon, Joaquim Gonçalves Garcia Barreto Campos

Rectified-linear-unit (ReLU) neural networks, which play a prominent role in deep learning, generate continuous and piecewise-linear (CPWL) functions. While they provide a powerful parametric representation, the mapping between the parameter and function s ...

2023

On the symmetries in the dynamics of wide two-layer neural networks

Lénaïc Chizat

We consider the idealized setting of gradient flow on the population risk for infinitely wide two-layer ReLU neural networks (without bias), and study the effect of symmetries on the learned parameters and predictors. We first describe a general class of s ...

AMER INST MATHEMATICAL SCIENCES-AIMS2023

(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability

Nicolas Henri Bernard Flammarion, Scott William Pesme, Mathieu Even

In this paper, we investigate the impact of stochasticity and large stepsizes on the implicit regularisation of gradient descent (GD) and stochastic gradient descent (SGD) over diagonal linear networks. We prove the convergence of GD and SGD with macroscop ...

2023

Deep Learning Meets Sparse Regularization

Rahul Parhi

Deep learning (DL) has been wildly successful in practice, and most of the state-of-the-art machine learning methods are based on neural networks (NNs). Lacking, however, is a rigorous mathematical theory that adequately explains the amazing performance of ...

2023

Covariance Estimation for Random Surfaces beyond Separability

Tomas Masák

This thesis focuses on non-parametric covariance estimation for random surfaces, i.e.~functional data on a two-dimensional domain. Non-parametric covariance estimation lies at the heart of functional data analysis, and considerations of statistical and com ...

EPFL2022

Sparsest piecewise-linear regression of one-dimensional data

Michaël Unser, Julien René Pierre Fageot, Thomas Jean Debarre, Quentin Alain Denoyelle

We study the problem of one-dimensional regression of data points with total-variation (TV) regularization (in the sense of measures) on the second derivative, which is known to promote piecewise-linear solutions with few knots. While there are efficient a ...

Elsevier2021

Fast screening of covariates in population models empowered by machine learning

Jan Sickmann Hesthaven, Nadia Terranova

One of the objectives of Pharmacometry (PMX) population modeling is the identification of significant and clinically relevant relationships between parameters and covariates. Here, we demonstrate how this complex selection task could benefit from supervise ...

2021

Real-Time Multi-Ion-Monitoring Front-End With Interference Compensation by Multi-Output Support Vector Regressor

Giovanni De Micheli, Sandro Carrara, Mandresy Ivan Ny Hanitra, Francesca Criscuolo

Ion-sensors play a major role in physiology and healthcare monitoring since they are capable of continuously collecting biological data from body fluids. Nevertheless, ion interference from background electrolytes present in the sample is a paramount chall ...

2021

Improving sample and feature selection with principal covariates regression

Michele Ceriotti, Edgar Albert Engel, Benjamin Aaron Helfrecht, Sergei Kliavinek

Selecting the most relevant features and samples out of a large set of candidates is a task that occurs very often in the context of automated data analysis, where it improves the computational performance and often the transferability of a model. Here we ...

2021

Sparsely Observed Functional Time Series: Theory and Applications

Tomas Rubin

Functional time series is a temporally ordered sequence of not necessarily independent random curves. While the statistical analysis of such data has been traditionally carried out under the assumption of completely observed functional data, it may well ha ...

EPFL2021

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Florent Gérard Krzakala, Lenka Zdeborová

We consider a commonly studied supervised classification of a synthetic dataset whose labels are generated by feeding a one-layer non-linear neural network with random iid inputs. We study the generalization performances of standard classifiers in the high ...

Curran Associates, Inc.2020

Generalized approximate survey propagation for high-dimensional estimation

Yue Lu, Luca Saglietti

In generalized linear estimation (GLE) problems, we seek to estimate a signal that is observed through a linear transform followed by a component-wise, possibly nonlinear and noisy, channel. In the Bayesian optimal setting, generalized approximate message ...

IOP PUBLISHING LTD2020

Simultaneous autoregressive models for spatial extremes

Emeric Rolland Georges Thibaud

Motivated by the widespread use of large gridded data sets in the atmospheric sciences, we propose a new model for extremes of areal data that is inspired by the simultaneous autoregressive (SAR) model in classical spatial statistics. Our extreme SAR model ...

WILEY2020

A Sensor Fusion Approach to the Estimation of Instantaneous Velocity Using Single Wearable Sensor During Sprint

Kamiar Aminian, Frédéric Meyer, Salil Apte, Farzin Dadashi, Vincent Gremeaux

Power-Force-Velocity profile obtained during a sprint test is crucial for designing personalized training and evaluating injury risks. Estimation of instantaneous velocity is requisite for developing these profiles and the predominant method for this estim ...

2020

Adaptive hot water production based on Supervised Learning

Alexandre Massoud Alahi, Dolaana Khovalyg, Amirreza Heidari

A major challenge in the common approach of hot water generation in residential houses lies in the highly stochastic nature of domestic hot water (DHW) demand. Learning hot water use behavior enables water heating systems to continuously adapt to the stoch ...

2021

Structure-property maps with Kernel principal covariates regression

Michele Ceriotti, Guillaume André Jean Fraux, Benjamin Aaron Helfrecht

Data analyses based on linear methods constitute the simplest, most robust, and transparent approaches to the automatic processing of large amounts of data for building supervised or unsupervised machine learning models. Principal covariates regression (PC ...

2020

An Introductory Guide to Fano's Inequality with Applications in Statistical Estimation

Volkan Cevher, Jonathan Mark Scarlett

Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning. While numerous information-theoretic ...

Cambridge University Press2021