Publications associées à Generalization error

Generalization and Personalization of Machine Learning for Multimodal Mobile Sensing in Everyday Life

A range of behavioral and contextual factors, including eating and drinking behavior, mood, social context, and other daily activities, can significantly impact an individual's quality of life and overall well-being. Therefore, inferring everyday life aspe ...

EPFL2024

Dynamical simulation via quantum machine learning with provable generalization

Zoë Holmes

Much attention has been paid to dynamical simulation and quantum machine learning (QML) independently as applications for quantum advantage, while the possibility of using QML to enhance dynamical simulations has not been thoroughly investigated. Here we d ...

Amer Physical Soc2024

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization

Volkan Cevher, Fanghui Liu

This paper studies kernel ridge regression in high dimensions under covariate shifts and analyzes the role of importance re-weighting. We first derive the asymptotic expansion of high dimensional kernels under covariate shifts. By a bias-variance decomposi ...

2024

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Microtome Publishing2024

Data-Driven Control and Optimization under Noisy and Uncertain Conditions

Baiwei Guo

Control systems operating in real-world environments often face disturbances arising from measurement noise and model mismatch. These factors can significantly impact the perfor- mance and safety of the system. In this thesis, we aim to leverage data to de ...

EPFL2023

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Federated Learning under Covariate Shifts with Generalization Guarantees

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Thomas Michaelsen Pethick

This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance. To handle covariate shifts, we formulate a new global model training paradigm and propose Federated Impor ...

2023

Benign Overfitting in Deep Neural Networks under Lazy Training

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayesoptimal test error for classification while obtaining (nearly) zero-trai ...

2023

A Study on Gradient-based Meta-learning for Robust Deep Digital Twins

Olga Fink, Raffael Pascal Theiler, Michele Viscione

Deep-learning-based digital twins (DDT) are a promising tool for data-driven system health management because they can be trained directly on operational data. A major challenge for efficient training however is that industrial datasets remain unlabeled. T ...

Research Publishing2023

Error scaling laws for kernel classification under source and capacity conditions

Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Hugo Chao Cui, Bruno Loureiro

In this manuscript we consider the problem of kernel classification. While worst-case bounds on the decay rate of the prediction error with the number of samples are known for some classifiers, they often fail to accurately describe the learning curves of ...

2023

A Theory of Finite-Width Neural Networks: Generalization, Scaling Laws, and the Loss Landscape

Berfin Simsek

Deep learning has achieved remarkable success in various challenging tasks such as generating images from natural language or engaging in lengthy conversations with humans. The success in practice stems from the ability to successfully train massive neural ...

EPFL2023

Role of stochastic noise and generalization error in the time propagation of neural-network quantum states

Giuseppe Carleo

Neural-network quantum states (NQS) have been shown to be a suitable variational ansatz to simulate out-of-equilibrium dynamics in two-dimensional systems using timedependent variational Monte Carlo (t-VMC). In particular, stable and accurate time propagat ...

SCIPOST FOUNDATION2022

The very knotty lenser: Exploring the role of regularization in source and potential reconstructions using Gaussian process regression

Georgios Vernardos

Reconstructing lens potentials and lensed sources can easily become an underconstrained problem, even when the degrees of freedom are low, due to degeneracies, particularly when potential perturbations superimposed on a smooth lens are included. Regulariza ...

OXFORD UNIV PRESS2022

OptTTA: Learnable Test-Time Augmentation for Source-Free Medical Image Segmentation Under Domain Shift

Jean-Philippe Thiran, Guillaume Marc Georges Vray, Devavrat Tomar, Seyedbehzad Bozorgtabar

As distribution shifts are inescapable in realistic clinical scenarios due to inconsistencies in imaging protocols, scanner vendors, and across different centers, well-trained deep models incur a domain generalization problem in unseen environments. Despit ...

PMLR2022

Comparison of non-parametric T2 relaxometry methods for myelin water quantification

Jean-Philippe Thiran, Tobias Kober, Tom Hilbert, Erick Jorge Canales Rodriguez, Gian Franco Piredda, Marco Pizzolato, Thomas Yu, Alessandro Daducci, Nicolas Kunz

Multi-component T2 relaxometry allows probing tissue microstructure by assessing compartment-specific T2 relaxation times and water fractions, including the myelin water fraction. Non-negative least squares (NNLS) with zero-order Tikhonov regularization is ...

2021

Disparity Between Batches as a Signal for Early Stopping

Patrick Thiran, Mahsa Forouzesh

We propose a metric for evaluating the generalization ability of deep neural networks trained with mini-batch gradient descent. Our metric, called gradient disparity, is the l2 norm distance between the gradient vectors of two mini-batches drawn from the t ...

Springer2021

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. ...

2021

Injecting Knowledge in Data-driven Vehicle Trajectory Predictors

Alexandre Massoud Alahi, Ismail Nejjar, Mohammadhossein Bahari

Vehicle trajectory prediction tasks have been commonly tackled from two distinct perspectives: either with knowledge-driven methods or more recently with data-driven ones. On the one hand, we can explicitly implement domain-knowledge or physical priors suc ...

2021

Semantic Perturbations with Normalizing Flows for Improved Generalization

Martin Jaggi, Tatjana Chavdarova, Sebastian Urban Stich

Data augmentation is a widely adopted technique for avoiding overfitting when training deep neural networks. However, this approach requires domain-specific knowledge and is often limited to a fixed set of hard-coded transformations. Recently, several work ...

IEEE2021

KPC: Learning-Based Model Predictive Control with Deterministic Guarantees

Colin Neil Jones, Yuning Jiang, Paul Scharnhorst, Emilio Maddalena

We propose Kernel Predictive Control (KPC), a learning-based predictive control strategy that enjoys deterministic guarantees of safety. Noise-corrupted samples of the unknown system dynamics are used to learn several models through the formalism of non-pa ...

2021