Publications associées à Dilemme biais-variance

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization

This paper studies kernel ridge regression in high dimensions under covariate shifts and analyzes the role of importance re-weighting. We first derive the asymptotic expansion of high dimensional kernels under covariate shifts. By a bias-variance decomposi ...

2024

Meta-learning to address diverse Earth observation problems across resolutions

Devis Tuia, Benjamin Alexander Kellenberger, Marc Conrad Russwurm

Earth scientists study a variety of problems with remote sensing data, but they most often consider them in isolation from each other, which limits information flows across disciplines. In this work, we present METEOR, a meta-learning methodology for Earth ...

Springernature2024

Improving Generalization of Pretrained Language Models

Rabeeh Karimi Mahabadi

In this dissertation, we propose multiple methods to improve transfer learning for pretrained language models (PLMs). Broadly, transfer learning is a powerful technique in natural language processing, where a language model is first pre-trained on a data-r ...

EPFL2023

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Data-Driven Control and Optimization under Noisy and Uncertain Conditions

Baiwei Guo

Control systems operating in real-world environments often face disturbances arising from measurement noise and model mismatch. These factors can significantly impact the perfor- mance and safety of the system. In this thesis, we aim to leverage data to de ...

EPFL2023

Validation of semi-analytical, semi-empirical covariance matrices for two-point correlation function for early DESI data

Cheng Zhao

We present an extended validation of semi-analytical, semi-empirical covariance matrices for the two-point correlation function (2PCF) on simulated catalogs representative of luminous red galaxies (LRGs) data collected during the initial 2 months of operat ...

OXFORD UNIV PRESS2023

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Tensor approximation of the self-diffusion matrix of tagged particle processes

Christoph Max Strössner

The objective of this paper is to investigate a new numerical method for the approximation of the self-diffusion matrix of a tagged particle process defined on a grid. While standard numerical methods make use of long-time averages of empirical means of de ...

ACADEMIC PRESS INC ELSEVIER SCIENCE2023

Stochastic distributed learning with gradient quantization and double-variance reduction

Sebastian Urban Stich, Konstantin Mishchenko

We consider distributed optimization over several devices, each sending incremental model updates to a central server. This setting is considered, for instance, in federated learning. Various schemes have been designed to compress the model updates in orde ...

TAYLOR & FRANCIS LTD2022

The very knotty lenser: Exploring the role of regularization in source and potential reconstructions using Gaussian process regression

Georgios Vernardos

Reconstructing lens potentials and lensed sources can easily become an underconstrained problem, even when the degrees of freedom are low, due to degeneracies, particularly when potential perturbations superimposed on a smooth lens are included. Regulariza ...

OXFORD UNIV PRESS2022

Perturbation theory models for LSST-era galaxy clustering: Tests with subpercent mock catalog measurements in Fourier and configuration space

Jonathan Andrew Blazek

We analyze the clustering of galaxies using the z = 1.006 snapshot of the CosmoDC2 simulation, a high-fidelity synthetic galaxy catalog designed to validate analysis methods for the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST). We prese ...

American Physical Society2022

Leveraging topology, geometry, and symmetries for efficient Machine Learning

Michaël Defferrard

When learning from data, leveraging the symmetries of the domain the data lies on is a principled way to combat the curse of dimensionality: it constrains the set of functions to learn from. It is more data efficient than augmentation and gives a generaliz ...

EPFL2022

Unbiased Monte Carlo cluster updates with autoregressive neural networks

Giuseppe Carleo, Riccardo Rossi, Dian Wu

Efficient sampling of complex high-dimensional probability distributions is a central task in computational science. Machine learning methods like autoregressive neural networks, used with Markov chain Monte Carlo sampling, provide good approximations to s ...

AMER PHYSICAL SOC2021

Practical issues with modeling extreme Brazilian rainfall

Anthony Christopher Davison, Isolde Santos Previdelli, Paulo Vitor Da Costa Pereira

Accurately quantifying extreme rainfall is important for the design of hydraulic structures, for flood mapping and zoning and for disaster management. In order to produce maps of estimates of 25-year rainfall return levels in Brazil, we selected 893 shorte ...

BRAZILIAN STATISTICAL ASSOCIATION2021

Continual Learning for Natural Language Generation in Task-oriented Dialog Systems

Boi Faltings, Mengjie Zhao, Fei Mi, Liangwei Chen

Natural language generation (NLG) is an essential component of task-oriented dialog systems. Despite the recent success of neural approaches for NLG, they are typically developed in an offline manner for particular domains. To better fit real-life applicat ...

2021

Euclid : Effects of sample covariance on the number counts of galaxy clusters

Georges Meylan, Yi Wang, Richard Massey

Aims. We investigate the contribution of shot-noise and sample variance to uncertainties in the cosmological parameter constraints inferred from cluster number counts, in the context of the Euclid survey. ...

EDP SCIENCES S A2021

Semantic Perturbations with Normalizing Flows for Improved Generalization

Martin Jaggi, Tatjana Chavdarova, Sebastian Urban Stich

Data augmentation is a widely adopted technique for avoiding overfitting when training deep neural networks. However, this approach requires domain-specific knowledge and is often limited to a fixed set of hard-coded transformations. Recently, several work ...

IEEE2021

Towards a pragmatist dealing with algorithmic bias in medical machine learning

Machine Learning (ML) is on the rise in medicine, promising improved diagnostic, therapeutic and prognostic clinical tools. While these technological innovations are bound to transform health care, they also bring new ethical concerns to the forefront. One ...

2021

Forward-reflected-backward method with variance reduction

Volkan Cevher, Ahmet Alacaoglu

We propose a variance reduced algorithm for solving monotone variational inequalities. Without assuming strong monotonicity, cocoercivity, or boundedness of the domain, we prove almost sure convergence of the iterates generated by the algorithm to a soluti ...

2021

Multilevel ensemble Kalman filtering for spatio-temporal processes

Fabio Nobile, Hakon Andreas Hoel

We design and analyse the performance of a multilevel ensemble Kalman filter method (MLEnKF) for filtering settings where the underlying state-space model is an infinite-dimensional spatio-temporal process. We consider underlying models that needs to be si ...

Springer2021