Publications related to Early stopping

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization

This paper studies kernel ridge regression in high dimensions under covariate shifts and analyzes the role of importance re-weighting. We first derive the asymptotic expansion of high dimensional kernels under covariate shifts. By a bias-variance decomposi ...

2024

Gradient boosting with extreme-value theory for wildfire prediction

Jonathan Koh Boon Han

This paper details the approach of the team Kohrrelation in the 2021 Extreme Value Analysis data challenge, dealing with the prediction of wildfire counts and sizes over the contiguous US. Our approach uses ideas from extreme-value theory in a machine lear ...

2023

Data-Driven Control and Optimization under Noisy and Uncertain Conditions

Baiwei Guo

Control systems operating in real-world environments often face disturbances arising from measurement noise and model mismatch. These factors can significantly impact the perfor- mance and safety of the system. In this thesis, we aim to leverage data to de ...

EPFL2023

Federated Learning under Covariate Shifts with Generalization Guarantees

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Thomas Michaelsen Pethick

This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance. To handle covariate shifts, we formulate a new global model training paradigm and propose Federated Impor ...

2023

Automatic Parameterization for Aerodynamic Shape Optimization via Deep Geometric Learning

Pascal Fua, Zhen Wei

We propose two deep learning models that fully automate shape parameterization for aerodynamic shape optimization. Both models are optimized to parameterize via deep geometric learning to embed human prior knowledge into learned geometric patterns, elimina ...

2023

Classification of fall directions via wearable motion sensors

Mustafa Sahin Turan

Effective fall-detection and classification systems are vital in mitigating severe medical and economical consequences of falls to people in the fall risk groups. One class of such systems is based on wearable sensors. While there is a vast amount of acade ...

ACADEMIC PRESS INC ELSEVIER SCIENCE2022

The very knotty lenser: Exploring the role of regularization in source and potential reconstructions using Gaussian process regression

Georgios Vernardos

Reconstructing lens potentials and lensed sources can easily become an underconstrained problem, even when the degrees of freedom are low, due to degeneracies, particularly when potential perturbations superimposed on a smooth lens are included. Regulariza ...

OXFORD UNIV PRESS2022

System Support for Robust Distributed Learning

Arsany Hany Abdelmessih Guirguis

Machine learning (ML) applications are ubiquitous. They run in different environments such as datacenters, the cloud, and even on edge devices. Despite where they run, distributing ML training seems the only way to attain scalable, high-quality learning. B ...

EPFL2022

ADAGRAD Avoids Saddle Points

Kimon Antonakopoulos, Xiao Wang

Adaptive first-order methods in optimization are prominent in machine learning and data science owing to their ability to automatically adapt to the landscape of the function being optimized. However, their convergence guarantees are typically stated in te ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

Semantic Perturbations with Normalizing Flows for Improved Generalization

Martin Jaggi, Tatjana Chavdarova, Sebastian Urban Stich

Data augmentation is a widely adopted technique for avoiding overfitting when training deep neural networks. However, this approach requires domain-specific knowledge and is often limited to a fixed set of hard-coded transformations. Recently, several work ...

IEEE2021

KPC: Learning-Based Model Predictive Control with Deterministic Guarantees

Colin Neil Jones, Yuning Jiang, Paul Scharnhorst, Emilio Maddalena

We propose Kernel Predictive Control (KPC), a learning-based predictive control strategy that enjoys deterministic guarantees of safety. Noise-corrupted samples of the unknown system dynamics are used to learn several models through the formalism of non-pa ...

2021

Deterministic error bounds for kernel-based learning techniques under bounded noise

Colin Neil Jones, Paul Scharnhorst, Emilio Maddalena

We consider the problem of reconstructing a function from a finite set of noise-corrupted samples. Two kernel algorithms are analyzed, namely kernel ridge regression and epsilon-support vector regression. By assuming the ground-truth function belongs to th ...

2021

Disparity Between Batches as a Signal for Early Stopping

Patrick Thiran, Mahsa Forouzesh

We propose a metric for evaluating the generalization ability of deep neural networks trained with mini-batch gradient descent. Our metric, called gradient disparity, is the l2 norm distance between the gradient vectors of two mini-batches drawn from the t ...

Springer2021

Geodesic Distance on Optimally Regularized Functional Connectomes Uncovers Individual Fingerprints

Enrico Amico

Background: Functional connectomes (FCs) have been shown to provide a reproducible individual fingerprint, which has opened the possibility of personalized medicine for neuro/psychiatric disorders. Thus, developing accurate ways to compare FCs is essential ...

MARY ANN LIEBERT, INC2021

Accelerating Gradient Boosting Machines

Sai Praneeth Reddy Karimireddy

Gradient Boosting Machine (GBM) introduced by Friedman (2001) is a widely popular ensembling technique and is routinely used in competitions such as Kaggle and the KDD-Cup (Chen and Guestrin, 2016). In this work, we propose an Accelerated Gradient Boosting ...

ADDISON-WESLEY PUBL CO2020

AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks

Sabine Süsstrunk, Majed El Helou, Frederike Dümbgen

The large capacity of neural networks enables them to learn complex functions. To avoid overfitting, networks however require a lot of training data that can be expensive and time-consuming to collect. A common practical approach to attenuate overfitting i ...

IEEE2020

Noninvasive estimation of aortic hemodynamics and cardiac contractility using machine learning

Nikolaos Stergiopoulos, Georgios Rovas, Vasiliki Bikia, Stamatia Zoi Pagoulatou

Cardiac and aortic characteristics are crucial for cardiovascular disease detection. However, noninvasive estimation of aortic hemodynamics and cardiac contractility is still challenging. This paper investigated the potential of estimating aortic systolic ...

Nature Research2020

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

A Linearly Convergent Proximal Gradient Algorithm for Decentralized Optimization

Ali H. Sayed, Kun Yuan, Sulaiman A S A E Alghunaim

Decentralized optimization is a powerful paradigm that finds applications in engineering and learning design. This work studies decentralized composite optimization problems with non-smooth regularization terms. Most existing gradient-based proximal decent ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2019

AdaptHD: Adaptive Efficient Training for Brain-Inspired Hyperdimensional Computing

Giovanni De Micheli, Samuel Bosch, Mohsen Imani

Brain-inspired Hyperdimensional (HD) computing is a promising solution for energy-efficient classification. However, the existing HD computing algorithms have a lack of controllability on the training iterations which often results in slow training or dive ...

2019