Publications related to Stability (learning theory)

Fixing the problems of deep neural networks will require better training data and learning algorithms

Martin Schrimpf, Adrien Christophe Doerig, Matthias Bethge, Jianghao Liu, Kuntal Ghosh

Bowers et al. argue that deep neural networks (DNNs) are poor models of biological vision because they often learn to rival human accuracy by relying on strategies that differ markedly from those of humans. We show that this problem is worsening as DNNs ar ...

Cambridge Univ Press2023

Through-Plane Super-Resolution With Autoencoders in Diffusion Magnetic Resonance Imaging of the Developing Human Brain

Meritxell Bach Cuadra, Gabriel Girard, Erick Jorge Canales Rodriguez, Hamza Kebiri

Fetal brain diffusion magnetic resonance images (MRI) are often acquired with a lower through-plane than in-plane resolution. This anisotropy is often overcome by classical upsampling methods such as linear or cubic interpolation. In this work, we employ a ...

FRONTIERS MEDIA SA2022

An Equivalence Between Data Poisoning and Byzantine Gradient Attacks

Rachid Guerraoui, Sadegh Farhadkhani, Le Nguyen Hoang, Oscar Jean Olivier Villemaud

To study the resilience of distributed learning, the “Byzantine" literature considers a strong threat model where workers can report arbitrary gradients to the parameter server. Whereas this model helped obtain several fundamental results, it has sometimes ...

PMLR2022

Practical Byzantine-resilient Stochastic Gradient Descent

Sébastien Louis Alexandre Rouault

Algorithms are everywhere. The recipe for the frangipane cake is an algorithm. If all the listed ingredients are available and the cook is sufficiently deft, after a finite number of small, well-defined steps a delicious dessert will exit the oven. Now, wh ...

EPFL2022

Finite Littlestone Dimension Implies Finite Information Complexity

Michael Christoph Gastpar, Aditya Pradeep, Ido Nachum

We prove that every online learnable class of functions of Littlestone dimension d admits a learning algorithm with finite information complexity. Towards this end, we use the notion of a globally stable algorithm. Generally, the information complexity of ...

2022

Role of stochastic noise and generalization error in the time propagation of neural-network quantum states

Giuseppe Carleo

Neural-network quantum states (NQS) have been shown to be a suitable variational ansatz to simulate out-of-equilibrium dynamics in two-dimensional systems using timedependent variational Monte Carlo (t-VMC). In particular, stable and accurate time propagat ...

SCIPOST FOUNDATION2022

Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?

Rachid Guerraoui, John Stephan, Nirupam Gupta, Sébastien Louis Alexandre Rouault, Rafaël Benjamin Pinot

This paper addresses the problem of combining Byzantine resilience with privacy in machine learning (ML). Specifically, we study if a distributed implementation of the renowned Stochastic Gradient Descent (SGD) learning algorithm is feasible with both diff ...

ASSOC COMPUTING MACHINERY2021

Social NCE: Contrastive Learning of Socially-aware Motion Representations

Alexandre Massoud Alahi, Yuejiang Liu

Learning socially-aware motion representations is at the core of recent advances in multi-agent problems, such as human motion forecasting and robot navigation in crowds. Despite promising progress, existing representations learned with neural networks sti ...

2021

Optimal Adversarial Policies in the Multiplicative Learning System With a Malicious Expert

Negar Kiyavash, Seyed Jalal Etesami

We consider a learning system based on the conventional multiplicative weight ( MW) rule that combines experts' advice to predict a sequence of true outcomes. It is assumed that one of the experts is malicious and aims to impose the maximum loss on the sys ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Predictive user-based relocation through incentives in one-way car-sharing systems

Nikolaos Geroliminis, Patrick Stefan Adriaan Stokkink

Car-sharing systems are an attractive alternative to private vehicles due to their benefits in terms of mobility and sustainability. However, the distribution of vehicles throughout the network in one-way systems is disturbed due to asymmetry and stochasti ...

2021

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. ...

2021

Adaptation and Learning Over Networks Under Subspace Constraints & x2014;Part II: Performance Analysis

Ali H. Sayed, Stefan Vlaski, Roula Nassif

Part & x00A0;I of this paper considered optimization problems over networks where agents have individual objectives to meet, or individual parameter vectors to estimate, subject to subspace constraints that require the objectives across the network to lie ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

Idiap and UAM Participation at MEX-A3T Evaluation Campaign

Petr Motlicek

This paper describes our participation in the shared evaluation campaign of MexA3T 2020. Our main goal wasto evaluate a Supervised Autoencoder (SAE) learning algorithm in text classification tasks. For our experiments,we used three different sets of featur ...

CEUR Workshop Proceedings2020

Strengthened Information-theoretic Bounds on the Generalization Error

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The following problem is considered: given a joint distribution P XY and an event E, bound P XY (E) in terms of P X P Y (E) (where P X P Y is the product of the marginals of P XY ) and a measure of dependence of X and Y. Such bounds have direct application ...

IEEE2019

A Generalized Representer Theorem for Hilbert Space - Valued Functions

Colin Neil Jones, Sanket Sanjay Diwale

The necessary and sufficient conditions for existence of a generalized representer theorem are presented for learning Hilbert space - valued functions. Representer theorems involving explicit basis functions and Reproducing Kernels are a common occurrence ...

2019

Crowdsourcing with Fairness, Diversity and Budget Constraints

Boi Faltings, Naman Goel

Recent studies have shown that the labels collected from crowdworkers can be discriminatory with respect to sensitive attributes such as gender and race. This raises questions about the suitability of using crowdsourced data for further use, such as for tr ...

ASSOC COMPUTING MACHINERY2019

Consistent Tomography Under Partial Observations Over Adaptive Networks

Ali H. Sayed

This paper studies the problem of inferring whether an agent is directly influenced by another agent over a network. Agent i influences agent j if they are connected (according to the network topology), and if agent j uses the data from agent i to update i ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Learning and Adaptive Data Analysis via Maximal Leakage

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

There has been growing interest in studying connections between generalization error of learning algorithms and information measures. In this work, we generalize a result that employs the maximal leakage, a measure of leakage of information, and explore ho ...

IEEE2019

Chaining Mutual Information and Tightening Generalization Bounds

Emmanuel Abbé

Bounding the generalization error of learning algorithms has a long history, which yet falls short in explaining various generalization successes including those of deep learning. Two important difficulties are (i) exploiting the dependencies between the h ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018