Publications by Sébastien Louis Alexandre Rouault

Practical Byzantine-resilient Stochastic Gradient Descent

Algorithms are everywhere. The recipe for the frangipane cake is an algorithm. If all the listed ingredients are available and the cook is sufficiently deft, after a finite number of small, well-defined steps a delicious dessert will exit the oven. Now, wh ...

EPFL2022

Genuinely distributed Byzantine machine learning

Rachid Guerraoui, El Mahdi El Mhamdi, Le Nguyen Hoang, Sébastien Louis Alexandre Rouault, Arsany Hany Abdelmessih Guirguis

Machine learning (ML) solutions are nowadays distributed, according to the so-called server/worker architecture. One server holds the model parameters while several workers train the model. Clearly, such architecture is prone to various types of component ...

2022

Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

Byzantine-resilient Stochastic Gradient Descent (SGD) aims at shielding model training from Byzantine faults, be they ill-labeled training datapoints, exploited software/hardware vulnerabilities, or malicious worker nodes in a distributed setting. Two rece ...

2021

Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)

Rachid Guerraoui, El Mahdi El Mhamdi, Le Nguyen Hoang, Sébastien Louis Alexandre Rouault, Arsany Hany Abdelmessih Guirguis, Sadegh Farhadkhani

We study Byzantine collaborative learning, where n nodes seek to collectively learn from each others' local data. The data distribution may vary from one node to another. No node is trusted, and f < n nodes can behave arbitrarily. We prove that collaborati ...

2021

AKSEL: Fast Byzantine SGD

Rachid Guerraoui, El Mahdi El Mhamdi, Alexandre David Olivier Maurer, Sébastien Louis Alexandre Rouault

Modern machine learning architectures distinguish servers and workers. Typically, a d-dimensional model is hosted by a server and trained by n workers, using a distributed stochastic gradient descent (SGD) optimization scheme. At each SGD step, the goal is ...

Schloss Dagstuhl--Leibniz-Zentrum für Informatik2021

Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?

Rachid Guerraoui, Sébastien Louis Alexandre Rouault, John Stephan, Nirupam Gupta, Rafaël Benjamin Pinot

This paper addresses the problem of combining Byzantine resilience with privacy in machine learning (ML). Specifically, we study if a distributed implementation of the renowned Stochastic Gradient Descent (SGD) learning algorithm is feasible with both diff ...

ASSOC COMPUTING MACHINERY2021

GARFIELD: System Support for Byzantine Machine Learning (Regular Paper)

Rachid Guerraoui, Sébastien Louis Alexandre Rouault, Arsany Hany Abdelmessih Guirguis

We present GARFIELD, a library to transparently make machine learning (ML) applications, initially built with popular (but fragile) frameworks, e.g., TensorFlow and PyTorch, Byzantine–resilient. GARFIELD relies on a novel object–oriented design, reducing t ...

IEEE2021

Genuinely Distributed Byzantine Machine Learning

Rachid Guerraoui, El Mahdi El Mhamdi, Le Nguyen Hoang, Sébastien Louis Alexandre Rouault, Arsany Hany Abdelmessih Guirguis

Machine Learning (ML) solutions are nowadays distributed, according to the so-called server/worker architecture. One server holds the model parameters while several workers train the model. Clearly, such architecture is prone to various types of component ...

Association for Computing Machinery2020

Fast and Robust Distributed Learning in High Dimension

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

Could a gradient aggregation rule (GAR) for distributed machine learning be both robust and fast? This paper answers by the affirmative through Multi-Bulyan. Given n workers, f of which are arbitrary malicious (Byzantine) and m = n − f are not, we prove th ...

2020

Byzantine machine learning

Rachid Guerraoui, El Mahdi El Mhamdi, Mahsa Taziki, Sébastien Louis Alexandre Rouault

The present invention concerns computer-implemented methods for training a machine learning model using Stochastic Gradient Descent, SGD. In one embodiment, the method is performed by a first computer in a distributed computing environment and comprises pe ...

2020

AGGREGATHOR: Byzantine Machine Learning via Robust Gradient Aggregation

Rachid Guerraoui, El Mahdi El Mhamdi, Georgios Damaskinos, Sébastien Louis Alexandre Rouault, Arsany Hany Abdelmessih Guirguis

We present AGGREGATHOR, a framework that implements state-of-the-art robust (Byzantine-resilient) distributed stochastic gradient descent. Following the standard parameter server model, we assume that a minority of worker machines can be controlled by an a ...

2019

The Hidden Vulnerability of Distributed Learning in Byzantium

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

While machine learning is going through an era of celebrated success, concerns have been raised about the vulnerability of its backbone: stochastic gradient descent (SGD). Recent approaches have been proposed to ensure the robustness of distributed SGD aga ...

2018

On The Robustness of a Neural Network

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

With the development of neural networks based machine learning and their usage in mission critical applications, voices are rising against the \textit{black box} aspect of neural networks as it becomes crucial to understand their limits and capabilities. W ...

2017