Publications de El Mahdi El Mhamdi

On the Strategyproofness of the Geometric Median

Rachid Guerraoui, El Mahdi El Mhamdi, Le Nguyen Hoang, Sadegh Farhadkhani

The geometric median, an instrumental component of the secure machine learning toolbox, is known to be effective when robustly aggregating models (or gradients), gathered from potentially malicious (or strategic) users. What is less known is the extent to ...

2023

Genuinely distributed Byzantine machine learning

Rachid Guerraoui, El Mahdi El Mhamdi, Le Nguyen Hoang, Sébastien Louis Alexandre Rouault, Arsany Hany Abdelmessih Guirguis

Machine learning (ML) solutions are nowadays distributed, according to the so-called server/worker architecture. One server holds the model parameters while several workers train the model. Clearly, such architecture is prone to various types of component ...

2022

Removing Algorithmic Discrimination (With Minimal Individual Error)

Rachid Guerraoui, El Mahdi El Mhamdi, Le Nguyen Hoang, Alexandre David Olivier Maurer

We address for the first time the problem of correcting group discriminations within a score function, while minimizing the individual error. Each group is described by a probability density function on the set of profiles. We first solve the problem analy ...

2022

Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

Byzantine-resilient Stochastic Gradient Descent (SGD) aims at shielding model training from Byzantine faults, be they ill-labeled training datapoints, exploited software/hardware vulnerabilities, or malicious worker nodes in a distributed setting. Two rece ...

2021

Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)

Rachid Guerraoui, El Mahdi El Mhamdi, Le Nguyen Hoang, Sébastien Louis Alexandre Rouault, Arsany Hany Abdelmessih Guirguis, Sadegh Farhadkhani

We study Byzantine collaborative learning, where n nodes seek to collectively learn from each others' local data. The data distribution may vary from one node to another. No node is trusted, and f < n nodes can behave arbitrarily. We prove that collaborati ...

2021

AKSEL: Fast Byzantine SGD

Rachid Guerraoui, El Mahdi El Mhamdi, Alexandre David Olivier Maurer, Sébastien Louis Alexandre Rouault

Modern machine learning architectures distinguish servers and workers. Typically, a d-dimensional model is hosted by a server and trained by n workers, using a distributed stochastic gradient descent (SGD) optimization scheme. At each SGD step, the goal is ...

Schloss Dagstuhl--Leibniz-Zentrum für Informatik2021

Genuinely Distributed Byzantine Machine Learning

Rachid Guerraoui, El Mahdi El Mhamdi, Le Nguyen Hoang, Sébastien Louis Alexandre Rouault, Arsany Hany Abdelmessih Guirguis

Machine Learning (ML) solutions are nowadays distributed, according to the so-called server/worker architecture. One server holds the model parameters while several workers train the model. Clearly, such architecture is prone to various types of component ...

Association for Computing Machinery2020

Robust Distributed Learning

El Mahdi El Mhamdi

Whether it occurs in artificial or biological substrates, {\it learning} is a {distributed} phenomenon in at least two aspects. First, meaningful data and experiences are rarely found in one location, hence {\it learners} have a strong incentive to work t ...

EPFL2020

Fast and Robust Distributed Learning in High Dimension

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

Could a gradient aggregation rule (GAR) for distributed machine learning be both robust and fast? This paper answers by the affirmative through Multi-Bulyan. Given n workers, f of which are arbitrary malicious (Byzantine) and m = n − f are not, we prove th ...

2020

Byzantine machine learning

Rachid Guerraoui, El Mahdi El Mhamdi, Mahsa Taziki, Sébastien Louis Alexandre Rouault

The present invention concerns computer-implemented methods for training a machine learning model using Stochastic Gradient Descent, SGD. In one embodiment, the method is performed by a first computer in a distributed computing environment and comprises pe ...

2020

Byzantine tolerant gradient descent for distributed machine learning with adversaries

Rachid Guerraoui, El Mahdi El Mhamdi, Peva François Blanchard, Julien Michel Stainer

The present application concerns a computer-implemented method for training a machine learning model in a distributed fashion, using Stochastic Gradient Descent, SGD, wherein the method is performed by a first computer in a distributed computing environmen ...

2019

Exploring The Borderlands Of The Gathering Problem

Rachid Guerraoui, El Mahdi El Mhamdi, Alexandre David Olivier Maurer, Vladislav Tempez

Problems of pattern formation have been extensively studied in distributed computing. One of this problems is the gathering problem: agents must gather at a same position in a distributed manner. When gathering is not possible, a close problem is the conve ...

2019

AGGREGATHOR: Byzantine Machine Learning via Robust Gradient Aggregation

Rachid Guerraoui, El Mahdi El Mhamdi, Georgios Damaskinos, Sébastien Louis Alexandre Rouault, Arsany Hany Abdelmessih Guirguis

We present AGGREGATHOR, a framework that implements state-of-the-art robust (Byzantine-resilient) distributed stochastic gradient descent. Following the standard parameter server model, we assume that a minority of worker machines can be controlled by an a ...

2019

Asynchronous Byzantine Machine Learning (the case of SGD)

Rachid Guerraoui, El Mahdi El Mhamdi, Rhicheek Patra, Mahsa Taziki, Georgios Damaskinos

Asynchronous distributed machine learning solutions have proven very effective so far, but always assuming perfectly functioning workers. In practice, some of the workers can however exhibit Byzantine behavior, caused by hardware failures, software bugs, c ...

2018

The Hidden Vulnerability of Distributed Learning in Byzantium

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

While machine learning is going through an era of celebrated success, concerns have been raised about the vulnerability of its backbone: stochastic gradient descent (SGD). Recent approaches have been proposed to ensure the robustness of distributed SGD aga ...

2018

Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning

Rachid Guerraoui, El Mahdi El Mhamdi, Alexandre David Olivier Maurer, Hadrien Hendrikx

In reinforcement learning, agents learn by performing actions and observing their outcomes. Sometimes, it is desirable for a human operator to \textit{interrupt} an agent in order to prevent dangerous situations from happening. Yet, as part of their learni ...

EPFL2017

On The Robustness of a Neural Network

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

With the development of neural networks based machine learning and their usage in mission critical applications, voices are rising against the \textit{black box} aspect of neural networks as it becomes crucial to understand their limits and capabilities. W ...

2017

Brief Announcement: Byzantine-Tolerant Machine Learning

Rachid Guerraoui, El Mahdi El Mhamdi, Peva François Blanchard, Julien Michel Stainer

We report on \emph{Krum}, the first \emph{provably} Byzantine-tolerant aggregation rule for distributed Stochastic Gradient Descent (SGD). Krum guarantees the convergence of SGD even in a distributed setting where (asymptotically) up to half of the workers ...

ACM2017

When Neurons Fail - Technical Report

Rachid Guerraoui, El Mahdi El Mhamdi

Neural networks have been traditionally considered robust in the sense that their precision degrades gracefully with the failure of neurons and can be compensated by additional learning phases. Nevertheless, critical applications for which neural networks ...

EPFL2016

Is light-induced degradation of a-Si:H/c-Si interfaces reversible?

Christophe Ballif, Bénédicte Demaurex, Stefaan De Wolf, El Mahdi El Mhamdi, Jakub Holovsky

Thin hydrogenated amorphous silicon (a-Si:H) films deposited on crystalline silicon (c-Si) surfaces are sensitive probes for the bulk electronic properties of a-Si:H. Here, we use such samples during repeated low-temperature annealing and visible-light soa ...

AIP American Institute of Physics2014