Publications by Nirupam Gupta | EPFL Graph Search

Robust Machine Learning - Distributed Methods for Safe AI

Rachid Guerraoui, Nirupam Gupta, Rafaël Benjamin Pinot

Today, machine learning algorithms are often distributed across multiple machines to leverage more computing power and more data. However, the use of a distributed framework entails a variety of security threats. In particular, some of the machines may mis ...

Springer2024

Robust and Private Federated Learning on LLMs

Rachid Guerraoui, Nirupam Gupta

Large Language Models (LLMs) have gained significant attention in recent years due to their potential to revolutionize various industries and sectors. However, scaling LLMs further requires access to substantial linguistic resources that are being rapidly ...

Springer Nature2024

Revisiting Ensembling in One-Shot Federated Learning

Rachid Guerraoui, Anne-Marie Kermarrec, Akash Balasaheb Dhasade, Rafael Pereira Pires, Nirupam Gupta, Rafaël Benjamin Pinot, Rishi Sharma, Youssef Allouah

Federated learning (FL) is an appealing approach to training machine learning models without sharing raw data. However, standard FL algorithms are iterative and thus induce a significant communication cost. One-shot federated learning (OFL) trades the iter ...

Curran Associates, Inc.2024

Brief Announcement: A Case for Byzantine Machine Learning

Rachid Guerraoui, Sadegh Farhadkhani, Nirupam Gupta, Rafaël Benjamin Pinot

The success of machine learning (ML) has been intimately linked with the availability of large amounts of data, typically collected from heterogeneous sources and processed on vast networks of computing devices (also called workers). Beyond accuracy, the u ...

ACM2024

Fine-Tuning Personalization in Federated Learning to Mitigate Adversarial Clients

Rachid Guerraoui, Nirupam Gupta, Rafaël Benjamin Pinot, Youssef Allouah

Federated learning (FL) is an appealing paradigm that allows a group of machines (a.k.a. clients) to learn collectively while keeping their data local. However, due to the heterogeneity between the clients' data distributions, the model obtained through th ...

Neural information processing systems foundation2024

Impact of Redundancy on Resilience in Distributed Optimization and Learning

Shuo Liu, Nirupam Gupta

This paper considers the problem of resilient distributed optimization and stochastic learning in a server-based architecture. The system comprises a server and multiple agents, where each agent has its own local cost function. The agents collaborate with ...

Assoc Computing Machinery2023

Byzantine Machine Learning: A Primer

Rachid Guerraoui, Nirupam Gupta, Rafaël Benjamin Pinot

The problem of Byzantine resilience in distributed machine learning, a.k.a., Byzantine machine learning, consists in designing distributed algorithms that can train an accurate model despite the presence of Byzantine nodes, i.e., nodes with corrupt data or ...

2023

Byzantine Fault-Tolerance in Federated Local SGD Under 2f-Redundancy

Nirupam Gupta

In this article, we study the problem of Byzantine fault-tolerance in a federated optimization setting, where there is a group of agents communicating with a centralized coordinator. We allow up to

f

Byzantine-faulty agents, which may not follow a prescr ...

Ieee-Inst Electrical Electronics Engineers Inc2023

On the Privacy-Robustness-Utility Trilemma in Distributed Learning

Rachid Guerraoui, John Stephan, Nirupam Gupta, Rafaël Benjamin Pinot, Youssef Allouah

The ubiquity of distributed machine learning (ML) in sensitive public domain applications calls for algorithms that protect data privacy, while being robust to faults and adversarial behaviors. Although privacy and robustness have been extensively studied ...

2023

Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneity

Rachid Guerraoui, Nirupam Gupta, Rafaël Benjamin Pinot, Youssef Allouah, Geovani Rizk

The theory underlying robust distributed learning algorithms, designed to resist adversarial machines, matches empirical observations when data is homogeneous. Under data heterogeneity however, which is the norm in practical scenarios, established lower bo ...

2023

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, Le Nguyen Hoang, John Stephan, Sadegh Farhadkhani, Nirupam Gupta, Rafaël Benjamin Pinot

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

PLMR2023

Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity

Rachid Guerraoui, John Stephan, Sadegh Farhadkhani, Nirupam Gupta, Rafaël Benjamin Pinot, Youssef Allouah

Byzantine machine learning (ML) aims to ensure the resilience of distributed learning algorithms to misbehaving (or Byzantine) machines. Although this problem received significant attention, prior works often assume the data held by the machines to be homo ...

PMLR2023

On Preconditioning of Decentralized Gradient-Descent When Solving a System of Linear Equations

Nirupam Gupta

This article considers solving an overdetermined system of linear equations in peer-to-peer multiagent networks. The network is assumed to be synchronous and strongly connected. Each agent has a set of local data points, and their goal is to compute a line ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums

Rachid Guerraoui, John Stephan, Sadegh Farhadkhani, Nirupam Gupta, Rafaël Benjamin Pinot

Byzantine resilience emerged as a prominent topic within the distributed machine learning community. Essentially, the goal is to enhance distributed optimization algorithms, such as distributed SGD, in a way that guarantees convergence despite the presence ...

PMLR2022

Democratizing Machine Learning

Alexandre David Olivier Maurer, Nirupam Gupta, Rafaël Benjamin Pinot

The increasing prevalence of personal devices motivates the design of algorithms that can leverage their computing power, together with the data they generate, in order to build privacy-preserving and effective machine learning models. However, traditional ...

IEEE2022

Iterative pre-conditioning for expediting the distributed gradient-descent method: The case of linear least-squares problem

Nirupam Gupta

This paper considers the multi-agent linear least-squares problem in a server-agent network architecture. The system comprises multiple agents, each with a set of local data points. The agents are connected to a server, and there is no inter-agent communic ...

PERGAMON-ELSEVIER SCIENCE LTD2022

Byzantine Fault-Tolerance in Decentralized Optimization under 2f-Redundancy

Nirupam Gupta

This paper considers the problem of exact Byzantine fault-tolerance in multi-agent decentralized optimization. We consider a complete peer-to-peer network of n agents; each agent has a local cost function, however, up to f of the agents are Byzantine fault ...

IEEE2021

Byzantine Fault-Tolerant Distributed Machine Learning with Norm-Based Comparative Gradient Elimination

Shuo Liu, Nirupam Gupta

This paper considers the Byzantine fault-tolerance problem in distributed stochastic gradient descent (D-SGD) method - a popular algorithm for distributed multi-agent machine learning. In this problem, each agent samples data points independently from a ce ...

IEEE COMPUTER SOC2021

Approximate Byzantine Fault-Tolerance in Distributed Optimization

Nirupam Gupta

This paper considers the problem of Byzantine fault-tolerance in distributed multi-agent optimization. In this problem, each agent has a local cost function, and in the fault-free case, the goal is to design a distributed algorithm that allows all the agen ...

ASSOC COMPUTING MACHINERY2021

Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?

Rachid Guerraoui, Sébastien Louis Alexandre Rouault, John Stephan, Nirupam Gupta, Rafaël Benjamin Pinot

This paper addresses the problem of combining Byzantine resilience with privacy in machine learning (ML). Specifically, we study if a distributed implementation of the renowned Stochastic Gradient Descent (SGD) learning algorithm is feasible with both diff ...

ASSOC COMPUTING MACHINERY2021