Publications associées à Consensus Control for Decentralized Deep Learning

Altruism, reciprocity, and tokens to reward forwarding data: Is that fair?

Verónica del Carmen Estrada Galiñanes, Arman Babaei

Decentralized storage networks offer services with intriguing possibilities to reduce inequalities in an extremely centralized market. Fair distribution of rewards, however, is still a persistent problem in the current generation of decentralized applicati ...

2024

Towards General-Purpose Decentralized Computing with Permissionless Extensibility

Enis Ceyhun Alp

Smart contracts have emerged as the most promising foundations for applications of the blockchain technology. Even though smart contracts are expected to serve as the backbone of the next-generation web, they have several limitations that hinder their wide ...

EPFL2024

Deep Learning Generalization with Limited and Noisy Labels

Mahsa Forouzesh

Deep neural networks have become ubiquitous in today's technological landscape, finding their way in a vast array of applications. Deep supervised learning, which relies on large labeled datasets, has been particularly successful in areas such as image cla ...

EPFL2023

Improving Generalization of Pretrained Language Models

Rabeeh Karimi Mahabadi

In this dissertation, we propose multiple methods to improve transfer learning for pretrained language models (PLMs). Broadly, transfer learning is a powerful technique in natural language processing, where a language model is first pre-trained on a data-r ...

EPFL2023

Fundamental Limits in Statistical Learning Problems: Block Models and Neural Networks

Elisabetta Cornacchia

This thesis focuses on two selected learning problems: 1) statistical inference on graphs models, and, 2) gradient descent on neural networks, with the common objective of defining and analysing the measures that characterize the fundamental limits. In th ...

EPFL2023

Communication-efficient distributed training of machine learning models

Thijs Vogels

In this thesis, we explore techniques for addressing the communication bottleneck in data-parallel distributed training of deep learning models. We investigate algorithms that either reduce the size of the messages that are exchanged between workers, or th ...

EPFL2023

Global, Passive Detection of Connection Tampering

Louis-Henri Manuel Jakob Merino

In-network devices around the world monitor and tamper with connections for many reasons, including intrusion prevention, combating spam or phishing, and country-level censorship. Connection tampering seeks to block access to specific domain names or keywo ...

Assoc Computing Machinery2023

Multi-agent reinforcement learning with graph convolutional neural networks for optimal bidding strategies of generation units in electricity markets

Olga Fink, Mina Montazeri

Finding optimal bidding strategies for generation units in electricity markets would result in higher profit. However, it is a challenging problem due to the system uncertainty which is due to the lack of knowledge of the strategies of other generation uni ...

PERGAMON-ELSEVIER SCIENCE LTD2023

Estimating the Topology of Preferential Attachment Graphs Under Partial Observability

Ali H. Sayed, Michele Cirillo

This work addresses the problem of learning the topology of a network from the signals emitted by the network nodes. These signals are generated over time through a linear diffusion process, where neighboring nodes exchange messages according to the underl ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2023

Equilibria in Network Constrained Energy Markets

Leonardo Massai

We study an energy market composed of producers who compete to supply energy to different markets and want to maximize their profits. The energy market is modeled by a graph representing a constrained power network where nodes represent the markets and lin ...

Elsevier2023

On the (In)security of Peer-to-Peer Decentralized Machine Learning

Mathilde Aliénor Raynal, Carmela González Troncoso, Dario Pasquini

In this work, we carry out the first, in-depth, privacy analysis of Decentralized Learning-a collaborative machine learning framework aimed at addressing the main limitations of federated learning. We introduce a suite of novel attacks for both passive and ...

IEEE COMPUTER SOC2023

Artificial Neural Network Training on an Optical Processor via Direct Feedback Alignment

Florent Gérard Krzakala, Julien Marcel Daniel Emmanuel Launay

Artificial Neural Networks (ANN) are habitually trained via the back-propagation (BP) algorithm. This approach has been extremely successful: Current models like GPT-3 have O(10 11 ) parameters, are trained on O(10 11 ) words and produce awe-inspiring resu ...

IEEE2023

Decentralized learning made easy with DecentralizePy

Anne-Marie Kermarrec, Akash Balasaheb Dhasade, Milos Vujasinovic, Rafael Pereira Pires, Rishi Sharma

Decentralized learning (DL) has gained prominence for its potential benefits in terms of scalability, privacy, and fault tolerance. It consists of many nodes that coordinate without a central server and exchange millions of parameters in the inherently ite ...

2023

Monotonicity Reasoning in the Age of Neural Foundation Models

Zeming Chen, Qiyue Gao

The recent advance of large language models (LLMs) demonstrates that these large-scale foundation models achieve remarkable capabilities across a wide range of language tasks and domains. The success of the statistical learning approach challenges our unde ...

Springer2023

Text Representation Learning for Low Cost Natural Language Understanding

Jan Frederik Jonas Florian Mai

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

EPFL2023

4M: Massively Multimodal Masked Modeling

Amir Roshan Zamir, Roman Christian Bachmann, Shuqing Teresa Yeo, David Mizrahi, Oguzhan Fatih Kar

Current machine learning models for vision are often highly specialized and limited to a single modality and task. In contrast, recent large language models exhibit a wide range of capabilities, hinting at a possibility for similarly versatile models in co ...

Neural Information Processing Systems (Nips)2023

TeSLA: Test-Time Self-Learning With Automatic Adversarial Augmentation

Jean-Philippe Thiran, Guillaume Marc Georges Vray, Seyedbehzad Bozorgtabar, Devavrat Tomar

Most recent test-time adaptation methods focus on only classification tasks, use specialized network architectures, destroy model calibration or rely on lightweight information from the source domain. To tackle these issues, this paper proposes a novel Tes ...

Los Alamitos2023

Model Predictive Control for Multi-Agent Systems under Limited Communication and Time-Varying Network Topology

Melanie Nicole Zeilinger, Danilo Saccani

In control system networks, reconfiguration of the controller when agents are leaving or joining the network is still an open challenge, in particular when operation constraints that depend on each agent's behavior must be met. Drawing our motivation from ...

IEEE2023

Caching and Neutrality

Aikaterini Argyraki, Pavlos Nikolopoulos, Muhammad Abdullah

We are used to defining network neutrality as absence of traffic differentiation, like policing or shaping. These mechanisms, however, are often not what determines end-users’ quality of experience (QoE). Most content today is accessed through edge caches, ...

ACM Association for Computing Machinery2023

Opinion Formation over Adaptive Networks

Virginia Bordignon

An adaptive network consists of multiple communicating agents, equipped with sensing and learning abilities that allow them to extract meaningful information from measurements. The objective of the network is to solve a global inference problem in a decent ...

EPFL2022