Publications by Martin Jaggi | EPFL Graph Search

Attention with Markov: A Curious Case of Single-layer Transformers

Michael Christoph Gastpar, Martin Jaggi, Marco Bondaschi

Attention-based transformers have achieved tremendous success across a variety of disciplines including natural languages. To deepen our understanding of their sequential modeling capabilities, there is a growing interest in using Markov input processes to ...

ICLR2025

CoBo: Collaborative Learning via Bilevel Optimization

Lie He, Martin Jaggi

Collaborative learning is an important tool to train multiple clients more effectively by enabling communication among clients. Identifying helpful clients, however, presents challenging and often introduces significant overhead. In this paper, we model cl ...

2024

The Privacy Power of Correlated Noise in Decentralized Learning

Rachid Guerraoui, Martin Jaggi, Anastasiia Koloskova, Youssef Allouah, Aymane El Firdoussi

Decentralized learning is appealing as it enables the scalable usage of large amounts of distributed data and resources (without resorting to any central entity), while promoting privacy since every user minimizes the direct exposure of their data. Yet, wi ...

PMLR2024

MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks

Mary-Anne Hartley, Thijs Vogels, Thierry Bossy, Martin Jaggi, Tanja Christina Käser Jacober, Vinitra Swamy, Malika Satayeva, Jibril Albachir Frej

Predicting multiple real-world tasks in a single model often requires a particularly diverse feature space. Multimodal (MM) models aim to extract the synergistic predictive potential of multiple data types to create a shared feature space with aligned sema ...

2023

Beyond Spectral Gap: The Role of the Topology in Decentralized Learning

Thijs Vogels, Hadrien Hendrikx, Martin Jaggi

In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster. In the decentralized setting, in which workers commu ...

Microtome Publishing2023

Modular Clinical Decision Support Networks (MoDN)-Updatable, interpretable, and portable predictions for evolving clinical environments

Mary-Anne Hartley, Thijs Vogels, Martin Jaggi

Clinical Decision Support Systems (CDSS) have the potential to improve and standardise care with probabilistic guidance. However, many CDSS deploy static, generic rule-based logic, resulting in inequitably distributed accuracy and inconsistent performance ...

PUBLIC LIBRARY SCIENCE2023

SIMSUM: Document-level Text Simplification via Simultaneous Summarization

Martin Jaggi

Document-level text simplification is a specific type of simplification which involves simplifying documents consisting of several sentences by rewriting them into fewer or more sentences. In this paper, we propose a new two-stage framework SIMSUM for auto ...

Assoc Computational Linguistics-Acl2023

DeepBreath-automated detection of respiratory pathology from lung auscultation in 572 pediatric outpatients across 5 countries

Daniel Mueller, Jonathan Dönz, Mary-Anne Hartley, Juliane Dervaux, Tatjana Chavdarova, Julien Niklas Heitmann, Martin Jaggi, Daniel Hinjos García

The interpretation of lung auscultation is highly subjective and relies on non-specific nomenclature. Computer-aided analysis has the potential to better standardize and automate evaluation. We used 35.9 hours of auscultation audio from 572 pediatric outpa ...

NATURE PORTFOLIO2023

SKILL: Structured Knowledge Infusion for Large Language Models

Martin Jaggi, Fedor Moiseev

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2022

Implicit Gradient Alignment in Distributed and Federated Learning

Martin Jaggi, Luis Felipe Barba Flores

A major obstacle to achieving global convergence in distributed and federated learning is the misalignment of gradients across clients, or mini-batches due to heterogeneity and stochasticity of the distributed data. In this work, we show that data heteroge ...

ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE2022

Predicting Changes in Depression Severity Using the PSYCHE-D (Prediction of Severity Change-Depression) Model Involving Person-Generated Health Data: Longitudinal Case-Control Observational Study

Mariko Makhmutova, Martin Jaggi

Background: In 2017, an estimated 17.3 million adults in the United States experienced at least one major depressive episode, with 35% of them not receiving any treatment. Underdiagnosis of depression has been attributed to many reasons, including stigma s ...

Jmir Publications, Inc2022

Effective Interplay Between Sparsity and Quantization: from Theory to Practice

Babak Falsafi, Martin Jaggi

The increasing size of deep neural networks (DNNs) necessitates effective model compression to reduce their computational and memory footprints. Sparsity and quantization are two prominent compression methods that have been shown to reduce DNNs' computatio ...

2022

High-speed identification of suspended carbon nanotubes using Raman spectroscopy and deep learning

Martin Jaggi, Luis Felipe Barba Flores

The identification of nanomaterials with the properties required for energy-efficient electronic systems is usually a tedious human task. A workflow to rapidly localize and characterize nanomaterials at the various stages of their integration into large-sc ...

SPRINGERNATURE2022

Masked Training of Neural Networks with Partial Gradients

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extra-gradient), limiting SGD updates to a sub ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

what If...? Pandemic Policy-decision-support to Guide a Cost-benefit-optimised, Country- Specific Response

Mary-Anne Hartley, Thierry Bossy, Tatjana Chavdarova, Prakhar Gupta, Martin Jaggi

Background After 18 months of responding to the COVID-19 pandemic, there is still no agreement on the optimal combination of mitigation strategies. The efficacy and collateral damage of pandemic policies are dependent on constantly evolving viral epidemiol ...

PUBLIC LIBRARY SCIENCE2022

Learning from History for Byzantine Robust Optimization

Lie He, Sai Praneeth Reddy Karimireddy, Martin Jaggi

Byzantine robustness has received significant attention recently given its importance for distributed and federated learning. In spite of this, we identify severe flaws in existing algorithms even when the data across the participants is identically distri ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Semantic Perturbations with Normalizing Flows for Improved Generalization

Tatjana Chavdarova, Martin Jaggi, Sebastian Urban Stich

Data augmentation is a widely adopted technique for avoiding overfitting when training deep neural networks. However, this approach requires domain-specific knowledge and is often limited to a fixed set of hard-coded transformations. Recently, several work ...

IEEE2021

Advances and Open Problems in Federated Learning

Ayfer Özgür Aydin, Florian Tramèr, Qiang Yang, Ananda Theertha Suresh, Badih Ghazi, Lie He, Martin Jaggi, Sebastian Urban Stich, Yang Liu

Federated learning (FL) is a machine learning setting where many clients (e.g., mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g., service provider), while keeping the training data decen ...

NOW PUBLISHERS INC2021

Lightweight Cross-Lingual Sentence Representation Learning

Prakhar Gupta, Martin Jaggi, Zhuoyuan Mao

Large-scale models for learning fixed-dimensional cross-lingual sentence representations like LASER (Artetxe and Schwenk, 2019b) lead to significant improvement in performance on downstream tasks. However, further increases and modifications based on such ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2021

LENA: Communication-Efficient Distributed Learning with Self-Triggered Gradient Uploads

Martin Jaggi, Sebastian Urban Stich, Hossein Shokri Ghadikolaei

In distributed optimization, parameter updates from the gradient computing node devices have to be aggregated in every iteration on the orchestrating server. When these updates are sent over an arbitrary commodity network, bandwidth and latency can be limi ...

MICROTOME PUBLISHING2021