Recherche

Afficher tous les résultats pour

Accueil

Personne

Martin Jaggi

À propos
Confidentialité
Mentions légales

Graph Chatbot

Publications associées (84)

Consensus Control for Decentralized Deep Learning

Tao Lin, Martin Jaggi, Sebastian Urban Stich, Anastasiia Koloskova, Lingjing Kong

Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters. Experiments in earlier works reveal that, even in a data-center setup, decentralized training often suffers fro ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Faster Parallel Training of Word Embeddings

Martin Jaggi

Word embeddings have gained increasing popularity in the recent years due to the Word2vec library and its extension fastText that uses subword information. In this paper, we aim at improving the execution speed of fastText training on homogeneous multi- an ...

IEEE COMPUTER SOC2021

Exact Optimization of Conformal Predictors via Incremental and Decremental Learning

Martin Jaggi

Conformal Predictors (CP) are wrappers around ML models, providing error guarantees under weak assumptions on the data distribution. They are suitable for a wide range of problems, from classification and regression to anomaly detection. Unfortunately, the ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

Tao Lin, Sai Praneeth Reddy Karimireddy, Martin Jaggi, Sebastian Urban Stich

Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization ch ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Equinox: Training (for Free) on a Custom Inference Accelerator

Babak Falsafi, Ahmet Caner Yüzügüler, Mario Paulo Drumond Lages De Oliveira, Arash Pourhabibi Zarandi, Louis Coulon, Martin Jaggi

DNN inference accelerators executing online services exhibit low average loads because of service demand variability, leading to poor resource utilization. Unfortunately, reclaiming idle inference cycles is difficult as other workloads can not execute on a ...

ACM2021

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Martin Jaggi, Sebastian Urban Stich, Amirkeivan Mohtashami

It has been experimentally observed that the efficiency of distributed training with stochastic gradient (SGD) depends decisively on the batch size and—in asynchronous implementations—on the gradient staleness. Especially, it has been observed that the spe ...

MICROTOME PUBLISHING2021

An accelerated communication-efficient primal-dual optimization framework for structured machine learning

Chenxin Ma, Martin Jaggi

Distributed optimization algorithms are essential for training machine learning models on very large-scale datasets. However, they often suffer from communication bottlenecks. Confronting this issue, a communication-efficient primal-dual coordinate ascent ...

2021

Self-Supervised Neural Topic Modeling

Martin Jaggi

Topic models are useful tools for analyzing and interpreting the main underlying themes of large corpora of text. Most topic models rely on word co-occurrence for computing a topic, i.e., a weighted set of words that together represent a high-level semanti ...

Assoc Computational Linguistics-Acl2021

Deep learning diagnostic and risk-stratification pattern detection for COVID-19 in digital lung auscultations: clinical protocol for a case-control and prospective cohort study

Mary-Anne Hartley, Martin Jaggi

Background Lung auscultation is fundamental to the clinical diagnosis of respiratory disease. However, auscultation is a subjective practice and interpretations vary widely between users. The digitization of auscultation acquisition and interpretation is a ...

2021

Interpreting Language Models Through Knowledge Graph Extraction

Martin Jaggi, Angeliki Romanou, Vinitra Swamy

Transformer-based language models trained on large text corpora have enjoyed immense popularity in the natural language processing community and are commonly used as a starting point for downstream tasks. While these models are undeniably useful, it is a c ...

2021

Optimizer Benchmarking Needs to Account for Hyperparameter Tuning

François Fleuret, Thijs Vogels, Prabhu Teja Sivaprasad, Martin Jaggi, Jan Frederik Jonas Florian Mai

The performance of optimizers, particularly in deep learning, depends considerably on their chosen hyperparameter configuration. The efficacy of optimizers is often studied under near-optimal problem-specific hyperparameters, and finding these settings may ...

2020

Model Fusion via Optimal Transport

Martin Jaggi, Sidak Pal Singh

Combining different models is a widely used paradigm in machine learning applications. While the most common approach is to form an ensemble of models and average their individual predictions, this approach is often rendered infeasible by given resource co ...

2020

Practical Low-Rank Communication Compression in Decentralized Deep Learning

Thijs Vogels, Sai Praneeth Reddy Karimireddy, Martin Jaggi

Lossy gradient compression has become a practical tool to overcome the communication bottleneck in centrally coordinated distributed training of machine learning models. However, algorithms for decentralized training with compressed communication over arbi ...

2020

A Unified Theory of Decentralized SGD with Changing Topology and Local Updates

Martin Jaggi, Sebastian Urban Stich, Anastasiia Koloskova

Decentralized stochastic optimization methods have gained a lot of attention recently, mainly because of their cheap per iteration cost, data locality, and their communication-efficiency. In this paper we introduce a unified convergence analysis that cover ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2020

Weight Erosion: An Update Aggregation Scheme for Personalized Collaborative Machine Learning

Mary-Anne Hartley, Sai Praneeth Reddy Karimireddy, Martin Jaggi, Felix Hans Michel Grimberg

Background. In medicine and other applications, the copying and sharing of data is impractical for a range of well-considered reasons. With federated learning (FL) techniques, machine learning models can be trained on data spread across several locations w ...

Springer, Cham2020

Evaluating the search phase of neural architecture search

Mathieu Salzmann, Claudiu-Cristian Musat, Kaicheng Yu, Martin Jaggi

Neural Architecture Search (NAS) aims to facilitate the design of deep networks fornew tasks. Existing techniques rely on two stages: searching over the architecture space and validating the best architecture. NAS algorithms are currently compared solely b ...

2020

Dynamic Model Pruning with Feedback

Tao Lin, Martin Jaggi, Sebastian Urban Stich, Daniil Dmitriev, Luis Felipe Barba Flores

Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased latency at inference. We propose a novel model compression method that gener ...

2020

On the Relationship between Self-Attention and Convolutional Layers

Jean-Baptiste Francis Marie Juliette Cordonnier, Andreas Loukas, Martin Jaggi

Recent trends of incorporating attention mechanisms in vision have led re- searchers to reconsider the supremacy of convolutional layers as a primary build- ing block. Beyond helping CNNs to handle long-range dependencies, Ramachandran et al. (2019) showed ...

2020

Automated Essay Scoring in Foreign Language Students Based on Deep Contextualised Word Representations

Martin Jaggi, Bojana Rankovic

We introduce a method for automated grading of handwritten essays written by foreign language learners of French. The handwriting recognition system allows digitising the essays for further processing and functions at a low character error rate. The transc ...

2020

A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!

Martin Jaggi, Anastasiia Koloskova

Decentralized optimization methods enable on-device training of machine learning models without a central coordinator. In many scenarios communication between devices is energy demanding and time consuming and forms the bottleneck of the entire system. We ...

MICROTOME PUBLISHING2020

Page 2 sur 5