Publications de Sai Praneeth Reddy Karimireddy

Learning from History for Byzantine Robust Optimization

Lie He, Sai Praneeth Reddy Karimireddy, Martin Jaggi

Byzantine robustness has received significant attention recently given its importance for distributed and federated learning. In spite of this, we identify severe flaws in existing algorithms even when the data across the participants is identically distri ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

Tao Lin, Sai Praneeth Reddy Karimireddy, Martin Jaggi, Sebastian Urban Stich

Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization ch ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Optimization methods for collaborative learning

Sai Praneeth Reddy Karimireddy

A traditional machine learning pipeline involves collecting massive amounts of data centrally on a server and training models to fit the data. However, increasing concerns about the privacy and security of user's data, combined with the sheer growth in the ...

EPFL2021

Weight Erosion: An Update Aggregation Scheme for Personalized Collaborative Machine Learning

Mary-Anne Hartley, Sai Praneeth Reddy Karimireddy, Martin Jaggi, Felix Hans Michel Grimberg

Background. In medicine and other applications, the copying and sharing of data is impractical for a range of well-considered reasons. With federated learning (FL) techniques, machine learning models can be trained on data spread across several locations w ...

Springer, Cham2020

Practical Low-Rank Communication Compression in Decentralized Deep Learning

Thijs Vogels, Sai Praneeth Reddy Karimireddy, Martin Jaggi

Lossy gradient compression has become a practical tool to overcome the communication bottleneck in centrally coordinated distributed training of machine learning models. However, algorithms for decentralized training with compressed communication over arbi ...

2020

Accelerating Gradient Boosting Machines

Sai Praneeth Reddy Karimireddy

Gradient Boosting Machine (GBM) introduced by Friedman (2001) is a widely popular ensembling technique and is routinely used in competitions such as Kaggle and the KDD-Cup (Chen and Guestrin, 2016). In this work, we propose an Accelerated Gradient Boosting ...

ADDISON-WESLEY PUBL CO2020

The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Updates

Sai Praneeth Reddy Karimireddy, Sebastian Urban Stich

We analyze (stochastic) gradient descent (SGD) with delayed updates on smooth quasi-convex and non-convex functions and derive concise, non-asymptotic, convergence rates. We show that the rate of convergence in all cases consists of two terms: (i) a stocha ...

Microtome Publishing2020

SCAFFOLD Stochastic Controlled Averaging for Federated Learning

Ananda Theertha Suresh, Sai Praneeth Reddy Karimireddy, Sebastian Urban Stich

Federated Averaging (FEDAVG) has emerged as the algorithm of choice for federated learning due to its simplicity and low communication cost. However, in spite of recent research efforts, its performance is not fully understood. We obtain tight convergence ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2020

Efficient Greedy Coordinate Descent for Composite Problems

Sai Praneeth Reddy Karimireddy, Martin Jaggi, Sebastian Urban Stich, Anastasiia Koloskova

Coordinate descent with random coordinate selection is the current state of the art for many large scale optimization problems. However, greedy selection of the steepest coordinate on smooth problems can yield convergence rates independent of the dimension ...

2019

Error Feedback Fixes SignSGD and other Gradient Compression Schemes

Sai Praneeth Reddy Karimireddy, Martin Jaggi, Sebastian Urban Stich, Quentin Rebjock

Sign-based algorithms (e.g. signSGD) have been proposed as a biased gradient compression technique to alleviate the communication bottleneck in training large neural networks across multiple workers. We show simple convex counter-examples where signSGD doe ...

PMLR2019

Adaptive balancing of gradient and update computation times using global geometry and approximate subproblems

Sai Praneeth Reddy Karimireddy, Martin Jaggi, Sebastian Urban Stich

2018

On Matching Pursuit and Coordinate Descent

Sai Praneeth Reddy Karimireddy, Martin Jaggi, Sebastian Urban Stich, Anant Raj

Two popular examples of first-order optimization methods over linear spaces are coordinate descent and matching pursuit algorithms, with their randomized variants. While the former targets the optimization by moving along coordinates, the latter considers ...

2018