Publications by Tao Lin | EPFL Graph Search

On Pitfalls of Test-time Adaptation

Alexandre Massoud Alahi, Yuejiang Liu, Tao Lin, Hao Zhao

2023

Algorithms for Efficient and Robust Distributed Deep Learning

The success of deep learning may be attributed in large part to remarkable growth in the size and complexity of deep neural networks. However, present learning systems raise significant efficiency concerns and privacy: (1) currently, training systems are l ...

EPFL2022

Learning Disentangled Behaviour Patterns for Wearable-based Human Activity Recognition

Tao Lin

In wearable-based human activity recognition (HAR) research, one of the major challenges is the large intra-class variability problem. The collected activity signal is often, if not always, coupled with noises or bias caused by personal, environmental, or ...

ASSOC COMPUTING MACHINERY2022

Consensus Control for Decentralized Deep Learning

Tao Lin, Martin Jaggi, Sebastian Urban Stich, Anastasiia Koloskova, Lingjing Kong

Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters. Experiments in earlier works reveal that, even in a data-center setup, decentralized training often suffers fro ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

Tao Lin, Sai Praneeth Reddy Karimireddy, Martin Jaggi, Sebastian Urban Stich

Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization ch ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them

Mathieu Salzmann, Sabine Süsstrunk, Tao Lin, Chen Liu

We analyze the influence of adversarial training on the loss landscape of machine learning models. To this end, we first provide analytical studies of the properties of adversarial loss functions under different adversarial budgets. We then demonstrate tha ...

2021

Dynamic Model Pruning with Feedback

Tao Lin, Martin Jaggi, Sebastian Urban Stich, Daniil Dmitriev, Luis Felipe Barba Flores

Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased latency at inference. We propose a novel model compression method that gener ...

2020

GA-Par: Dependable Microservice Orchestration Framework for Geo-Distributed Clouds

Tao Lin, Jie Xu

Recent advances in composing Cloud applications have been driven by deployments of inter-networking heterogeneous microservices across multiple Cloud datacenters. System dependability has been of the upmost importance and criticality to both service vendor ...

IEEE COMPUTER SOC2020

Ensemble Distillation for Robust Model Fusion in Federated Learning

Tao Lin, Martin Jaggi, Sebastian Urban Stich, Lingjing Kong

Federated Learning (FL) is a machine learning setting where many devices collaboratively train a machine learning model while keeping the training data decentralized. In most of the current training schemes the central model is refined by averaging the par ...

2020

Extrapolation for Large-batch Training in Deep Learning

Tao Lin, Martin Jaggi, Sebastian Urban Stich, Lingjing Kong

Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data. A major roadblock faced when increasing the batc ...

2020

Decentralized deep learning with arbitrary communication compression

Tao Lin, Martin Jaggi, Sebastian Urban Stich, Anastasiia Koloskova

Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks, as well as for efficient scaling to large compute clusters. As current approaches are limited by network bandwidth, we propose t ...

2019

Don't Use Large Mini-Batches, Use Local SGD

Tao Lin, Martin Jaggi, Sebastian Urban Stich

Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks. Drastic increases in the mini-batch sizes have lead to key efficiency and scalability gains in recent years. However, progress faces a major ...

2019

An Integrated Systems Genetics and Omics Toolkit to Probe Gene Function

Stephan Morgenthaler, Kristina Schoonjans, Johan Auwerx, Maroun Bou Sleiman, Hongbo Zhang, Daria Rukina, Karim Laurent Gariani, Xu Wang, Tao Lin, Peiling Luan, Hao Li, Vincenzo Sorrentino, Qingyao Huang, Naveed Ziari

2018

Gene locations may contribute to predicting gene regulatory relationships

Tao Lin

2018

Training DNNs with Hybrid Block Floating Point

Babak Falsafi, Tao Lin, Mario Paulo Drumond Lages De Oliveira, Martin Jaggi

The wide adoption of DNNs has given birth to unrelenting computing requirements, forcing datacenter operators to adopt domain-specific accelerators to train them. These accelerators typically employ densely packed full-precision floating-point arithmetic t ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Fog Orchestration for Internet of Things Services

Tao Lin, Jie Xu

Large-scale Internet of Things (IoT) services such as healthcare, smart cities, and marine monitoring are pervasive in cyber-physical environments. These complex IoT services are increasingly composed of sensors, devices, and compute resources within fog c ...

Ieee Computer Soc2017

Heterogeneous Recommendations: What You Might Like To Read After Watching Interstellar

Rachid Guerraoui, Anne-Marie Kermarrec, Rhicheek Patra, Tao Lin

Recommenders, as widely implemented nowadays by major e-commerce players like Netflix or Amazon, use collaborative filtering to suggest the most relevant items to their users. Clearly, the effectiveness of recommenders depends on the data they can exploit, ...

Assoc Computing Machinery2017

Hybrid Neural Networks for Learning the Trend in Time Series

Karl Aberer, Tian Guo, Tao Lin

The trend of time series characterizes the intermediate upward and downward behaviour of time series. Learning and forecasting the trend in time series data play an important role in many real applications, ranging from resource allocation in data centers, ...

2017