Publications by Jean-Baptiste Francis Marie Juliette Cordonnier

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

EPFL2023

Point-of-care lung ultrasonography for early identification of mild COVID-19: a prospective cohort of outpatients in a Swiss screening center

Jean-Yves Meuwly, Mary-Anne Hartley, Jean-Baptiste Francis Marie Juliette Cordonnier, Luca Bosso, Tanguy Espejo, Siméon Schaad

Objectives Early identification of SARS-CoV-2 infection is important to guide quarantine and reduce transmission. This study evaluates the diagnostic performance of lung ultrasound (LUS), an affordable, consumable-free point-of-care tool, for COVID-19 scre ...

BMJ PUBLISHING GROUP2022

Attention is not all you need: pure attention loses rank doubly exponentially with depth

Jean-Baptiste Francis Marie Juliette Cordonnier, Andreas Loukas

Attention-based architectures have become ubiquitous in machine learning. Yet, our understanding of the reasons for their effectiveness remains limited. This work proposes a new way to understand self-attention networks: we show that their output can be de ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Differentiable Patch Selection for Image Recognition

Aravindh Mahendran, Jean-Baptiste Francis Marie Juliette Cordonnier

Neural Networks require large amounts of memory and compute to process high resolution images, even when only a small part of the image is actually informative for the task at hand. We propose a method based on a differentiable Top-K operator to select the ...

IEEE COMPUTER SOC2021

On the Relationship between Self-Attention and Convolutional Layers

Jean-Baptiste Francis Marie Juliette Cordonnier, Andreas Loukas, Martin Jaggi

Recent trends of incorporating attention mechanisms in vision have led re- searchers to reconsider the supremacy of convolutional layers as a primary build- ing block. Beyond helping CNNs to handle long-range dependencies, Ramachandran et al. (2019) showed ...

2020

Extrapolating Paths with Graph Neural Networks

Jean-Baptiste Francis Marie Juliette Cordonnier, Andreas Loukas

We consider the problem of path inference: given a path prefix, i.e., a partially observed sequence of nodes in a graph, we want to predict which nodes are in the missing suffix. In particular, we focus on natural paths occurring as a by-product of the int ...

IJCAI, Inc.2019

Sparsified SGD with Memory

Jean-Baptiste Francis Marie Juliette Cordonnier, Martin Jaggi, Sebastian Urban Stich

Huge scale machine learning problems are nowadays tackled by distributed optimization algorithms, i.e. algorithms that leverage the compute power of many devices for training. The communication overhead is a key bottleneck that hinders perfect scalability. ...

2018