Publications related to Automatic differentiation

Efficient local linearity regularization to overcome catastrophic overfitting

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Elias Abad Rocamora

Catastrophic overfitting (CO) in single-step adversarial training (AT) results in abrupt drops in the adversarial test accuracy (even down to 0%). For models trained with multi-step AT, it has been observed that the loss function behaves locally linearly w ...

2024

Heterogeneity and developmental dynamics of mammalian neocortical progenitors

Gioele La Manno, Antonio Herrera Camacho, Leila Haj Abdullah Alieh

The central nervous system develops from a pool of neural progenitors which, depending on their location and time of division, generate cells committed to differentiate into specific kinds of neurons or glia. In the last decades, the developmental neurobio ...

Elsevier2023

Robust Collaborative Learning with Linear Gradient Overhead

Rachid Guerraoui, John Stephan, Sadegh Farhadkhani, Le Nguyen Hoang, Nirupam Gupta, Rafaël Benjamin Pinot

Collaborative learning algorithms, such as distributed SGD (or D-SGD), are prone to faulty machines that may deviate from their prescribed algorithm because of software or hardware bugs, poisoned data or malicious behaviors. While many solutions have been ...

PLMR2023

Layer-Wise Learning Framework for Efficient DNN Deployment in Biomedical Wearable Systems

David Atienza Alonso, Amir Aminifar, Tomas Teijeiro Campo, Alireza Amirshahi, Saleh Baghersalimi

The development of low-power wearable systems requires specialized techniques to accommodate their unique requirements and constraints. While significant advancements have been made in the inference phase of artificial intelligence, the training phase rema ...

2023

Contextual Stochastic Bilevel Optimization

Daniel Kuhn, Yifan Hu, Andreas Krause, Jie Wang

We introduce contextual stochastic bilevel optimization (CSBO) -- a stochastic bilevel optimization framework with the lower-level problem minimizing an expectation conditioned on some contextual information and the upper-level decision variable. This fram ...

2023

Deep Learning for 3D Surface Modelling and Reconstruction

Benoît Alain René Guillard

In recent years, there has been a significant revolution in the field of deep learning, which has demonstrated its effectiveness in automatically capturing intricate patterns from large datasets. However, the majority of these successes in Computer Vision ...

EPFL2023

Constrained Form-Finding of Tension-Compression Structures using Automatic Differentiation

Stefana Parascho, Pierluigi D'Acunto

This paper proposes a computational approach to form-find pin-jointed bar structures subjected to combinations of tension and compression forces. The generated equilibrium states can meet structural and geometrical constraints via gradient-based optimizati ...

ELSEVIER SCI LTD2023

Autoencoders reloaded

Hervé Bourlard, Selen Hande Kabil

In Bourlard and Kamp (Biol Cybern 59(4):291-294, 1998), it was theoretically proven that autoencoders (AE) with single hidden layer (previously called "auto-associative multilayer perceptrons") were, in the best case, implementing singular value decomposit ...

SPRINGER2022

Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery

Jonathan Paul Sauder

Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and of ...

2022

Continual Test-Time Domain Adaptation

Olga Fink, Qin Wang

Test-time domain adaptation aims to adapt a source pretrained model to a target domain without using any source data. Existing works mainly consider the case where the target domain is static. However, real-world machine perception systems are running in n ...

IEEE COMPUTER SOC2022

Efficient and Accurate Physically-Based Differentiable Rendering

Delio Aleardo Vicini

Physically-based rendering algorithms generate photorealistic images of virtual scenes. By simulating light paths in a scene, complex physical effects such as shadows, reflections and volumetric scattering can be reproduced. Over the last decade, physicall ...

EPFL2022

Using wavelets to capture deviations from smoothness in galaxy-scale strong lenses

Frédéric Courbin, Aymeric Alexandre Galan, Georgios Vernardos, Austin Chandler Peel

Modeling the mass distribution of galaxy-scale strong gravitational lenses is a task of increasing difficulty. The high-resolution and depth of imaging data now available render simple analytical forms ineffective at capturing lens structures spanning a la ...

EDP SCIENCES S A2022

Differentiable Patch Selection for Image Recognition

Jean-Baptiste Francis Marie Juliette Cordonnier, Aravindh Mahendran

Neural Networks require large amounts of memory and compute to process high resolution images, even when only a small part of the image is actually informative for the task at hand. We propose a method based on a differentiable Top-K operator to select the ...

IEEE COMPUTER SOC2021

A Non-Exponential Transmittance Model for Volumetric Scene Representations

Wenzel Alban Jakob, Delio Aleardo Vicini

We introduce a novel transmittance model to improve the volumetric representation of 3D scenes. The model can represent opaque surfaces in the volumetric light transport framework. Volumetric representations are useful for complex scenes, and become increa ...

ASSOC COMPUTING MACHINERY2021

Large Steps in Inverse Rendering of Geometry

Wenzel Alban Jakob, Baptiste Alexandre Marie Philippe Claude Nicolet

Inverse reconstruction from images is a central problem in many scientific and engineering disciplines. Recent progress on differentiable rendering has led to methods that can efficiently differentiate the full process of image formation with respect to mi ...

ASSOC COMPUTING MACHINERY2021

Novel Methods for Incorporating Prior Knowledge for Automatic Speech Assessment

Subrahmanya Pavankumar Dubagunta

Speech signal conveys several kinds of information such as a message, speaker identity, emotional state of the speaker and social state of the speaker. Automatic speech assessment is a broad area that refers to using automatic methods to predict human judg ...

EPFL2021

A first-order primal-dual method with adaptivity to local smoothness

Volkan Cevher, Maria-Luiza Vladarean

We consider the problem of finding a saddle point for the convex-concave objective

\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)

, where

f

is a convex function with locally Lipschitz gradient and

g

is convex and possibly non-smooth. We propose an ...

2021

Shape Reconstruction by Learning Differentiable Surface Representations

Pascal Fua, Mathieu Salzmann, Shaifali Parashar, Jan Bednarík, Erhan Gündogdu

Generative models that produce point clouds have emerged as a powerful tool to represent 3D surfaces, and the best current ones rely on learning an ensemble of parametric representations. Unfortunately, they offer no control over the deformations of the su ...

IEEE2020

Radiative Backpropagation: An Adjoint Method for Lightning-Fast Differentiable Rendering

Wenzel Alban Jakob, Sébastien Nicolas Speierer, Merlin Eléazar Nimier-David

Physically based differentiable rendering has recently evolved into a powerful tool for solving inverse problems involving light. Methods in this area perform a differentiable simulation of the physical process of light transport and scattering to estimate ...

ASSOC COMPUTING MACHINERY2020

A multiphase CMAQ version 5.0 adjoint

Athanasios Nenes

We present the development of a multiphase adjoint for the Community Multiscale Air Quality (CMAQ) model, a widely used chemical transport model. The adjoint model provides location- and time-specific gradients that can be used in various applications such ...

2020