Publications associées à F-divergence

On Speed and Advantage : Results in Information Velocity and Monitoring Problems

Information theory has allowed us to determine the fundamental limit of various communication and algorithmic problems, e.g., the channel coding problem, the compression problem, and the hypothesis testing problem. In this work, we revisit the assumptions ...

EPFL2024

A structured prediction approach for robot imitation learning

Aude Billard, Iason Batzianoulis, Anqing Duan

We propose a structured prediction approach for robot imitation learning from demonstrations. Among various tools for robot imitation learning, supervised learning has been observed to have a prominent role. Structured prediction is a form of supervised le ...

Sage Publications Ltd2023

A KL Divergence-Based Loss for In Vivo Ultrafast Ultrasound Image Enhancement with Deep Learning

Jean-Philippe Thiran, Roser Vinals Terres

Ultrafast ultrasound imaging, characterized by high frame rates, generates low-quality images. Convolutional neural networks (CNNs) have demonstrated great potential to enhance image quality without compromising the frame rate. However, CNNs have been most ...

2023

Lower-bounds on the Bayesian Risk in Estimation Procedures via f–Divergences

Michael Christoph Gastpar, Amedeo Roberto Esposito, Adrien Vandenbroucque

We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of f-Divergences. The results are then applied to specific settings of interest and compared to other notable results i ...

2022

Quadrature-free immersed isogeometric analysis

Pablo Antolin Sanchez, Thibaut Hirschler

This paper presents a novel method for solving partial differential equations on three-dimensional CAD geometries by means of immersed isogeometric discretizations that do not require quadrature schemes. It relies on a newly developed technique for the eva ...

SPRINGER2022

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. ...

2021

Locally Differentially-Private Randomized Response for Discrete Distribution Learning

Michael Christoph Gastpar, Adriano Pastore

We consider a setup in which confidential i.i.d. samples X1, . . . , Xn from an unknown finite-support distribution p are passed through n copies of a discrete privatization chan- nel (a.k.a. mechanism) producing outputs Y1, . . . , Yn. The channel law gua ...

2021

Is there an analog of Nesterov acceleration for gradient-based MCMC?

Nicolas Henri Bernard Flammarion, Xiang Cheng

We formulate gradient-based Markov chain Monte Carlo (MCMC) sampling as optimization on the space of probability measures, with Kullback-Leibler (KL) divergence as the objective functional. We show that an under-damped form of the Langevin algorithm perfor ...

2021

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

On Vacuous and Non-Vacuous Generalization Bounds for Deep Neural Networks

Konstantinos Pitas

Deep neural networks have been empirically successful in a variety of tasks, however their theoretical understanding is still poor. In particular, modern deep neural networks have many more parameters than training data. Thus, in principle they should over ...

EPFL2020

Entropy as a tool for crystal discovery

Pablo Miguel Piaggi

The computational prediction of crystal structures has emerged as an useful alternative to expensive and often cumbersome experiments. We propose an approach to the prediction of crystal structures and polymorphism based on reproducing the crystallization ...

EPFL2019

An adaptive primal-dual framework for nonsmooth convex minimization

Volkan Cevher, Quoc Tran Dinh, Ahmet Alacaoglu

We propose a new self-adaptive and double-loop smoothing algorithm to solve composite, nonsmooth, and constrained convex optimization problems. Our algorithm is based on Nesterov’s smoothing technique via general Bregman distance functions. It self-adaptiv ...

2020

An Equation-Free Approach For Second Order Multiscale Hyperbolic Problems In Non-Divergence Form

Doghonay Arjmand

The present study concerns the numerical homogenization of second order hyperbolic equations in non-divergence form, where the model problem includes a rapidly oscillating coefficient function. These small scales influence the large scale behavior, hence t ...

INT PRESS BOSTON, INC2018

Mean-Field methods for Structured Deep-Learning in Computer Vision

Pierre Bruno Baqué

In recent years, Machine Learning based Computer Vision techniques made impressive progress. These algorithms proved particularly efficient for image classification or detection of isolated objects. From a probabilistic perspective, these methods can predi ...

EPFL2018

Multi-Modal Mean-Fields via Cardinality-Based Clamping

Pascal Fua, François Fleuret

Mean Field inference is central to statistical physics. It has attracted much interest in the Computer Vision community to efficiently solve problems expressible in terms of large Conditional Random Fields. However, since it models the posterior probabilit ...

Ieee2017

A methodology to detect and locate low-permeability faults to reduce the risk of inducing seismicity of fluid injection operations in deep saline formations

Lyesse Laloui, Victor Vilarrasa Riano

Fluid injection in the subsurface has significantly increased over the last decades. Fluid injection causes pressure buildup, which reduces effective stresses and thus, shear failure conditions may eventually occur, inducing microseismic and seismic events ...

Elsevier Sci Ltd2017

Information theoretic clustering for unsupervised domain-adaptation

Petr Motlicek, Subhadeep Dey

The aim of the domain-adaptation task for speaker verification is to exploit unlabelled target domain data by using the labelled source domain data effectively. The i-vector based Probabilistic Linear Dis- criminant Analysis (PLDA) framework approaches thi ...

IEEE2016

Distribution-Matching Embedding for Visual Domain Adaptation

Mathieu Salzmann

Domain-invariant representations are key to addressing the domain shift problem where the training and test examples follow different distributions. Existing techniques that have attempted to match the distributions of the source and target domains typical ...

2016

Electrical Neuroimaging with Irrotational Sources

This paper discusses theoretical aspects of the modeling of the sources of the EEG (i.e., the bioelectromagnetic inverse problem or source localization problem). Using theHelmholtz decomposition (HD) of the current density vector (CDV) of the primary curre ...

Hindawi Publishing Corporation2015