Publications related to Statistical distance

Lower-bounds on the Bayesian Risk in Estimation Procedures via f–Divergences

Michael Christoph Gastpar, Amedeo Roberto Esposito, Adrien Vandenbroucque

We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of f-Divergences. The results are then applied to specific settings of interest and compared to other notable results i ...

2022

Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs

Vinitra Swamy, Tanja Christina Käser Jacober, Bahar Radmehr, Mirko Marras, Natasa Krco

Neural networks are ubiquitous in applied machine learning for education. Their pervasive success in predictive performance comes alongside a severe weakness, the lack of explainability of their decisions, especially relevant in human-centric fields. We im ...

2022

A Wasserstein-based measure of conditional dependence

Negar Kiyavash, Seyed Jalal Etesami, Kun Zhang

Measuring conditional dependencies among the variables of a network is of great interest to many disciplines. This paper studies some shortcomings of the existing dependency measures in detecting direct causal influences or their lack of ability for group ...

2022

Locally Differentially-Private Randomized Response for Discrete Distribution Learning

Michael Christoph Gastpar, Adriano Pastore

We consider a setup in which confidential i.i.d. samples X1, . . . , Xn from an unknown finite-support distribution p are passed through n copies of a discrete privatization chan- nel (a.k.a. mechanism) producing outputs Y1, . . . , Yn. The channel law gua ...

2021

Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. ...

2021

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

From Data to Decisions: Distributionally Robust Optimization is Optimal

Daniel Kuhn, Peyman Mohajerin Esfahani

We study stochastic programs where the decision-maker cannot observe the distribution of the exogenous uncertainties but has access to a finite set of independent samples from this distribution. In this setting, the goal is to find a procedure that transfo ...

2021

Entropy as a tool for crystal discovery

Pablo Miguel Piaggi

The computational prediction of crystal structures has emerged as an useful alternative to expensive and often cumbersome experiments. We propose an approach to the prediction of crystal structures and polymorphism based on reproducing the crystallization ...

EPFL2019

Optimistic Distributionally Robust Optimization for Nonparametric Likelihood Approximation

Daniel Kuhn, Viet Anh Nguyen, Soroosh Shafieezadeh Abadeh, Wolfram Wiesemann

The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of the likelihood ...

2019

Predicting polymorphism in molecular crystals using orientational entropy

Pablo Miguel Piaggi, Michele Parrinello

We introduce a computational method to discover polymorphs in molecular crystals at finite temperature. The method is based on reproducing the crystallization process starting from the liquid and letting the system discover the relevant polymorphs. This id ...

National Academy of Sciences2018

Information theoretic clustering for unsupervised domain-adaptation

Petr Motlicek, Subhadeep Dey

The aim of the domain-adaptation task for speaker verification is to exploit unlabelled target domain data by using the labelled source domain data effectively. The i-vector based Probabilistic Linear Dis- criminant Analysis (PLDA) framework approaches thi ...

IEEE2016

Distribution-Matching Embedding for Visual Domain Adaptation

Mathieu Salzmann

Domain-invariant representations are key to addressing the domain shift problem where the training and test examples follow different distributions. Existing techniques that have attempted to match the distributions of the source and target domains typical ...

2016

Optimality of Operator-Like Wavelets for Representing Sparse AR(1) Processes

Michaël Unser, Pedram Pad

The discrete cosine transform (DCT) is known to be asymptotically equivalent to the Karhunen-Loève transform (KLT) of Gaussian first-order auto-regressive (AR(1)) processes. Since being uncorrelated under the Gaussian hypothesis is synonymous with independ ...

IEEE2015

A small-scale hyperacute compound eye featuring active eye tremor: application to visual stabilization, target tracking, and short-range odometry

Dario Floreano

In this study, a miniature artificial compound eye (15 mm in diameter) called the curved artificial compound eye (CurvACE) was endowed for the first time with hyperacuity, using similar micro-movements to those occurring in the fly's compound eye. A period ...

Iop Publishing Ltd2015

Principal Polynomial Analysis

Devis Tuia

This paper presents a new framework for manifold learning based on a sequence of principal polynomials that capture the possibly nonlinear nature of the data. The proposed Principal Polynomial Analysis (PPA) generalizes PCA by modeling the directions of ma ...

World Scientific Publ Co Pte Ltd2014

An Information Theoretic Approach to Speaker Diarization of Meeting Recordings

Deepu Vijayasenan

In this thesis we investigate a non parametric approach to speaker diarization for meeting recordings based on an information theoretic framework. The problem is formulated using the Information Bottleneck (IB) principle. Unlike other approaches where the ...

EPFL2010

An Information Theoretic Approach to Speaker Diarization of Meeting Data

Hervé Bourlard, Deepu Vijayasenan, Fabio Valente

A speaker diarization system based on an information theoretic framework is described. The problem is formulated according to the Information Bottleneck (IB) principle. Unlike other approaches where the distance between speaker segments is arbitrarily intr ...

2009

Using Comparison of Parallel Phoneme Probability streams for OOV Word Detection

Mathew Magimai Doss, Hynek Hermansky, Tamara Tosic

In this paper, we investigate the approach of comparing two different parallel streams of phoneme posterior probability estimates for OOV word detection. The ﬁrst phoneme posterior probability stream is estimated using only the knowledge of short-term acou ...

2008