Publication

The Gray-Wyner Network and Wyner's Common Information for Gaussian Sources

Related publications (31)

Spectral Estimators for High-Dimensional Matrix Inference

A key challenge across many disciplines is to extract meaningful information from data which is often obscured by noise. These datasets are typically represented as large matrices. Given the current trend of ever-increasing data volumes, with datasets grow ...

EPFL2024

On Speed and Advantage : Results in Information Velocity and Monitoring Problems

Reka Inovan

Information theory has allowed us to determine the fundamental limit of various communication and algorithmic problems, e.g., the channel coding problem, the compression problem, and the hypothesis testing problem. In this work, we revisit the assumptions ...

EPFL2024

Impact of phylogeny on structural contact inference from protein sequence data

Nicola Dietler, Umberto Lupo, Anne-Florence Raphaëlle Bitbol

Local and global inference methods have been developed to infer structural contacts from multiple sequence alignments of homologous proteins. They rely on correlations in amino acid usage at contacting sites. Because homologous proteins share a common ance ...

ROYAL SOC2023

Geometrically-Conditioned Point Diffusion Models

Pascal Fua, Eduard Trulls Fortuny, Michal Jan Tyszkiewicz

Diffusion models generating images conditionally on text, such as Dall-E 2 [51] and Stable Diffusion[53], have recently made a splash far beyond the computer vision com- munity. Here, we tackle the related problem of generating point clouds, both unconditi ...

2023

THE PHASE TRANSITION FOR PLANAR GAUSSIAN PERCOLATION MODELS WITHOUT FKG

Alejandro Rivera

We develop techniques to study the phase transition for planar Gaussian percolation models that are not (necessarily) positively correlated. These models lack the property of positive associations (also known as the 'FKG inequality'), and hence many classi ...

Inst Mathematical Statistics-Ims2023

A Functional Perspective on Information Measures

Amedeo Roberto Esposito

Since the birth of Information Theory, researchers have defined and exploited various information measures, as well as endowed them with operational meanings. Some were born as a "solution to a problem", like Shannon's Entropy and Mutual Information. Other ...

EPFL2022

Mutual information in changing environments: Nonlinear interactions, out-of-equilibrium systems, and continuously varying diffusivities

Daniel Maria Busiello, Giorgio Nicoletti

Biochemistry, ecology, and neuroscience are examples of prominent fields aiming at describing interacting systems that exhibit nontrivial couplings to complex, ever-changing environments. We have recently shown that linear interactions and a switching envi ...

AMER PHYSICAL SOC2022

A Wasserstein-based measure of conditional dependence

Seyed Jalal Etesami, Negar Kiyavash, Kun Zhang

Measuring conditional dependencies among the variables of a network is of great interest to many disciplines. This paper studies some shortcomings of the existing dependency measures in detecting direct causal influences or their lack of ability for group ...

2022

Lower-bounds on the Bayesian Risk in Estimation Procedures via f–Divergences

Michael Christoph Gastpar, Adrien Vandenbroucque, Amedeo Roberto Esposito

We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of f-Divergences. The results are then applied to specific settings of interest and compared to other notable results i ...

2022

From Generalisation Error to Transportation-cost Inequalities and Back

Michael Christoph Gastpar, Amedeo Roberto Esposito

In this work, we connect the problem of bounding the expected generalisation error with transportation-cost inequalities. Exposing the underlying pattern behind both approaches we are able to generalise them and go beyond Kullback- Leibler Divergences/Mutu ...

2022

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential entropy of the co ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Statistical limits of dictionary learning: Random matrix theory and the spectral replica method

Nicolas Macris, Jean François Emmanuel Barbier

We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size. This is in contrast with most existin ...

AMER PHYSICAL SOC2022

Local Nondeterminism and Local Times of the Stochastic Wave Equation Driven by Fractional-Colored Noise

Cheuk Yin Lee

We investigate the existence and regularity of the local times of the solution to a linear system of stochastic wave equations driven by a Gaussian noise that is fractional in time and colored in space. Using Fourier analytic methods, we establish strong l ...

SPRINGER BIRKHAUSER2022

Propagation of singularities for the stochastic wave equation

Cheuk Yin Lee

We study the existence and propagation of singularities of the solution to a one-dimensional linear stochastic wave equation driven by an additive Gaussian noise that is white in time and colored in space. Our approach is based on a simultaneous law of the ...

ELSEVIER2022

Correlations from structure and phylogeny combine constructively in the inference of protein partners from sequences

Nicola Dietler, Anne-Florence Raphaëlle Bitbol, Andonis Samuel Rene Gerardos

Inferring protein-protein interactions from sequences is an important task in computational biology. Recent methods based on Direct Coupling Analysis (DCA) or Mutual Information (MI) allow to find interaction partners among paralogs of two protein families ...

Public Library Science2022

Optimal denoising of rotationally invariant rectangular matrices

Florent Gérard Krzakala, Lenka Zdeborová, Emanuele Troiani, Vittorio Erba

In this manuscript we consider denoising of large rectangular matrices: given a noisy observation of a signal matrix, what is the best way of recovering the signal matrix itself? For Gaussian noise and rotationally-invariant signal priors, we completely ch ...

2022

Feedback and Common Information: Bounds and Capacity for Gaussian Networks

Erixhen Sula

Network information theory studies the communication of information in a network and considers its fundamental limits. Motivating from the extensive presence of the networks in the daily life, the thesis studies the fundamental limits of particular network ...

EPFL2021

Mutual information for low-rank even-order symmetric tensor estimation

Nicolas Macris, Jean François Emmanuel Barbier, Clément Dominique Luneau

We consider a statistical model for finite-rank symmetric tensor factorization and prove a single-letter variational expression for its asymptotic mutual information when the tensor is of even order. The proof applies the adaptive interpolation method orig ...

OXFORD UNIV PRESS2021

A Comparison of optimal measurement-system design with engineering judgement for bridge load testing

Ian Smith, Numa Joy Bertola

Due to conservative approaches in construction design and practice, infrastructure often has hidden reserve capacity. When quantified, this reserve has potential to improve decisions related to asset management. Field measurements, collected through load t ...

2021

Common Information Components Analysis

Michael Christoph Gastpar, Erixhen Sula

Wyner's common information is a measure that quantifies and assesses the commonality between two random variables. Based on this, we introduce a novel two-step procedure to construct features from data, referred to as Common Information Components Analysis ...

2021