Publications associées à Entropie min

Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations

Given two jointly distributed random variables (X,Y), a functional representation of X is a random variable Z independent of Y, and a deterministic function g(⋅,⋅) such that X=g(Y,Z). The problem of finding a minimum entropy functional representation is kn ...

2023

A Functional Perspective on Information Measures

Amedeo Roberto Esposito

Since the birth of Information Theory, researchers have defined and exploited various information measures, as well as endowed them with operational meanings. Some were born as a "solution to a problem", like Shannon's Entropy and Mutual Information. Other ...

EPFL2022

Shannon entropy and degree of polarization of a speckle pattern

Abhijit Roy

The dependence of the Shannon entropy (SE) of a speckle pattern on the degree of polarization (DoP) of the pattern is investigated both experimentally and numerically. The superposition of two uncorrelated speckle patterns with polarization diversity is ut ...

OPTICAL SOC AMER2021

From Data to Decisions: Distributionally Robust Optimization is Optimal

Daniel Kuhn, Peyman Mohajerin Esfahani

We study stochastic programs where the decision-maker cannot observe the distribution of the exogenous uncertainties but has access to a finite set of independent samples from this distribution. In this setting, the goal is to find a procedure that transfo ...

2021

Robust Generalization via f−Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

Given two probability measures P and Q and an event E, we provide bounds on P(E) in terms of Q(E) and f-divergences. In particular, the bounds are instantiated when the measures considered are a joint distribution and the corresponding product of marginals ...

IEEE2020

Gaussian Process Regression for Maximum Entropy Distribution

Mohammadhossein Gorji, Mohsen Sadr

Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck for practical clo ...

2020

Gaussian Process Regression for Maximum Entropy Distribution

Mohsen Sadr

Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck for practical clo ...

2019

Negentropic linguistic evolution: A comparison of seven languages

Frédéric Kaplan, Vincent Christian Buntinx

The relationship between the entropy of language and its complexity has been the subject of much speculation – some seeing the increase of linguistic entropy as a sign of linguistic complexification or interpreting entropy drop as a marker of greater regul ...

Red de Humanidades Digitales A. C.2018

Polarization of the Renyi Information Dimension With Applications to Compressed Sensing

Emmanuel Abbé, Saeid Haghighatshoar

In this paper, we show that the Hadamard matrix acts as an extractor over the reals of the Renyi Information Dimension (RID), in an analogous way to how it acts as an extractor of the discrete entropy over finite fields. More precisely, we prove that the R ...

Ieee-Inst Electrical Electronics Engineers Inc2017

Spatial Coupling as a Proof Technique and Three Applications

Rüdiger Urbanke, Nicolas Macris, Andrei Giurgiu

The aim of this paper is to show that spatial coupling can be viewed not only as a means to build better graphical models, but also as a tool to better understand uncoupled models. The starting point is the observation that some asymptotic properties of gr ...

Ieee-Inst Electrical Electronics Engineers Inc2016

The Bethe Free Energy Allows to Compute the Conditional Entropy of Graphical Code Instances: A Proof From the Polymer Expansion

Nicolas Macris, Marc Vuffray

The main objective of this paper is to explore the precise relationship between the Bethe free energy (or entropy) and the Shannon conditional entropy of graphical error correcting codes. The main result shows that the Bethe free energy associated with a l ...

Ieee-Inst Electrical Electronics Engineers Inc2016

Affine-malleable Extractors, Spectrum Doubling, and Application to Privacy Amplification

Divesh Aggarwal

The study of seeded randomness extractors is a major line of research in theoretical computer science. The goal is to construct deterministic algorithms which can take a weak random source x with min-entropy k and a uniformly random seed Y of length d, and ...

Ieee2016

Statistical-thermodynamics modelling of the built environment in relation to urban ecology

Jean-Louis Scartezzini, Nahid Mohajeri Pour Rayeni

Various aspects of the built environment have important effects on ecology. Providing suitable metrics for the built forms so as to quantify and model their internal relations and external ecological footprints, however, remains a challenge. Here we provid ...

Elsevier2015

Multi Terminal Probabilistic Compressed Sensing

Saeid Haghighatshoar

In this paper, the 'Approximate Message Passing' (AMP) algorithm, initially developed for compressed sensing of signals under i.i.d. Gaussian measurement matrices, has been extended to a multi-terminal setting (MAMP algorithm). It has been shown that simil ...

Ieee2014

A New Entropy Power Inequality for Integer-Valued Random Variables

Emmanuel Abbé, Saeid Haghighatshoar

The entropy power inequality (EPI) yields lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for spe ...

Institute of Electrical and Electronics Engineers2014

Hierarchical sensor placement using joint entropy and the effect of modeling error

Ian Smith, Maria Papadopoulou

Good prediction of the behavior of wind around buildings improves designs for natural ventilation in warm climates. However wind modeling is complex, predictions are often inaccurate due to the large uncertainties in parameter values. The goal of this work ...

MDPI AG2014

Development of Electrokinetically Based Two-Dimensional Separation Methodologies for Proteomic Studies

Mohammad Reza Pourhaghighi

Proteomic studies require analytical methods capable of separating, identifying and quantifying thousands of proteins. Consequently, developing separation methodologies with high resolving power necessary to separate complex protein samples prior to protei ...

EPFL2012

Uncovering the Physics of Frustrated Quantum Magnets using the Correlation Density Matrix Approach

This work is devoted to the study, the development, and the application of a new systematic method yielding the dominant correlations that govern a quantum many-body state in an unbiased way. The dominant correlations between any two disjoint blocks of a s ...

EPFL2010

A Point is Normal for Almost All Maps beta x+ alpha mod 1 or Generalised beta-Transformations

Charles Pfister, Bastien Faller

We consider the map T-alpha,T-beta(x) := beta x + alpha mod 1, which admits a unique probability measure of maximal entropy. For x is an element of [0, 1], we show that the orbit of x is mu(alpha,beta)-normal for almost all (alpha, beta) is an element of [ ...

Cambridge University Press2009

Proof of replica formulas in the high noise regime for communication using LDGM codes

Nicolas Macris, Shrinivas Kudekar

We consider communication over a binary input memoryless output symmetric channel with low density generator matrix codes and optimal maximum a posteriori decoding. It is known that the problem of computing the average conditional entropy, over such code e ...

2008