Publications associées à Principe d'entropie maximale

Information Spectrum Converse for Minimum Entropy Couplings and Functional Representations

Given two jointly distributed random variables (X,Y), a functional representation of X is a random variable Z independent of Y, and a deterministic function g(⋅,⋅) such that X=g(Y,Z). The problem of finding a minimum entropy functional representation is kn ...

2023

Learning curves for the multi-class teacher-student perceptron

Lenka Zdeborová, Bruno Loureiro, Elisabetta Cornacchia, Bruno Loureiro, Francesca Mignacco

One of the most classical results in high-dimensional learning theory provides a closed-form expression for the generalisation error of binary classification with a single-layer teacher-student perceptron on i.i.d. Gaussian inputs. Both Bayes-optimal (BO) ...

IOP Publishing Ltd2023

T-5 configurations and hyperbolic systems

Carl Johan Peter Johansson, Riccardo Tione

In this paper, we study the rank-one convex hull of a differential inclusion associated to entropy solutions of a hyperbolic system of conservation laws. This was introduced in [B. Kirchheim, S. Muller and V. S(sic)ver & aacute;k, Studying Nonlinear PDE by ...

WORLD SCIENTIFIC PUBL CO PTE LTD2023

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential entropy of the co ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Exploring quantum perceptron and quantum neural network structures with a teacher-student scheme

Near-term quantum devices can be used to build quantum machine learning models, such as quantum kernel methods and quantum neural networks (QNN), to perform classification tasks. There have been many proposals on how to use variational quantum circuits as ...

2022

A Maximum-Likelihood-Based Two-User Receiver for LoRa Chirp Spread-Spectrum Modulation

Andreas Peter Burg, Alexios Konstantinos Balatsoukas Stimming, Orion Afisiadis, Joachim Tobias Tapparel, Mathieu Pierre Xhonneux

Long Range (LoRa) is an emerging low-power wide-area network technology offering long-range wireless connectivity to Internet of Things (IoT) devices. For energy efficiency reasons, LoRa end nodes implement a nonslotted ALOHA multiple access scheme to tran ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Alpha-NML Universal Predictors

Michael Christoph Gastpar, Marco Bondaschi

Inspired by Sibson’s alpha-mutual information, we introduce a new parametric class of universal predictors. This class interpolates two well-known predictors, the mixture estimator, that includes the Laplace and the Krichevsky-Trofimov predictors, and the ...

2022

Atomic subgraphs and the statistical mechanics of networks

Sofia Charlotta Olhede

We develop random graph models where graphs are generated by connecting not only pairs of vertices by edges, but also larger subsets of vertices by copies of small atomic subgraphs of arbitrary topology. This allows for the generation of graphs with extens ...

2021

Differential Entropy of the Conditional Expectation under Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

This paper considers an additive Gaussian noise channel with arbitrarily distributed finite variance input signals. It studies the differential entropy of the minimum mean-square error (MMSE) estimator and provides a new lower bound which connects the diff ...

IEEE2021

Gaussian Process Regression for Maximum Entropy Distribution

Mohammadhossein Gorji, Mohsen Sadr

Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck for practical clo ...

2020

Entropy in Spin Relaxation, Spintronics, and Magnetic Resonance

Ferenc Simon

The entropy change during spin relaxation for a realistic model system is studied, whose spin dynamics can be handled with the Boltzmann equation. The time evolution of the von Neumann entropy is monitored during the process and is compared with the recent ...

WILEY-V C H VERLAG GMBH2020

Coarse-grained entropy production with multiple reservoirs: Unraveling the role of time scales and detailed balance in biology-inspired systems

Daniel Maria Busiello, Amos Maritan, Deepak Gupta

A general framework to describe a vast majority of biology-inspired systems is to model them as stochastic processes in which multiple couplings are in play at the same time. Molecular motors, chemical reaction networks, catalytic enzymes, and particles ex ...

2020

Artificial Neural Network Approach to the Analytic Continuation Problem

Oleg Yazyev, Quansheng Wu, Lei Wang, Romain Fournier

Inverse problems are encountered in many domains of physics, with analytic continuation of the imaginary Green's function into the real frequency domain being a particularly important example. However, the analytic continuation problem is ill defined and c ...

AMER PHYSICAL SOC2020

Information Theoretic Characterization of Uncertainty Distinguishes Surprise From Accuracy Signals in the Brain

Leyla Loued-Khenissi, Kerstin Preuschoff

Uncertainty presents a problem for both human and machine decision-making. While utility maximization has traditionally been viewed as the motive force behind choice behavior, it has been theorized that uncertainty minimization may supersede reward motivat ...

2020

Gaussian Process Regression for Maximum Entropy Distribution

Mohsen Sadr

Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck for practical clo ...

2019

Entropy as a tool for crystal discovery

Pablo Miguel Piaggi

The computational prediction of crystal structures has emerged as an useful alternative to expensive and often cumbersome experiments. We propose an approach to the prediction of crystal structures and polymorphism based on reproducing the crystallization ...

EPFL2019

Generalized maximum entropy estimation

Peyman Mohajerin Esfahani, John Lygeros, Tobias Sutter

We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel approximation scheme ...

2019

Entropy production for coarse-grained dynamics

Daniel Maria Busiello, Amos Maritan

Systems out of equilibrium exhibit a net production of entropy. We study the dynamics of a stochastic system represented by a Master equation (ME) that can be modeled by a Fokker-Planck equation in a coarse-grained, mesoscopic description. We show that the ...

2019

A sequence-dependent coarse-grain model of B-DNA with explicit description of bases and phosphate groups parametrised from large scale Molecular Dynamics simulations

Alessandro Samuele Patelli

We introduce a sequence-dependent coarse-grain model of double-stranded DNA with an explicit description of both the bases and the phosphate groups as interacting rigid-bodies. The model parameters are trained on extensive, state-of-the-art large scale mol ...

EPFL2019

An ensemble inverse optimal control approach for robotic task learning and adaptation

Aude Billard, Hang Yin, Ana Paiva

This paper contributes a novel framework to efficiently learn cost-to-go function representations for robotic tasks with latent modes. The proposed approach relies on the principle behind ensemble methods, where improved performance is obtained by aggregat ...

2019