Publications by Jean François Emmanuel Barbier

Statistical limits of dictionary learning: Random matrix theory and the spectral replica method

Nicolas Macris, Jean François Emmanuel Barbier

We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size. This is in contrast with most existin ...

AMER PHYSICAL SOC2022

Adaptive Path Interpolation Method for Sparse Systems: Application to a Censored Block Model

Nicolas Macris, Chun Lam Chan, Jean François Emmanuel Barbier

Recently, a new adaptive path interpolation method has been developed as a simple and versatile scheme to calculate exactly the asymptotic mutual information of Bayesian inference problems defined on dense factor graphs. These include random linear and gen ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Mutual information for low-rank even-order symmetric tensor estimation

Nicolas Macris, Jean François Emmanuel Barbier, Clément Dominique Luneau

We consider a statistical model for finite-rank symmetric tensor factorization and prove a single-letter variational expression for its asymptotic mutual information when the tensor is of even order. The proof applies the adaptive interpolation method orig ...

OXFORD UNIV PRESS2021

Concentration of Multi-overlaps for Random Dilute Ferromagnetic Spin Models

Nicolas Macris, Chun Lam Chan, Jean François Emmanuel Barbier

We consider mean field ferromagnetic spin models on dilute random graphs and prove that, with suitable one-body infinitesimal perturbations added to the Hamiltonian, the multi-overlaps concentrate for all temperatures, both with respect to the thermal Gibb ...

SPRINGER2020

High-dimensional rank-one nonsymmetric matrix decomposition: the spherical case

Nicolas Macris, Jean François Emmanuel Barbier, Clément Dominique Luneau

We consider the problem of estimating a rank-one nonsymmetric matrix under additive white Gaussian noise. The matrix to estimate can be written as the outer product of two vectors and we look at the special case in which both vectors are uniformly distribu ...

IEEE2020

Mutual Information and Optimality of Approximate Message-Passing in Random Linear Estimation

Nicolas Macris, Mohamad Baker Dia, Jean François Emmanuel Barbier, Florent Gérard Krzakala

We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access. There has been a ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

Entropy and mutual information in models of deep neural networks

Nicolas Macris, Jean François Emmanuel Barbier, Clément Dominique Luneau, Florent Gérard Krzakala, Lenka Zdeborová

We examine a class of stochastic deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) we show how entropies and mutual informations can be derived from heuristic statistical physics ...

2019

The committee machine: computational to statistical gaps in learning a two-layers neural network

Nicolas Macris, Jean François Emmanuel Barbier, Florent Gérard Krzakala, Lenka Zdeborová

Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks. In this paper, we provide a rigor ...

IOP PUBLISHING LTD2019

Mutual Information for the Stochastic Block Model by the Adaptive Interpolation Method

Nicolas Macris, Chun Lam Chan, Jean François Emmanuel Barbier

We rigorously derive a single-letter variational expression for the mutual information of the asymmetric two-groups stochastic block model in the dense graph regime. Existing proofs in the literature are indirect, as they involve mapping the model to a ran ...

IEEE2019

The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference

Nicolas Macris, Jean François Emmanuel Barbier

In recent years important progress has been achieved towards proving the validity of the replica predictions for the (asymptotic) mutual information (or free energy) in Bayesian inference problems. The proof techniques that have emerged appear to be quite ...

SPRINGER HEIDELBERG2019

The adaptive interpolation method for proving replica formulas. Applications to the Curie-Weiss and Wigner spike models

Nicolas Macris, Jean François Emmanuel Barbier

In this contribution we give a pedagogic introduction to the newly introduced adaptive interpolation method to prove in a simple and unified way replica formulas for Bayesian optimal inference problems. Many aspects of this method can already be explained ...

IOP PUBLISHING LTD2019

Universal Sparse Superposition Codes With Spatial Coupling and GAMP Decoding

Nicolas Macris, Mohamad Baker Dia, Jean François Emmanuel Barbier

Sparse superposition codes, or sparse regression codes, constitute a new class of codes, which was first introduced for communication over the additive white Gaussian noise (AWGN) channel. It has been shown that such codes are capacity-achieving over the A ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Mutual Information for Low-Rank Even-Order Symmetric Tensor Factorization

Nicolas Macris, Jean François Emmanuel Barbier, Clément Dominique Luneau

We consider a statistical model for finite-rank symmetric tensor factorization and prove a single-letter variational expression for its mutual information when the tensor is of even order. The proof uses the adaptive interpolation method, for which rank-on ...

IEEE2019

Optimal errors and phase transitions in high-dimensional generalized linear models

Nicolas Macris, Jean François Emmanuel Barbier, Florent Gérard Krzakala, Lenka Zdeborová

Generalized linear models (GLMs) are used in high-dimensional machine learning, statistics, communications, and signal processing. In this paper we analyze GLMs when the data matrix is random, as relevant in problems such as compressed sensing, error-corre ...

National Academy of Sciences2019

The Mutual Information in Random Linear Estimation Beyond i.i.d. Matrices

Nicolas Macris, Jean François Emmanuel Barbier, Florent Gérard Krzakala

There has been definite progress recently in proving the variational single-letter formula given by the heuristic replica method for various estimation problems. In particular, the replica formula for the mutual information in the case of noisy linear esti ...

IEEE2018

The committee machine: Computational to statistical gaps in learning a two-layers neural network

Nicolas Macris, Jean François Emmanuel Barbier, Florent Gérard Krzakala, Lenka Zdeborová

Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks. In this contribution, we provide ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Adaptive Path Interpolation for Sparse Systems: Application to a Simple Censored Block Model

Nicolas Macris, Chun Lam Chan, Jean François Emmanuel Barbier

A new adaptive path interpolation method has been recently developed as a simple and versatile scheme to calculate exactly the asymptotic mutual information of Bayesian inference problems defined on dense factor graphs. These include random linear and gene ...

IEEE2018

Entropy and mutual information in models of deep neural networks

Nicolas Macris, Jean François Emmanuel Barbier, Clément Dominique Luneau, Florent Gérard Krzakala, Lenka Zdeborová

We examine a class of stochastic deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutual informations can be derived from heuristic statistical physics ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Scampi: a robust approximate message-passing framework for compressive imaging

Jean François Emmanuel Barbier, Florent Gérard Krzakala

Reconstruction of images from noisy linear measurements is a core problem in image processing, for which convex optimization methods based on total variation (TV) minimization have been the long-standing state-of-the-art. We present an alternative probabil ...

Iop Publishing Ltd2016