Publications by Hugo Chao Cui | EPFL Graph Search

A phase transition between positional and semantic learning in a solvable model of dot-product attention

Freya Behrens, Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová

Many empirical studies have provided evidence for the emergence of algorithmic mechanisms (abilities) in the learning of language models, that lead to qualitative improvements in the model capabilities. Yet, a theoretical characterization of how such mecha ...

2025

Bayes-optimal learning of deep random networks of extensive-width*

Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width ...

IOP Publishing Ltd2025

High-dimensional learning of narrow neural networks

Hugo Chao Cui

Recent years have been marked with the fast-pace diversification and increasing ubiquity of machine learning (ML) applications. Yet, a firm theoretical understanding of the surprising efficiency of neural networks (NNs) to learn from high-dimensional data ...

2025

Asymptotics of feature learning in two-layer networks after one gradient-step

Yue Lu, Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Luca Pesce

In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step. Leveraging the insight from (Ba et al., 2022), we model t ...

2024

Topics in statistical physics of high-dimensional machine learning

Hugo Chao Cui

In the past few years, Machine Learning (ML) techniques have ushered in a paradigm shift, allowing the harnessing of ever more abundant sources of data to automate complex tasks. The technical workhorse behind these important breakthroughs arguably lies in ...

EPFL2024

Deterministic equivalent and error universality of deep random features learning

Hugo Chao Cui

This manuscript considers the problem of learning a random Gaussian network function using a fully connected network with frozen intermediate layers and trainable readout layer. This problem can be seen as a natural generalization of the widely studied ran ...

2024

High-dimensional Asymptotics of Denoising Autoencoders

Hugo Chao Cui, Lenka Zdeborová

We address the problem of denoising data from a Gaussian mixture using a twolayer non-linear autoencoder with tied weights and a skip connection. We consider the high-dimensional limit where the number of training samples and the input dimension jointly te ...

2023

Bayes-optimal Learning of Deep Random Networks of Extensive-width

Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová

We consider the problem of learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights. We consider the asymptotic limit where the number of samples, the input dimension and the network width ...

2023

Error scaling laws for kernel classification under source and capacity conditions

Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro, Bruno Loureiro

In this manuscript we consider the problem of kernel classification. While worst-case bounds on the decay rate of the prediction error with the number of samples are known for some classifiers, they often fail to accurately describe the learning curves of ...

2023

Generalization error rates in kernel regression: the crossover from the noiseless to noisy regime*

Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro

In this manuscript we consider kernel ridge regression (KRR) under the Gaussian design. Exponents for the decay of the excess generalization error of KRR have been reported in various works under the assumption of power-law decay of eigenvalues of the feat ...

2022

Large deviations of semisupervised learning in the stochastic block model

Hugo Chao Cui, Lenka Zdeborová, Luca Saglietti

In semisupervised community detection, the membership of a set of revealed nodes is known in addition to the graph structure and can be leveraged to achieve better inference accuracies. While previous works investigated the case where the revealed nodes ar ...

AMER PHYSICAL SOC2022

Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regime

Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro

In this manuscript we consider Kernel Ridge Regression (KRR) under the Gaussian design. Exponents for the decay of the excess generalization error of KRR have been reported in various works under the assumption of power-law decay of eigenvalues of the feat ...

2021

Learning curves of generic features maps for realistic datasets with a teacher-student model

Hugo Chao Cui, Florent Gérard Krzakala, Lenka Zdeborová, Bruno Loureiro

Teacher-student models provide a framework in which the typical-case performance of high-dimensional supervised learning can be described in closed form. The assumptions of Gaussian i.i.d. input data underlying the canonical teacher-student model may, howe ...

2021