Publications de Levent Dogus Sagun

Scaling description of generalization with number of parameters in deep learning

Clément Hongler, Mario Geiger, Arthur Ulysse Jacot-Guillarmod, Matthieu Wyart, Franck Raymond Gabriel, Stefano Spigler, Levent Dogus Sagun, Arthur Jacot

Supervised deep learning involves the training of neural networks with a large number N of parameters. For large enough N, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsitybased arguments would suggest th ...

IOP PUBLISHING LTD2020

A jamming transition from under- to over-parametrization affects generalization in deep learning

Matthieu Wyart, Stefano Spigler, Levent Dogus Sagun

In this paper we first recall the recent result that in deep networks a phase transition, analogous to the jamming transition of granular media, delimits the over- and under-parametrized regimes where fitting can or cannot be achieved. The analysis leading ...

IOP PUBLISHING LTD2019

Comparing dynamics: deep neural networks versus glassy systems

Mario Geiger, Matthieu Wyart, Stefano Spigler, Levent Dogus Sagun, Marco Baity-Jesi

We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are (1) the complexity of the loss landscape and of the dynamics within it, and ( ...

IOP PUBLISHING LTD2019

Jamming transition as a paradigm to understand the loss landscape of deep neural networks

Mario Geiger, Matthieu Wyart, Stefano Spigler, Levent Dogus Sagun, Marco Baity-Jesi

Deep learning has been immensely successful at a variety of tasks, ranging from classification to artificial intelligence. Learning corresponds to fitting training data, which is implemented by descending a very high-dimensional loss function. Understandin ...

AMER PHYSICAL SOC2019

Comparing Dynamics: Deep Neural Networks versus Glassy Systems

Rainer Wesche, Mario Geiger, Stefano Spigler, Levent Dogus Sagun, Marco Baity-Jesi

We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are the complexity of the loss-landscape and of the dynamics within it, and to wh ...

2018