Publications de Mario Geiger | EPFL Graph Search

A recipe for cracking the quantum scaling limit with machine learned electron densities

A long-standing goal of science is to accurately simulate large molecular systems using quantum mechanics. The poor scaling of current quantum chemistry algorithms on classical computers, however, imposes an effective limit of about a few dozen atoms on tr ...

IOP Publishing Ltd2023

Relative stability toward diffeomorphisms indicates performance in deep nets

Mario Geiger, Matthieu Wyart, Leonardo Petrini, Alessandro Favero

Understanding why deep nets can classify data in large dimensions remains a challenge. It has been proposed that they do so by becoming stable to diffeomorphisms, yet existing empirical measurements support that it is often not the case. We revisit this qu ...

IOP Publishing Ltd2022

E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials

Mario Geiger

An E(3)-equivariant deep learning interatomic potential is introduced for accelerating molecular dynamics simulations. The method obtains state-of-the-art accuracy, can faithfully describe dynamics of complex systems with remarkable sample efficiency. ...

Nature Portfolio2022

Geometric compression of invariant manifolds in neural networks

Mario Geiger, Matthieu Wyart, Leonardo Petrini

We study how neural networks compress uninformative input space in models where data lie in d dimensions, but the labels of which only vary within a linear manifold of dimension d(parallel to) < d. We show that for a one-hidden-layer network initialized wi ...

IOP PUBLISHING LTD2021

Finding symmetry breaking order parameters with Euclidean neural networks

Mario Geiger

Curie's principle states that "when effects show certain asymmetry, this asymmetry must be found in the causes that gave rise to them." We demonstrate that symmetry equivariant neural networks uphold Curie's principle and can be used to articulate many sym ...

AMER PHYSICAL SOC2021

Relative stability toward diffeomorphisms indicates performance in deep nets

Mario Geiger, Matthieu Wyart, Leonardo Petrini, Alessandro Favero

Understanding why deep nets can classify data in large dimensions remains a challenge. It has been proposed that they do so by becoming stable to diffeomorphisms, yet existing empirical measurements support that it is often not the case. We revisit this qu ...

2021

Landscape and training regimes in deep learning

Mario Geiger, Matthieu Wyart, Leonardo Petrini

Deep learning algorithms are responsible for a technological revolution in a variety oftasks including image recognition or Go playing. Yet, why they work is not understood.Ultimately, they manage to classify data lying in high dimension – a feat generical ...

2021

Loss landscape and symmetries in Neural Networks

Mario Geiger

Neural networks (NNs) have been very successful in a variety of tasks ranging from machine translation to image classification. Despite their success, the reasons for their performance are still not well-understood. This thesis explores two main themes: lo ...

EPFL2021

Asymptotic learning curves of kernel methods: empirical data versus teacher-student paradigm

Mario Geiger, Matthieu Wyart, Stefano Spigler

How many training data are needed to learn a supervised task? It is often observed that the generalization error decreases as n(-beta) where n is the number of training examples and beta is an exponent that depends on both data and algorithm. In this work ...

IOP PUBLISHING LTD2020

Disentangling feature and lazy training in deep neural networks

Mario Geiger, Arthur Ulysse Jacot-Guillarmod, Matthieu Wyart, Stefano Spigler, Arthur Jacot

Two distinct limits for deep learning have been derived as the network width h -> infinity, depending on how the weights of the last layer scale with h. In the neural tangent Kernel (NTK) limit, the dynamics becomes linear in the weights and is described b ...

IOP PUBLISHING LTD2020

Scaling description of generalization with number of parameters in deep learning

Clément Hongler, Mario Geiger, Arthur Ulysse Jacot-Guillarmod, Matthieu Wyart, Franck Raymond Gabriel, Stefano Spigler, Levent Dogus Sagun, Arthur Jacot

Supervised deep learning involves the training of neural networks with a large number N of parameters. For large enough N, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsitybased arguments would suggest th ...

IOP PUBLISHING LTD2020

Comparing dynamics: deep neural networks versus glassy systems

Mario Geiger, Matthieu Wyart, Stefano Spigler, Levent Dogus Sagun, Marco Baity-Jesi

We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are (1) the complexity of the loss landscape and of the dynamics within it, and ( ...

IOP PUBLISHING LTD2019

Jamming transition as a paradigm to understand the loss landscape of deep neural networks

Mario Geiger, Matthieu Wyart, Stefano Spigler, Levent Dogus Sagun, Marco Baity-Jesi

Deep learning has been immensely successful at a variety of tasks, ranging from classification to artificial intelligence. Learning corresponds to fitting training data, which is implemented by descending a very high-dimensional loss function. Understandin ...

AMER PHYSICAL SOC2019

A General Theory of Equivariant CNNs on Homogeneous Spaces

Mario Geiger

We present a general theory of Group equivariant Convolutional Neural Networks (G-CNNs) on homogeneous spaces such as Euclidean space and the sphere. Feature maps in these networks represent fields on a homogeneous base space, and layers are equivariant ma ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2019

3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data

Mario Geiger

We present a convolutional network that is equivariant to rigid body motions. The model uses scalar-, vector-, and tensor fields over 3D Euclidean space to represent data, and equivariant convolutions to map between such representations. These SE(3)-equiva ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Comparing Dynamics: Deep Neural Networks versus Glassy Systems

Rainer Wesche, Mario Geiger, Stefano Spigler, Levent Dogus Sagun, Marco Baity-Jesi

We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are the complexity of the loss-landscape and of the dynamics within it, and to wh ...

2018

CFSpro: ray tracing for design and optimization of complex fenestration systems using mixed dimensionality approach

Jean-Louis Scartezzini, Andreas Schueler, André Gabriel Kostro, Mario Geiger

Advanced optical ray tracing software, CFSpro, was developed for the study and optimization of complex fenestration systems (CFSs). Using an algorithm mixing 2D and 3D approaches, accurate computation of large numbers of rays in extruded geometries can be ...

Optical Soc Amer2016

Thermal solar collector with VO2 absorber coating and V1-xWxO2 thermochromic glazing – Temperature matching and triggering

Rosendo Sanjines, Andreas Schueler, Mario Geiger, Antonio Paone

Overheating is a common problem both with the use of active and passive solar energy in thermal solar energy systems and in highly glazed buildings, even in central European latitudes. In solar thermal collectors, the elevated temperatures occurring during ...

2014

Embedded microstructures for daylighting and seasonal thermal control

Yves Leterrier, Jean-Louis Scartezzini, Andreas Schueler, André Gabriel Kostro, Mario Geiger, Marina Aymara González Lazo, Nicolas Jolissaint

A novel concept for an advanced fenestration system was studied and samples were produced to demonstrate the feasibility. The resulting novel glazing will combine the functions of daylighting, glare protection, and seasonal thermal control. Coated microstr ...

Spie-Int Soc Optical Engineering2012

Towards microstructured glazing for daylighting and thermal control

Jean-Louis Scartezzini, Andreas Schueler, André Gabriel Kostro, Mario Geiger

Glass is a central element to modern architecture and can cover up to 100% of a building façade. The main purpose of large glazed areas is to create bright, comfortable and healthy spaces. It was shown that increasing natural light in offices reduces sickn ...

EPFL2011