Publications related to Artificial neuron

On the number of regions of piecewise linear neural networks

Michaël Unser, Alexis Marie Frederic Goujon

Many feedforward neural networks (NNs) generate continuous and piecewise-linear (CPWL) mappings. Specifically, they partition the input domain into regions on which the mapping is affine. The number of these so-called linear regions offers a natural metric ...

2024

Safe Deep Neural Networks

Kyle Michael Matoba

The capabilities of deep learning systems have advanced much faster than our ability to understand them. Whilst the gains from deep neural networks (DNNs) are significant, they are accompanied by a growing risk and gravity of a bad outcome. This is troubli ...

EPFL2024

3D trigonal FAPbI(3)-based multilevel resistive switching nonvolatile memory for artificial neural synapse

Mohammad Khaja Nazeeruddin, Paul Joseph Dyson, Yong Ding, Li Tao, Yao Zhang, Jun Zhang, Bowen Jiang

Hybrid perovskites have attracted enormous attention in the next generation resistive switching (RS) memristor for the artificial synapses, owing to their ambipolar charge transport, long diffusion length, and tunable visible bandgap. However, the variable ...

WILEY2023

Stable parameterization of continuous and piecewise-linear functions

Michaël Unser, Alexis Marie Frederic Goujon, Joaquim Gonçalves Garcia Barreto Campos

Rectified-linear-unit (ReLU) neural networks, which play a prominent role in deep learning, generate continuous and piecewise-linear (CPWL) functions. While they provide a powerful parametric representation, the mapping between the parameter and function s ...

2023

A Neural-Network-Based Convex Regularizer for Inverse Problems

Michaël Unser, Pakshal Narendra Bohra, Alexis Marie Frederic Goujon, Sebastian Jonas Neumayer, Stanislas Ducotterd

The emergence of deep-learning-based methods to solve image-reconstruction problems has enabled a significant increase in quality. Unfortunately, these new methods often lack reliability and explainability, and there is a growing interest to address these ...

2023

An exact mapping from ReLU networks to spiking neural networks

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Ana Stanojevic, Giovanni Cherubini, Angeliki Pantazi

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an ...

2023

A Theory of Finite-Width Neural Networks: Generalization, Scaling Laws, and the Loss Landscape

Berfin Simsek

Deep learning has achieved remarkable success in various challenging tasks such as generating images from natural language or engaging in lengthy conversations with humans. The success in practice stems from the ability to successfully train massive neural ...

EPFL2023

Benign Overfitting in Deep Neural Networks under Lazy Training

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayesoptimal test error for classification while obtaining (nearly) zero-trai ...

2023

Playing Brains: The Ethical Challenges Posed by Silicon Sentience and Hybrid Intelligence in DishBrain

The convergence of human and artificial intelligence is currently receiving considerable scholarly attention. Much debate about the resulting Hybrid Minds focuses on the integration of artificial intelligence into the human brain through intelligent brain- ...

2023

Expectation consistency for calibration of neural networks

Florent Gérard Krzakala, Lenka Zdeborová, Lucas Andry Clarte, Bruno Loureiro, Bruno Loureiro

Despite their incredible performance, it is well reported that deep neural networks tend to be overoptimistic about their prediction confidence. Finding effective and efficient calibration methods for neural networks is therefore an important endeavour tow ...

2023

A new variable shape parameter strategy for RBF approximation using neural networks

Jan Sickmann Hesthaven

The choice of the shape parameter highly effects the behaviour of radial basis function (RBF) approximations, as it needs to be selected to balance between the ill-conditioning of the interpolation matrix and high accuracy. In this paper, we demonstrate ho ...

PERGAMON-ELSEVIER SCIENCE LTD2023

How deep convolutional neural networks lose spatial information with training

Matthieu Wyart, Umberto Maria Tomasini, Francesco Cagnetta, Leonardo Petrini

A central question of machine learning is how deep nets manage to learn tasks in high dimensions. An appealing hypothesis is that they achieve this feat by building a representation of the data where information irrelevant to the task is lost. For image da ...

Iop Publishing Ltd2023

On a Finite-Size Neuronal Population Equation

Tilo Schwalger, Valentin Marc Schmutz, Eva Löcherbach

Population equations for infinitely large networks of spiking neurons have a long tradition in theoret-ical neuroscience. In this work, we analyze a recent generalization of these equations to populations of finite size, which takes the form of a nonlinear ...

SIAM PUBLICATIONS2023

From event-based surprise to lifelong learning. A journey in the timescales of adaptation

Martin Louis Lucien Rémy Barry

Humans and animals constantly adapt to their environment over the course of their life. This thesis seeks to integrate various timescales of adaptation, ranging from the adaptation of synaptic connections between spiking neurons (milliseconds), rapid behav ...

EPFL2023

Sharp asymptotics on the compression of two-layer neural networks

Marco Mondelli

In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M < N nodes. More precisely, we consider the setting in which the weights of the target network are i.i.d. sub-Gaussian, and we minimiz ...

IEEE2022

Mesoscopic description of hippocampal replay and metastability in spiking neural networks with short-term plasticity

Tilo Schwalger, Valentin Marc Schmutz

Bottom-up models of functionally relevant patterns of neural activity provide an explicit link between neuronal dynamics and computation. A prime example of functional activity patterns are propagating bursts of place-cell activities called hippocampal rep ...

PUBLIC LIBRARY SCIENCE2022

Taming neuronal noise with large networks

Valentin Marc Schmutz

How does reliable computation emerge from networks of noisy neurons? While individual neurons are intrinsically noisy, the collective dynamics of populations of neurons taken as a whole can be almost deterministic, supporting the hypothesis that, in the br ...

EPFL2022

Nuclei with Up to A=6 Nucleons with Artificial Neural Network Wave Functions

Giuseppe Carleo

The ground-breaking works of Weinberg have opened the way to calculations of atomic nuclei that are based on systematically improvable Hamiltonians. Solving the associated many-body Schrodinger equation involves non-trivial difficulties, due to the non-per ...

SPRINGER WIEN2022

The EBRAINS Hodgkin-Huxley Neuron Builder: An online resource for building data-driven neuron models

Jean-Denis Georges Emile Courcol, Michele Migliore, Genrich Ivaska, Carmen Alina Lupascu, Luca Leonardo Bologna, Shailesh Appukuttan

In the last decades, brain modeling has been established as a fundamental tool for understanding neural mechanisms and information processing in individual cells and circuits at different scales of observation. Building data-driven brain models requires th ...

FRONTIERS MEDIA SA2022

Hands-on reservoir computing: a tutorial for practical implementation

Matteo Cucchi, Giuseppe Ciccone

This manuscript serves a specific purpose: to give readers from fields such as material science, chemistry, or electronics an overview of implementing a reservoir computing (RC) experiment with her/his material system. Introductory literature on the topic ...

IOP Publishing Ltd2022