Publications associées à Rprop

Biologically plausible unsupervised learning in shallow and deep neural networks

The way our brain learns to disentangle complex signals into unambiguous concepts is fascinating but remains largely unknown. There is evidence, however, that hierarchical neural representations play a key role in the cortex. This thesis investigates biolo ...

EPFL2021

Second-order guarantees in centralized, federated and decentralized nonconvex optimization

Ali H. Sayed, Stefan Vlaski

Rapid advances in data collection and processing capabilities have allowed for the use of increasingly complex models that give rise to nonconvex optimization problems. These formulations, however, can be arbitrarily difficult to solve in general, in the s ...

INT PRESS BOSTON, INC2020

Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

Florent Gérard Krzakala, Julien Marcel Daniel Emmanuel Launay

Despite being the workhorse of deep learning, the backpropagation algorithm is no panacea. It enforces sequential layer updates, thus preventing efficient parallelization of the training process. Furthermore, its biological plausibility is being challenged ...

Curran Associates, Inc.2020

Backpropagation-Friendly Eigendecomposition

Pascal Fua, Mathieu Salzmann, Zheng Dang, Wei Wang, Yinlin Hu

Eigendecomposition (ED) is widely used in deep networks. However, the backpropagation of its results tends to be numerically unstable, whether using ED directly or approximating it with the Power Iteration method, particularly when dealing with large matri ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2019

Biologically plausible deep learning – but how far can we go with shallow networks?

Wulfram Gerstner, Johanni Michael Brea, Bernd Albert Illing

Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining s ...

2019

Non-intrusive reduced order modeling of nonlinear problems using neural networks

Jan Sickmann Hesthaven, Stefano Ubbiali

We develop a non-intrusive reduced basis (RB) method for parametrized steady-state partial differential equations (PDEs). The method extracts a reduced basis from a collection of high-fidelity solutions via a proper orthogonal decomposition (POD) and emplo ...

2018

Optimal Distributed Learning with Multi-pass Stochastic Gradient Methods

Volkan Cevher, Junhong Lin

We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We investigate distributed stochastic gradient methods (SGM), with mini-batches and multi-passes over th ...

2018

Accelerating Machine Learning with Non-Volatile Memory: exploring device and circuit tradeoffs

Alessandro Fumarola, Severin Sidler

Large arrays of the same nonvolatile memories (NVM) being developed for Storage-Class Memory (SCM) such as Phase Change Memory (PCM) and Resistance RAM (ReRAM) - can also be used in non-Von Neumann neuromorphic computational schemes, with device conductanc ...

Ieee2016

A neural approach to drugs monitoring for personalized medicine

Giovanni De Micheli, Sandro Carrara, Camilla Baj-Rossi, Marius Schirmer

The development of fast and mobile drug detection is an important aspect of personalized medicine. It enables the quick assessment of inter-individual differences in drug metabolism and corresponding adjustments of the dose. Recent developments of amperome ...

2015

Application of Artificial Neural Networks in Assessing the Equilibrium Depth of Local Scour Around Bridge Piers

Philippe Müllhaupt, Ehsan Sarshari

Scour can have the effect of subsidence of the piers in bridges, which can ultimately lead to the total collapse of these systems. Effective bridge design needs appropriate information on the equilibrium depth of local scour. The flow field around bridge p ...

2015

Railway demand forecasting in revenue management using neural networks

Shadi Sharif Azadeh

This study analyses the use of neural networks to produce accurate forecasts of total bookings and cancellations before departure, of a major European rail operator. Effective forecasting models, can improve revenue performance of transportation companies ...

2013