Publication

Non-filamentary non-volatile memory elements as synapses in neuromorphic systems

Yusuf Leblebici, Alessandro Fumarola, Severin Sidler
2019
Conference Papers

Abstract

Crossbar arrays of non-volatile memory (NVM) devices represent one possible path for implementing highly energy-efficient neuromorphic computing systems. For Deep Neural Networks (DNN), where information can be encoded as analog voltage and current levels, such arrays can represent matrices of synaptic weights, implementing the matrix-vector multiplication needed for algorithms such as backpropagation in a massively-parallel fashion. Previous research demonstrated a large-scale hardware-software implementation based on phasechange memories and analyzed the potential speed and power advantages over GPU-based training. In this proceeding we will discuss extensions of this work leveraging a different class of memory elements. Using the concept of jump-tables we simulate the impact of real conductance response of non-filamentary resistive devices based on Pr0.3Ca0.7 MnO3 (PCMO). With the same approach as of In we simulate a three-layer neural network with training accuracy > 90% on the MNIST dataset. The higher ON/OFF conductance ratio of improved Al/Mo/PCMO devices together with new programming strategies can lead to further accuracy improvement. Finally, we show that the bidirectional programming of Al/Mo/PCMO can be used to implement highdensity neuromorphic systems with a single conductance per synapse, at only a slight degradation to accuracy.

Official source

https://infoscience.epfl.ch/entities/publication/81a9f22a-dfa7-479b-a5dc-cf721ca94528

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Yusuf Leblebici, Alessandro Fumarola, Severin Sidler
2019
Conference Papers

Abstract

Official source

https://infoscience.epfl.ch/entities/publication/81a9f22a-dfa7-479b-a5dc-cf721ca94528

About this result

Related concepts (32)

Convolutional neural network

Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.

Feedforward neural network

A feedforward neural network (FNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. Its flow is uni-directional, meaning that the information in the model flows in only one direction—forward—from the input nodes, through the hidden nodes (if any) and to the output nodes, without any cycles or loops, in contrast to recurrent neural networks, which have a bi-directional flow.

Recurrent neural network

A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.

Related publications (30)

Generalization of Scaled Deep ResNets in the Mean-Field Regime

Volkan Cevher, Grigorios Chrysos, Fanghui Liu

Despite the widespread empirical success of ResNet, the generalization properties of deep ResNet are rarely explored beyond the lazy training regime. In this work, we investigate scaled ResNet in the limit of infinitely deep and wide neural networks, of wh ...

2024

Exploring High-Performance and Energy-Efficient Architectures for Edge AI-Enabled Applications

Joshua Alexander Harrison Klein

The desire and ability to place AI-enabled applications on the edge has grown significantly in recent years. However, the compute-, area-, and power-constrained nature of edge devices are stressed by the needs of the AI-enabled applications, due to a gener ...

EPFL2024

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Related publications (30)

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024