Publication

Multiple task learning in recurrent neural networks using layered reservoirs

Martin Hasler, Ali Ajdari Rad, Mahdi Jalili
2009
Conference Papers

Abstract

In this paper, to learn multiple tasks sharing same inputs, a two-layer architecture for a reservoir based recurrent neural network is proposed. The inputs are fed into the general workspace layer where the weights are adapted to provide maximum information for the next layers in an unsupervised approach. The outputs from this layer drive the task specific layers where the corresponding weights are optimized in a supervised process concerning the output error for each given task. Simulation results on a two-task example show that the proposed layered scheme outperforms other approaches, namely, the one with multiple readouts on a single reservoir and the one with two separate reservoirs.

Official source

https://infoscience.epfl.ch/entities/publication/5c793c86-3e84-4267-bb2e-1bb3695b3a1f

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Martin Hasler, Ali Ajdari Rad, Mahdi Jalili
2009
Conference Papers

Abstract

Official source

https://infoscience.epfl.ch/entities/publication/5c793c86-3e84-4267-bb2e-1bb3695b3a1f

About this result

Related concepts (32)

Feature learning

In machine learning, feature learning or representation learning is a set of techniques that allows a system to automatically discover the representations needed for feature detection or classification from raw data. This replaces manual feature engineering and allows a machine to both learn the features and use them to perform a specific task. Feature learning is motivated by the fact that machine learning tasks such as classification often require input that is mathematically and computationally convenient to process.

Recurrent neural network

A recurrent neural network (RNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. In contrast to uni-directional feedforward neural network, it is a bi-directional artificial neural network, meaning that it allows the output from some nodes to affect subsequent input to the same nodes. Their ability to use internal state (memory) to process arbitrary sequences of inputs makes them applicable to tasks such as unsegmented, connected handwriting recognition or speech recognition.

Convolutional neural network

Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections. For example, for each neuron in the fully-connected layer 10,000 weights would be required for processing an image sized 100 × 100 pixels.

Related publications (30)

Topics in statistical physics of high-dimensional machine learning

Hugo Chao Cui

In the past few years, Machine Learning (ML) techniques have ushered in a paradigm shift, allowing the harnessing of ever more abundant sources of data to automate complex tasks. The technical workhorse behind these important breakthroughs arguably lies in ...

EPFL2024

Robust machine learning for neuroscientific inference

Steffen Schneider

Modern neuroscience research is generating increasingly large datasets, from recording thousands of neurons over long timescales to behavioral recordings of animals spanning weeks, months, or even years. Despite a great variety in recording setups and expe ...

EPFL2024

Random matrix methods for high-dimensional machine learning models

Antoine Philippe Michel Bodin

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

EPFL2024