Publications related to Temporal difference learning

Unveiling the complexity of learning and decision-making

Reinforcement learning (RL) is crucial for learning to adapt to new environments. In RL, the prediction error is an important component that compares the expected and actual rewards. Dopamine plays a critical role in encoding these prediction errors. In my ...

EPFL2024

Computationally Efficient Reinforcement Learning: Targeted Exploration leveraging Simple Rules

Colin Neil Jones, Loris Di Natale, Bratislav Svetozarevic

Model-free Reinforcement Learning (RL) generally suffers from poor sample complexity, mostly due to the need to exhaustively explore the state-action space to find well-performing policies. On the other hand, we postulate that expert knowledge of the syste ...

2023

Beyond Spectral Gap: The Role of the Topology in Decentralized Learning

Martin Jaggi, Thijs Vogels, Hadrien Hendrikx

In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster. In the decentralized setting, in which workers commu ...

Microtome Publishing2023

Brain signals of a Surprise-Actor-Critic model: Evidence for multiple learning modules in human decision making

Wulfram Gerstner, Johanni Michael Brea, Alireza Modirshanechi, Kerstin Preuschoff, Marco Philipp Lehmann, Vasiliki Liakoni

Learning how to reach a reward over long series of actions is a remarkable capability of humans, and potentially guided by multiple parallel learning modules. Current brain imaging of learning modules is limited by (i) simple experimental paradigms, (ii) e ...

ACADEMIC PRESS INC ELSEVIER SCIENCE2022

Identifiability and Generalizability from Multiple Experts in Inverse Reinforcement Learning

Volkan Cevher, Paul Thierry Yves Rolland, Luca Viano, Boris Nikolov

While Reinforcement Learning (RL) aims to train an agent from a reward function in a given environment, Inverse Reinforcement Learning (IRL) seeks to recover the reward function from observing an expert’s behavior. It is well known that, in general, variou ...

2022

Temporal Pathways to Learning: How Learning Emerges in an Open-ended Collaborative Activity

Pierre Dillenbourg, Mortadha Abderrahim, Jauwairia Nasir, Aditi Kothiyal

The learning process depends on the nature of the learning environment, particularly in the case of open-ended learning environments, where the learning process is considered to be non-linear. In this paper, we report on the findings of employing a multimo ...

2022

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Surprise-based model estimation in reinforcement learning: algorithms and brain signatures

Vasiliki Liakoni

Learning how to act and adapting to unexpected changes are remarkable capabilities of humans and other animals. In the absence of a direct recipe to follow in life, behaviour is often guided by rewarding and by surprising events. A positive or a negative o ...

EPFL2021

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

Martin Jaggi, Amirkeivan Mohtashami, Sebastian Urban Stich

It has been experimentally observed that the efficiency of distributed training with stochastic gradient (SGD) depends decisively on the batch size and—in asynchronous implementations—on the gradient staleness. Especially, it has been observed that the spe ...

MICROTOME PUBLISHING2021

On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them

Sabine Süsstrunk, Mathieu Salzmann, Tao Lin, Chen Liu

We analyze the influence of adversarial training on the loss landscape of machine learning models. To this end, we first provide analytical studies of the properties of adversarial loss functions under different adversarial budgets. We then demonstrate tha ...

2021

Improving Multi-agent Coordination by Learning to Estimate Contention

Boi Faltings, Panayiotis Danassis, Florian Wiedemair

We present a multi-agent learning algorithm, ALMA-Learning, for efficient and fair allocations in large-scale systems. We circumvent the traditional pitfalls of multi-agent learning (e.g., the moving target problem, the curse of dimensionality, or the need ...

2021

Improving Multi-agent Coordination by Learning to Estimate Contention

Boi Faltings, Panayiotis Danassis, Florian Wiedemair

We present a multi-agent learning algorithm, ALMA-Learning, for efficient and fair allocations in large-scale systems. We circumvent the traditional pitfalls of multi-agent learning (e.g., the moving target problem, the curse of dimensionality, or the need ...

2021

Reward boosts reinforcement-based motor learning

Friedhelm Christoph Hummel, Pierre Theopistos Vassiliadis, Julie Duqué

Besides relying heavily on sensory and reinforcement feedback, motor skill learning may also depend on the level of motivation experienced during training. Yet, how motivation by reward modulates motor learning remains unclear. In 90 healthy subjects, we i ...

2021

Neural Correlates of Reinforcement Learning: eligibility trace, reward prediction error, novelty and surprise

He Xu

In reinforcement learning (RL), an agent makes sequential decisions to maximise the reward it can obtain from an environment. During learning, the actual and expected outcomes are compared to tell whether a decision was good or bad. The difference between ...

EPFL2020

Age-Dependent Statistical Learning Trajectories Reveal Differences in Information Weighting

Steffen Alexander Herff

Statistical learning (SL) is the ability to generate predictions based on probabilistic dependencies in the environment, an ability that is present throughout life. The effect of aging on SL is still unclear. Here, we explore statistical learning in health ...

AMER PSYCHOLOGICAL ASSOC2020

AdaptHD: Adaptive Efficient Training for Brain-Inspired Hyperdimensional Computing

Giovanni De Micheli, Samuel Bosch, Mohsen Imani

Brain-inspired Hyperdimensional (HD) computing is a promising solution for energy-efficient classification. However, the existing HD computing algorithms have a lack of controllability on the training iterations which often results in slow training or dive ...

2019

One-shot learning and eligibility traces in sequential decision making

Marco Philipp Lehmann

When humans or animals perform an action that led to a desired outcome, they show a tendency to repeat it. The mechanisms underlying learning from past experience and adapting future behavior are still not fully understood. In this thesis, I study how huma ...

EPFL2018

Time course of prediction errors in sequential decision making

Michael Herzog, He Xu

In reinforcement learning, an agent makes sequential decisions to maximize reward. During learning, the actual and expected outcome are compared to tell whether a decision was good or bad. The difference between the actual outcome and expected outcome is t ...

2018

Model-based reinforcement learning and navigation in animals and machines

Dane Sterling Corneil

For decades, neuroscientists and psychologists have observed that animal performance on spatial navigation tasks suggests an internal learned map of the environment. More recently, map-based (or model-based) reinforcement learning has become a highly activ ...

EPFL2018

Neural Correlates of Sequential Decision Making

Michael Herzog, He Xu

In chess, a series of moves is made until a delayed sparse feedback (win, loss) is issued, which makes it impossible to evaluate the value of a single move. There are powerful reinforcement learning (RL) algorithms, which can cope with these sequential dec ...

2017