Publications by Alireza Modirshanechi

Seeking the new, learning from the unexpected: Computational models of surprise and novelty in the brain

Human babies have a natural desire to interact with new toys and objects, through which they learn how the world around them works, e.g., that glass shatters when dropped, but a rubber ball does not. When their predictions are proven incorrect, such as whe ...

EPFL2024

Computational models of intrinsic motivation for curiosity and creativity

Wulfram Gerstner, Alireza Modirshanechi, Sophia Becker

We link Ivancovsky et al.'s novelty-seeking model (NSM) to computational models of intrinsically motivated behavior and learning. We argue that dissociating different forms of curiosity, creativity, and memory based on the involvement of distinct intrinsic ...

2024

Curiosity-driven exploration: foundations in neuroscience and computational modeling

Wulfram Gerstner, Alireza Modirshanechi

Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus th ...

Cell Press2023

Surprise and novelty in the brain

Wulfram Gerstner, Johanni Michael Brea, Alireza Modirshanechi, Sophia Becker

Notions of surprise and novelty have been used in various experimental and theoretical studies across multiple brain areas and species. However, 'surprise' and 'novelty' refer to different quantities in different studies, which raises concerns about whethe ...

CURRENT BIOLOGY LTD2023

Distributed and specific encoding of sensory, motor, and decision information in the mouse neocortex during goal-directed behavior

Wulfram Gerstner, Carl Petersen, Sylvain Crochet, Vahid Esmaeili, Anastasiia Oryshchuk, Alireza Modirshanechi, Christos Sourmpis

Goal-directed behaviors involve coordinated activity in many cortical areas, but whether the encoding of task variables is distributed across areas or is more specifically represented in distinct areas remains unclear. Here, we compared representations of ...

Cell Press2023

A taxonomy of surprise definitions

Wulfram Gerstner, Johanni Michael Brea, Alireza Modirshanechi

Surprising events trigger measurable brain activity and influence human behavior by affecting learning, memory, and decision-making. Currently there is, however, no consensus on the definition of surprise. Here we identify 18 mathematical definitions of su ...

2022

Brain signals of a Surprise-Actor-Critic model: Evidence for multiple learning modules in human decision making

Wulfram Gerstner, Marco Philipp Lehmann, Vasiliki Liakoni, Kerstin Preuschoff, Johanni Michael Brea, Alireza Modirshanechi

Learning how to reach a reward over long series of actions is a remarkable capability of humans, and potentially guided by multiple parallel learning modules. Current brain imaging of learning modules is limited by (i) simple experimental paradigms, (ii) e ...

ACADEMIC PRESS INC ELSEVIER SCIENCE2022

Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making

Wulfram Gerstner, Marco Philipp Lehmann, Michael Herzog, He Xu, Alireza Modirshanechi

Classic reinforcement learning (RL) theories cannot explain human behavior in the absence of external reward or when the environment changes. Here, we employ a deep sequential decision-making paradigm with sparse reward and abrupt environmental changes. To ...

2021

Rapid suppression and sustained activation of distinct cortical regions for a delayed sensory-triggered motor response

Wulfram Gerstner, Carl Petersen, Sylvain Crochet, Samuel Pavio Muscinelli, Georgios Foustoukos, Vahid Esmaeili, Keita Tamura, Anastasiia Oryshchuk, Marta Boscaglia, Yanqi Liu, Alireza Modirshanechi

The neuronal mechanisms generating a delayed motor response initiated by a sensory cue remain elusive. Here, we tracked the precise sequence of cortical activity in mice transforming a brief whisker stimulus into delayed licking using wide-field calcium im ...

2021

Learning in Volatile Environments With the Bayes Factor Surprise

Wulfram Gerstner, Vasiliki Liakoni, Johanni Michael Brea, Alireza Modirshanechi

Surprise-based learning allows agents to rapidly adapt to nonstationary stochastic environments characterized by sudden changes. We show that exact Bayesian inference in a hierarchical model gives rise to a surprise-modulated trade-off between forgetting o ...

MIT PRESS2021