Lecture

Reinforcement Learning: One-step Horizon (Bandit Problems)

In course

This course provides an overview and introduces modern methods for reinforcement learning (RL.) The course starts with the fundamentals of RL, such as Q-learning, and delves into commonly used approac

Description

This lecture introduces the concept of Bandit Problems in Reinforcement Learning, where one has to choose between different actions and immediately receives a reward. The slides cover topics such as one-step horizon games, Q-values, optimal policy, iterative update rules, empirical averaging, and convergence in expectation.

Instructors (2)

Caglar Gulcehre

Wulfram Gerstner

Wulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.

Official source

Related lectures (30)

Interactive Lecture: Reinforcement Learning

Explores advanced reinforcement learning topics, including policies, value functions, Bellman recursion, and on-policy TD control.

Introduction to Reinforcement Learning: Concepts and Applications

Introduces reinforcement learning, covering its concepts, applications, and key algorithms.

Reinforcement Learning: Q-Learning

Covers Q-Learning, a model-free reinforcement learning algorithm, and its application to Tic-Tac-Toe with examples and quizzes.

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

Deep Learning Agents: Reinforcement Learning

Explores Deep Learning Agents in Reinforcement Learning, emphasizing neural network approximations and challenges in training multiagent systems.