Lecture

Deep Deterministic Policy Gradient for Continuous Control

In course

This course provides an overview and introduces modern methods for reinforcement learning (RL.) The course starts with the fundamentals of RL, such as Q-learning, and delves into commonly used approac

Description

This lecture introduces the Deep Deterministic Policy Gradient (DDPG) algorithm, which combines Policy Gradient with Q-learning in an actor-critic architecture to efficiently train neural networks for continuous action spaces. The instructor explains the need for DDPG in dealing with infinitely many values for continuous actions, the use of a policy network to map states deterministically to continuous actions, and the training process involving target networks and replay buffers. Results from different environments are presented, showing the performance of DDPG compared to other methods like PPO and TRPO. The lecture concludes by discussing the importance of efficient exploration in reinforcement learning and strategies to maintain policy entropy for better performance.

Instructors (2)

Wulfram Gerstner

Wulfram Gerstner is Director of the Laboratory of Computational Neuroscience LCN at the EPFL. His research in computational neuroscience concentrates on models of spiking neurons and spike-timing dependent plasticity, on the problem of neuronal coding in single neurons and populations, as well as on the link between biologically plausible learning rules and behavioral manifestations of learning. He teaches courses for Physicists, Computer Scientists, Mathematicians, and Life Scientists at the EPFL. After studies of Physics in Tübingen and at the Ludwig-Maximilians-University Munich (Master 1989), Wulfram Gerstner spent a year as a visiting researcher in Berkeley. He received his PhD in theoretical physics from the Technical University Munich in 1993 with a thesis on associative memory and dynamics in networks of spiking neurons. After short postdoctoral stays at Brandeis University and the Technical University of Munich, he joined the EPFL in 1996 as assistant professor. Promoted to Associate Professor with tenure in February 2001, he is since August 2006 a full professor with double appointment in the School of Computer and Communication Sciences and the School of Life Sciences. Wulfram Gerstner has been invited speaker at numerous international conferences and workshops. He has served on the editorial board of the Journal of Neuroscience, Network: Computation in Neural Systems', Journal of Computational Neuroscience', and `Science'.

Caglar Gulcehre

Official source

Related lectures (29)

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

The Hidden Convex Optimization Landscape of Deep Neural Networks

Explores the hidden convex optimization landscape of deep neural networks, showcasing the transition from non-convex to convex models.

Statistical Physics in Machine Learning: Understanding Deep Learning

Explores the application of statistical physics in understanding deep learning with a focus on neural networks and machine learning challenges.

Foundations of Deep Learning: Transformer Architecture Overview

Covers the foundational concepts of deep learning and the Transformer architecture, focusing on neural networks, attention mechanisms, and their applications in sequence modeling tasks.

Deep Learning Agents: Reinforcement Learning

Explores Deep Learning Agents in Reinforcement Learning, emphasizing neural network approximations and challenges in training multiagent systems.