Concept

Bellman equation

Related lectures (32)

Explores the Asset Selling Problem to maximize long-term reward without a deadline.

Covers the fundamentals of optimal control theory, focusing on defining OCPs, existence of solutions, performance criteria, physical constraints, and the principle of optimality.

Infinite-Horizon Problems: Formulation & Complexity

Covers infinite-horizon problems in Applied Probability and Stochastic Processes.

Stochastic Optimal Control: Martingale Theorem

Explores Stochastic Optimal Control, emphasizing Optimal Consumption and Investment, the Martingale Representation Theorem, and the Verification Theorem.

Advanced Machine Learning: Discrete Reinforcement Learning

Introduces the basics of Reinforcement Learning, covering discrete states, actions, policies, value functions, MDPs, and optimal policies.

Linear Programming Techniques in Reinforcement Learning

Covers the linear programming approach to reinforcement learning, focusing on its applications and advantages in solving Markov decision processes.

Asset Selling: Optimal Revenue Policy

Explores asset selling dynamics, optimal revenue policy, acceptance thresholds, and commodity price impact.

Interactive Lecture: Reinforcement Learning

Explores advanced reinforcement learning topics, including policies, value functions, Bellman recursion, and on-policy TD control.

Bellman Equation: Value Consistency and Optimal Actions

Covers the Bellman equation, Q-values, discount factor, and optimal actions.

Optimal Portfolio Allocation: Euler Equation and Dynamic Programming

Covers the Euler equation, dynamic programming, and optimal consumption in portfolio allocation.

Optimal Marketing Strategy

Covers decision-making in marketing based on customer behavior for optimal strategies.

Asset Selling Problems

Discusses asset selling problems and optimal selling policies based on market conditions and termination states.

Numerical Simulation of SDEs: Monte Carlo & Optimal Control

Covers Monte Carlo methods, variance reduction, and stochastic optimal control, exploring simulation techniques, efficiency, and investment dynamics.

Dynamic Programming: Optimal Control

Explores Dynamic Programming for optimal control, focusing on stability, stationary policy, and recursive solutions.

Dynamic Programming: Optimal Control

Explores Dynamic Programming for optimal control, covering machine replacement, Markov chains, control policies, and linear quadratic problems.

Hamilton-Jacobi Equation: Exact Fields

Explores the Hamilton-Jacobi equation, optimal control, and Bellman principle in exact fields.

Temporal difference learning

Covers Reinforcement Learning theory, Q-Learning, and multi-step horizons.

Optimal Control: OCPs

Covers Optimal Control Problems focusing on necessary conditions, existence of optimal controls, and numerical solutions.

Newton's method on Riemannian manifolds

Covers Newton's method on Riemannian manifolds, focusing on second-order optimality conditions and quadratic convergence.

Riemannian gradient descent

Explores Riemannian gradient descent, covering Taylor expansions, optimality conditions, algorithm templates, line search, and critical points.