Lecture

Trust Region Methods: Coming up with the Algorithm

Related lectures (22)

Explores practical aspects of solving parity games, including winning strategies, algorithms, complexity, determinism, and heuristic approaches.

Optimisation with Constraints: Interior Point Algorithm

Explores optimization with constraints using KKT conditions and interior point algorithm on two examples of quadratic programming.

Simplex Algorithm: Basics

Introduces the Simplex algorithm for solving flow problems and handling negative cost cycles.

RTR practical aspects + tCG

Explores practical aspects of Riemannian trust-region optimization and introduces the truncated conjugate gradient method.

Trust region methods: Global convergence with minimal effort

Explores trust region methods for global convergence with minimal effort in optimization on manifolds.

Branch & Bound: Optimization

Covers the Branch & Bound algorithm for efficient exploration of feasible solutions and discusses LP relaxation, portfolio optimization, Nonlinear Programming, and various optimization problems.

Trust region methods: framework & algorithms

Covers trust region methods, focusing on the framework and algorithms.

The Trouble with Quadratic Penalties: ALM as a Fix

Explores the challenges of quadratic penalties in optimization and the use of Augmented Lagrangian Methods (ALM) as a solution.

Optimization Techniques: Stochastic Gradient Descent and Beyond

Discusses optimization techniques in machine learning, focusing on stochastic gradient descent and its applications in constrained and non-convex problems.

Nonlinear Optimization Methods

Covers methods for solving nonlinear optimization problems, including direct search, Newton-Raphson, and branch and bound.

Descent methods and line search: Finiteness of the line search algorithm

Explores the Wolfe conditions for line search algorithms and proves the finiteness of the line search parameter.

Choosing a Step Size

Explores choosing a step size in optimization on manifolds, including backtracking line-search and the Armijo method.

Optimization Methods: Theory Discussion

Explores optimization methods, including unconstrained problems, linear programming, and heuristic approaches.

Linear Models for Classification

Covers linear models for classification, including SVM, decision boundaries, support vectors, and Lagrange duality.

Stochastic Gradient Descent: Non-convex Optimization Techniques

Discusses Stochastic Gradient Descent and its application in non-convex optimization, focusing on convergence rates and challenges in machine learning.

Gradient Descent

Covers the concept of gradient descent, a universal algorithm used to find the minimum of a function.

Hydrologic Parameters Optimization

Covers hydrologic parameters optimization using RS MINERVE software and analyzes the best solution.

Optimality of Convergence Rates: Accelerated Gradient Descent

Explores the optimality of convergence rates in convex optimization, focusing on accelerated gradient descent and adaptive methods.

Optimisation in Energy Systems

Explores optimization in energy system modeling, covering decision variables, objective functions, and different strategies with their pros and cons.

Newton Method: Convergence Analysis

Explores the Newton method for root finding and its convergence analysis, including the modified Newton method.