Lectures related to Descent Methods and Newton Method with Line Search

Explores linear systems and iterative methods like gradient descent and conjugate gradient for efficient solutions.

Covers iterative methods for solving linear equations and analyzing convergence, including error control and positive definite matrices.

Gradient Descent

Covers the concept of gradient descent in scalar cases, focusing on finding the minimum of a function by iteratively moving in the direction of the negative gradient.

Quasi-Newton Methods

Introduces Quasi-Newton methods for optimization, explaining their advantages over traditional approaches like Gradient Descent and Newton's Method.

Newton's Method: Optimization & Indefiniteness

Covers Newton's Method for optimization and discusses the caveats of indefiniteness in optimization problems.

Coordinate Descent: Efficient Optimization Techniques

Covers coordinate descent, a method for optimizing functions by updating one coordinate at a time.

Newton's Method: Optimization Techniques

Explores optimization techniques like gradient descent, line search, and Newton's method for efficient problem-solving.

Newton Method: Convergence and Quadratic Care

Covers the Newton method and its convergence properties near the optimal point.

Linear Systems: Convergence and Methods

Explores linear systems, convergence, and solving methods with a focus on CPU time and memory requirements.

Optimality of Convergence Rates: Accelerated Gradient Descent

Explores the optimality of convergence rates in convex optimization, focusing on accelerated gradient descent and adaptive methods.

Conjugate Gradient Method: Iterative Optimization

Covers the conjugate gradient method, stopping criteria, and convergence properties in iterative optimization.

Convergence Analysis: Iterative Methods

Covers the convergence analysis of iterative methods and the conditions for convergence.

Newton Method: Convergence Analysis

Explores the Newton method for root finding and its convergence analysis, including the modified Newton method.

Lipschitz continuous Hessian and Newton's method

Explores the convergence of Newton's method and the CG algorithm for solving linear equations.

Optimization Techniques: Gradient Method Overview

Discusses the gradient method for optimization, focusing on its application in machine learning and the conditions for convergence.

Optimization Methods in Machine Learning

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Coordinate Descent: Optimization Strategies

Explores coordinate descent optimization strategies, emphasizing simplicity in optimization through one-coordinate updates and discussing the implications of different approaches.

Richardson Method: Preconditioned Iterative Solvers

Covers the Richardson method for solving linear systems with preconditioned iterative solvers and introduces the gradient method.

Optimization without Constraints: Gradient Method

Covers optimization without constraints using the gradient method to find the function's minimum.

Iterative Methods for Linear Equations

Introduces iterative methods for solving linear equations and discusses the gradient method for minimizing errors.