Lectures related to Newton's Method: Optimization Techniques

Explores optimization methods, including unconstrained problems, linear programming, and heuristic approaches.

Explores the role of computation in data mathematics, focusing on iterative methods, optimization, estimators, and descent principles.

Newton's Method: Optimization & Indefiniteness

Covers Newton's Method for optimization and discusses the caveats of indefiniteness in optimization problems.

Convex Optimization: Gradient Algorithms

Covers convex optimization problems and gradient-based algorithms to find the global minimum.

Gradient Descent

Covers the concept of gradient descent in scalar cases, focusing on finding the minimum of a function by iteratively moving in the direction of the negative gradient.

Optimization Methods in Machine Learning

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Optimization without Constraints: Gradient Method

Covers optimization without constraints using the gradient method to find the function's minimum.

Quasi-Newton Methods

Introduces Quasi-Newton methods for optimization, explaining their advantages over traditional approaches like Gradient Descent and Newton's Method.

Gradient Descent Methods: Theory and Computation

Explores gradient descent methods for smooth convex and non-convex problems, covering iterative strategies, convergence rates, and challenges in optimization.

Optimality of Convergence Rates: Accelerated Gradient Descent

Explores the optimality of convergence rates in convex optimization, focusing on accelerated gradient descent and adaptive methods.

Gradient Descent: Principles and Applications

Covers gradient descent, its principles, applications, and convergence rates in optimization for machine learning.

Faster Gradient Descent: Projected Optimization Techniques

Covers faster gradient descent methods and projected gradient descent for constrained optimization in machine learning.

Coordinate Descent: Efficient Optimization Techniques

Covers coordinate descent, a method for optimizing functions by updating one coordinate at a time.

Optimization Basics: Unconstrained Optimization and Gradient Descent

Covers optimization basics, including unconstrained optimization and gradient descent methods for finding optimal solutions.

TR global convergence (end) + CG

Covers the trust-region method and introduces the truncated conjugate gradients method.

Optimization Techniques: Gradient Method Overview

Discusses the gradient method for optimization, focusing on its application in machine learning and the conditions for convergence.

Optimization Techniques: Stochastic Gradient Descent and Beyond

Discusses optimization techniques in machine learning, focusing on stochastic gradient descent and its applications in constrained and non-convex problems.

Nonlinear Optimization

Covers line search, Newton's method, BFGS, and conjugate gradient in nonlinear optimization.

Stochastic Optimization: Algorithms and Methods

Explores stochastic optimization algorithms and methods for convex problems with smooth and nonsmooth risks.

Stochastic Gradient Descent: Non-convex Optimization Techniques

Discusses Stochastic Gradient Descent and its application in non-convex optimization, focusing on convergence rates and challenges in machine learning.