Lectures related to Optimization Basics: Norms, Convexity, Differentiability

Mathematics of Data: Optimization Basics

Covers optimization basics, including metrics, norms, convexity, gradients, and logistic regression, with a focus on strong convexity and convergence rates.

Convex Functions

Covers the properties and operations of convex functions.

Optimization Techniques: Gradient Descent and Convex Functions

Provides an overview of optimization techniques, focusing on gradient descent and properties of convex functions in machine learning.

Mathematics of Data: Optimization Basics

Covers basics on optimization, including norms, Lipschitz continuity, and convexity concepts.

Proximal and Subgradient Descent: Optimization Techniques

Discusses proximal and subgradient descent methods for optimization in machine learning.

Convexity and Jacobians

Explores convexity, Jacobians, subdifferentials, and convergence rates in optimization and function analysis.

Proximal Gradient Descent: Optimization Techniques in Machine Learning

Discusses proximal gradient descent and its applications in optimizing machine learning algorithms.

Faster and Projected Gradient Descent: Optimization Techniques

Discusses advanced optimization techniques, focusing on faster and projected gradient descent methods in machine learning.

Gradient Descent Methods: Theory and Computation

Explores gradient descent methods for smooth convex and non-convex problems, covering iterative strategies, convergence rates, and challenges in optimization.

Convex Optimization: Gradient Descent

Explores VC dimension, gradient descent, convex sets, and Lipschitz functions in convex optimization.

Convex Optimization: Convex Functions

Covers the concept of convex functions and their applications in optimization problems.

Subgradients and Convex Functions

Explores subgradients in convex functions, emphasizing non-differentiable yet convex scenarios and properties of subdifferentials.

Convex Optimization: Examples of Convex Functions

Explores convex optimization, convex functions, and their properties, including strict convexity and strong convexity, as well as different types of convex functions like linear affine functions and norms.

Optimal Transport: Rockafellar Theorem

Explores the Rockafellar Theorem in optimal transport, focusing on c-cyclical monotonicity and convex functions.

Convexity: Functions and Global Minima

Explores convex functions, global minima, and their relationship with differentiability.

Faster Gradient Descent: Projected Optimization Techniques

Covers faster gradient descent methods and projected gradient descent for constrained optimization in machine learning.

Gradient Descent Methods

Covers gradient descent methods for convex and nonconvex problems, including smooth unconstrained convex minimization, maximum likelihood estimation, and examples like ridge regression and image classification.

Gradient Descent: Principles and Applications

Covers gradient descent, its principles, applications, and convergence rates in optimization for machine learning.

Optimization Basics

Introduces optimization basics, covering logistic regression, derivatives, convex functions, gradient descent, and second-order methods.

Optimization Methods: Convergence and Trade-offs

Covers optimization methods, convergence guarantees, trade-offs, and variance reduction techniques in numerical optimization.