Lectures related to Gradient Descent: Optimization

Optimization Basics

Introduces optimization basics, covering logistic regression, derivatives, convex functions, gradient descent, and second-order methods.

Convergence Criteria: Necessary Conditions

Explains necessary conditions for convergence in optimization problems.

Optimization Methods in Machine Learning

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Deep Learning Building Blocks

Covers tensors, loss functions, autograd, and convolutional layers in deep learning.

Proximal Gradient Descent: Optimization Techniques in Machine Learning

Discusses proximal gradient descent and its applications in optimizing machine learning algorithms.

Coordinate Descent: Optimization Strategies

Explores coordinate descent optimization strategies, emphasizing simplicity in optimization through one-coordinate updates and discussing the implications of different approaches.

Stochastic Gradient Descent: Non-convex Optimization Techniques

Discusses Stochastic Gradient Descent and its application in non-convex optimization, focusing on convergence rates and challenges in machine learning.

Proximal and Subgradient Descent: Optimization Techniques

Discusses proximal and subgradient descent methods for optimization in machine learning.

Optimization Techniques: Gradient Descent and Convex Functions

Provides an overview of optimization techniques, focusing on gradient descent and properties of convex functions in machine learning.

Gradient Descent: Optimization Techniques

Explores gradient descent, loss functions, and optimization techniques in neural network training.

Linear and Logistic Regression

Introduces linear and logistic regression, covering parametric models, multi-output prediction, non-linearity, gradient descent, and classification applications.

Optimization: Gradient Descent and Subgradients

Explores optimization methods like gradient descent and subgradients for training machine learning models, including advanced techniques like Adam optimization.

Gradient Descent: Principles and Applications

Covers gradient descent, its principles, applications, and convergence rates in optimization for machine learning.

Faster and Projected Gradient Descent: Optimization Techniques

Discusses advanced optimization techniques, focusing on faster and projected gradient descent methods in machine learning.

Differentiability of Functions of Several Variables

Covers the differentiability of functions of multiple variables and the significance of directional derivatives and gradients.

Optimization Techniques: Convexity in Machine Learning

Covers optimization techniques in machine learning, focusing on convexity and its implications for efficient problem-solving.

Tangent to Graph of a Function

Explores finding the equation of the tangent line to a function's graph at a point.

Faster Gradient Descent: Projected Optimization Techniques

Covers faster gradient descent methods and projected gradient descent for constrained optimization in machine learning.

Convex Optimization: Gradient Flow

Explores convex optimization, emphasizing the importance of minimizing functions within a convex set and the significance of continuous processes in studying convergence rates.

Optimization Techniques: Stochastic Gradient Descent and Beyond

Discusses optimization techniques in machine learning, focusing on stochastic gradient descent and its applications in constrained and non-convex problems.