Lectures related to Stochastic Gradient Descent: Optimization Techniques

From Stochastic Gradient Descent to Non-Smooth Optimization

Covers stochastic optimization, sparsity, and non-smooth minimization via subgradient descent.

Stochastic Gradient Descent: Optimization Techniques

Explores Stochastic Gradient Descent with Averaging, comparing it with Gradient Descent, and discusses challenges in non-convex optimization and sparse recovery techniques.

Optimization Methods in Machine Learning

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent

Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.

Stochastic Gradient Descent: Non-convex Optimization Techniques

Discusses Stochastic Gradient Descent and its application in non-convex optimization, focusing on convergence rates and challenges in machine learning.

Stochastic Optimization: Algorithms and Methods

Explores stochastic optimization algorithms and methods for convex problems with smooth and nonsmooth risks.

Optimization Techniques: Stochastic Gradient Descent and Beyond

Discusses optimization techniques in machine learning, focusing on stochastic gradient descent and its applications in constrained and non-convex problems.

LASSO Regression: Sparse Signal Induction

Explores LASSO regression for inducing sparsity in signals through gradient descent.

Optimization: Gradient Descent and Subgradients

Explores optimization methods like gradient descent and subgradients for training machine learning models, including advanced techniques like Adam optimization.

Adaptive Gradient Methods: Part 1

Explores adaptive gradient methods and their impact on optimization scenarios, including AdaGrad, ADAM, and RMSprop.

Structures in Non-Convex Optimization

Explores non-convex optimization in deep learning, covering critical points, SGD convergence, saddle points, and adaptive gradient methods.

Structures in Non-Convex Optimization

Covers non-convex optimization, deep learning training problems, stochastic gradient descent, adaptive methods, and neural network architectures.

Optimality of Convergence Rate: Acceleration in Gradient Descent

Explores the optimality of convergence rate in gradient descent and acceleration techniques for convex and non-convex problems.

Optimization Techniques: Convexity in Machine Learning

Covers optimization techniques in machine learning, focusing on convexity and its implications for efficient problem-solving.

Coordinate Descent: Efficient Optimization Techniques

Covers coordinate descent, a method for optimizing functions by updating one coordinate at a time.

Structures in Non-Convex Optimization

Delves into structures in non-convex optimization, emphasizing scalable optimization for deep learning.

Proximal Gradient Descent: Optimization Techniques in Machine Learning

Discusses proximal gradient descent and its applications in optimizing machine learning algorithms.

Gradient Descent

Covers the concept of gradient descent in scalar cases, focusing on finding the minimum of a function by iteratively moving in the direction of the negative gradient.

Coordinate Descent: Optimization Strategies

Explores coordinate descent optimization strategies, emphasizing simplicity in optimization through one-coordinate updates and discussing the implications of different approaches.

Optimization Techniques: Convexity and Algorithms in Machine Learning

Covers optimization techniques in machine learning, focusing on convexity, algorithms, and their applications in ensuring efficient convergence to global minima.