Lectures related to Stochastic gradient descent

Explores adaptive gradient methods and their impact on optimization scenarios, including AdaGrad, ADAM, and RMSprop.

Optimization: Gradient Descent and Subgradients

Explores optimization methods like gradient descent and subgradients for training machine learning models, including advanced techniques like Adam optimization.

Stochastic Optimization: Algorithms and Methods

Explores stochastic optimization algorithms and methods for convex problems with smooth and nonsmooth risks.

Stochastic Gradient Descent: Non-convex Optimization Techniques

Discusses Stochastic Gradient Descent and its application in non-convex optimization, focusing on convergence rates and challenges in machine learning.

Neural Networks: Training and Optimization

Explores neural network training, optimization, and environmental considerations, with insights into PCA and K-means clustering.

Optimization Techniques: Stochastic Gradient Descent and Beyond

Discusses optimization techniques in machine learning, focusing on stochastic gradient descent and its applications in constrained and non-convex problems.

Optimization in Machine Learning: Gradient Descent

Covers optimization in machine learning, focusing on gradient descent for linear and logistic regression, stochastic gradient descent, and practical considerations.

Neural Networks: Training and Optimization

Explores the training and optimization of neural networks, addressing challenges like non-convex loss functions and local minima.

Coordinate Descent: Optimization Strategies

Explores coordinate descent optimization strategies, emphasizing simplicity in optimization through one-coordinate updates and discussing the implications of different approaches.

Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent

Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.

Gradient Descent: Optimization Techniques

Explores gradient descent, loss functions, and optimization techniques in neural network training.

Multilayer Perceptron: Training and Optimization

Explores the multilayer perceptron model, training, optimization, data preprocessing, activation functions, backpropagation, and regularization.

Gradient Descent: Linear Regression

Covers the concept of gradient descent for linear regression, explaining the iterative process of updating parameters.

Adaptive Optimization Methods: Theory and Applications

Explores adaptive optimization methods that adapt locally and converge without knowing the smoothness constant.

Structures in Non-Convex Optimization

Explores non-convex optimization in deep learning, covering critical points, SGD convergence, saddle points, and adaptive gradient methods.

Deep Learning: Data Representation and Multilayer Perceptron

Covers data representation, MLP training, activation functions, and gradient-based learning in deep neural networks.

Optimization Methods: RMSprop and ADAM

Explores RMSprop and ADAM optimization methods in Artificial Neural Networks, focusing on error functions, momentum, and signal-to-noise ratio.

Introduction to Learning by Stochastic Gradient Descent: Simple Perceptron

Covers the derivation of the stochastic gradient descent formula for a simple perceptron and explores the geometric interpretation of classification.

Gradient Noise: Smooth and Nonsmooth Risks

Explores the impact of gradient noise on optimization algorithms, focusing on smooth and nonsmooth risk functions and the derivation of gradient noise moments.

Convolutional Neural Networks

Covers Convolutional Neural Networks, including layers, training strategies, standard architectures, tasks like semantic segmentation, and deep learning tricks.