Lectures related to Gradient Noise: Smooth and Nonsmooth Risks

Optimization Techniques: Stochastic Gradient Descent and Beyond

Discusses optimization techniques in machine learning, focusing on stochastic gradient descent and its applications in constrained and non-convex problems.

Stochastic Optimization: Algorithms and Methods

Explores stochastic optimization algorithms and methods for convex problems with smooth and nonsmooth risks.

Implicit Bias in Machine Learning

Explores implicit bias, gradient descent, stability in optimization algorithms, and generalization bounds in machine learning.

Optimization Techniques: Convexity in Machine Learning

Covers optimization techniques in machine learning, focusing on convexity and its implications for efficient problem-solving.

Stochastic Gradient Descent: Non-convex Optimization Techniques

Discusses Stochastic Gradient Descent and its application in non-convex optimization, focusing on convergence rates and challenges in machine learning.

Optimization Techniques: Convexity and Algorithms in Machine Learning

Covers optimization techniques in machine learning, focusing on convexity, algorithms, and their applications in ensuring efficient convergence to global minima.

Optimization Methods in Machine Learning

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Adaptive Gradient Methods: Part 1

Explores adaptive gradient methods and their impact on optimization scenarios, including AdaGrad, ADAM, and RMSprop.

Structures in Non-Convex Optimization

Covers non-convex optimization, deep learning training problems, stochastic gradient descent, adaptive methods, and neural network architectures.

Optimization: Gradient Descent and Subgradients

Explores optimization methods like gradient descent and subgradients for training machine learning models, including advanced techniques like Adam optimization.

Proximal Gradient Descent: Optimization Techniques in Machine Learning

Discusses proximal gradient descent and its applications in optimizing machine learning algorithms.

Neural Networks: Training and Optimization

Explores the training and optimization of neural networks, addressing challenges like non-convex loss functions and local minima.

Optimizing Loss Functions: Gradient Descent Variants

Explores early stopping in gradient descent and variants for faster optimization.

Adaptive Optimization Methods: Theory and Applications

Explores adaptive optimization methods that adapt locally and converge without knowing the smoothness constant.

Deep and Convolutional Networks: Generalization and Optimization

Explores deep and convolutional networks, covering generalization, optimization, and practical applications in machine learning.

Variance Reduction in Deep Learning

Explores variance reduction techniques in deep learning, covering gradient descent, stochastic gradient descent, SVRG method, and performance comparison of algorithms.

Faster Gradient Descent: Projected Optimization Techniques

Covers faster gradient descent methods and projected gradient descent for constrained optimization in machine learning.

Feed-forward Networks

Introduces feed-forward networks, covering neural network structure, training, activation functions, and optimization, with applications in forecasting and finance.

Double Descent Curves: Overparametrization

Explores double descent curves and overparametrization in machine learning models, highlighting the risks and benefits.

Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent

Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.