Lecture

Structures in Non-Convex Optimization

Related lectures (29)

The Hidden Convex Optimization Landscape of Deep Neural Networks

Explores the hidden convex optimization landscape of deep neural networks, showcasing the transition from non-convex to convex models.

Explores non-convex optimization in deep learning, covering critical points, SGD convergence, saddle points, and adaptive gradient methods.

Deep Learning III

Delves into deep learning optimization, challenges, SGD variants, critical points, overparametrized networks, and adaptive methods.

Structures in Non-Convex Optimization

Covers non-convex optimization, deep learning training problems, stochastic gradient descent, adaptive methods, and neural network architectures.

Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent

Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.

Non-Convex Optimization: Techniques and Applications

Covers non-convex optimization techniques and their applications in machine learning.

Perception: Data-Driven Approaches

Explores perception in deep learning for autonomous vehicles, covering image classification, optimization methods, and the role of representation in machine learning.

Neural Networks: Training and Optimization

Explores the training and optimization of neural networks, addressing challenges like non-convex loss functions and local minima.

Optimization Methods in Machine Learning

Explores optimization methods in machine learning, emphasizing gradients, costs, and computational efforts for efficient model training.

Implicit Bias in Machine Learning

Explores implicit bias, gradient descent, stability in optimization algorithms, and generalization bounds in machine learning.

Optimization Methods: Convergence and Trade-offs

Covers optimization methods, convergence guarantees, trade-offs, and variance reduction techniques in numerical optimization.

Multilayer Neural Networks: Deep Learning

Covers the fundamentals of multilayer neural networks and deep learning.

Variance Reduction in Deep Learning

Explores variance reduction techniques in deep learning, covering gradient descent, stochastic gradient descent, SVRG method, and performance comparison of algorithms.

Deep Learning: Data Representations and Neural Networks

Covers data representations, Bag of Words, histograms, data pre-processing, and neural networks.

Adversarial Machine Learning: Fundamentals and Techniques

Explores adversarial machine learning, covering the generation of adversarial examples, robustness challenges, and techniques like Fast Gradient Sign Method.

Double Descent Curves: Overparametrization

Explores double descent curves and overparametrization in machine learning models, highlighting the risks and benefits.

Generalization in Deep Learning

Explores generalization in deep learning, covering model complexity, implicit bias, and the double descent phenomenon.

Adaptive Optimization Methods: Theory and Applications

Explores adaptive optimization methods that adapt locally and converge without knowing the smoothness constant.

Deep Learning: Convolutional Neural Networks

Covers Convolutional Neural Networks, standard architectures, training techniques, and adversarial examples in deep learning.

Deep Learning Fundamentals

Introduces deep learning, from logistic regression to neural networks, emphasizing the need for handling non-linearly separable data.