Lecture

Generalization in Deep Learning

Related lectures (32)

Double Descent Curves: Overparametrization

Explores double descent curves and overparametrization in machine learning models, highlighting the risks and benefits.

Delves into the trade-off between model complexity and risk, generalization bounds, and the dangers of overfitting complex function classes.

Implicit Bias in Machine Learning

Explores implicit bias, gradient descent, stability in optimization algorithms, and generalization bounds in machine learning.

Complexity: Approximation-Estimation Trade-off

Explores the control of complexity in hypothesis spaces and the trade-off between approximation and estimation in risk decomposition.

Learning with Deep Neural Networks

Explores the success and challenges of deep learning, including overfitting, generalization, and the impact on various domains.

Neural Networks: Two Layers Neural Network

Covers the basics of neural networks, focusing on the development from two layers neural networks to deep neural networks.

Gradient Descent on Two-Layer ReLU Neural Networks

Analyzes gradient descent on two-layer ReLU neural networks, exploring global convergence, regularization, implicit bias, and statistical efficiency.

Understanding Generalization: Implicit Bias & Optimization

Explores the trade-off between complexity and risk in machine learning models, the benefits of overparametrization, and the implicit bias of optimization algorithms.

Deep Learning: Theory and Practice

By Prof. Volkan Cevher delves into the mathematics of deep learning, exploring model complexity, risk trade-offs, and the generalization mystery.

Numerical analysis

Covers advanced numerical analysis topics including deep neural networks and optimization methods.

Multilayer Neural Networks: Deep Learning

Covers the fundamentals of multilayer neural networks and deep learning.

Overfitting: Symptoms and Characteristics

Explores overfitting in polynomial regression, emphasizing the importance of generalization in machine learning and statistics.

Deep Learning: Theory and Applications

Explores the mathematics of deep learning, neural networks, and their applications in computer vision tasks, addressing challenges and the need for robustness.

Variance Reduction Techniques

Covers variance reduction techniques in optimization, focusing on gradient descent and stochastic gradient descent methods.

The Hidden Convex Optimization Landscape of Deep Neural Networks

Explores the hidden convex optimization landscape of deep neural networks, showcasing the transition from non-convex to convex models.

Deep Learning Modus Operandi

Explores the benefits of deeper networks in deep learning and the importance of over-parameterization and generalization.

Deep Learning III

Delves into deep learning optimization, challenges, SGD variants, critical points, overparametrized networks, and adaptive methods.

State Space Models: Expressivity of Transformers

Covers state space models and the expressivity of transformers in sequence copying tasks.

Deep Learning Building Blocks: Linear Layers

Explains the fundamental building blocks of deep learning, focusing on linear layers and activation functions.

Universal Approximation Theorem: MLP

Covers Multi-Layer Perceptrons (MLP) and their application from classification to regression, including the Universal Approximation Theorem and challenges with gradients.