Lecture

Convergence Analysis: Stochastic Gradient Algorithms

Related lectures (29)

Stochastic Optimization: Algorithms and Methods

Explores stochastic optimization algorithms and methods for convex problems with smooth and nonsmooth risks.

Explicit Stabilised Methods: Applications to Bayesian Inverse Problems

Explores explicit stabilised Runge-Kutta methods and their application to Bayesian inverse problems, covering optimization, sampling, and numerical experiments.

Multilayer Neural Networks: Deep Learning

Covers the fundamentals of multilayer neural networks and deep learning.

Optimization: Gradient Descent and Subgradients

Explores optimization methods like gradient descent and subgradients for training machine learning models, including advanced techniques like Adam optimization.

Pizza Making Process

Covers the process of making pizza, sampling, averages, dispersion, residuals, and normal distribution.

Neural networks under SGD

Explores neural networks trained under Stochastic Gradient Descent, discussing hidden layers, square loss function, and particle evolution.

Theory of MCMC

Covers the theory of Markov Chain Monte Carlo (MCMC) sampling and discusses convergence conditions, transition matrix choice, and target distribution evolution.

Normal Distribution: Characteristics and Examples

Covers the characteristics and importance of the normal distribution, including examples and treatment scenarios.

Neural Networks: Training and Optimization

Explores neural network training, optimization, and environmental considerations, with insights into PCA and K-means clustering.

Optimization in Machine Learning: Gradient Descent

Covers optimization in machine learning, focusing on gradient descent for linear and logistic regression, stochastic gradient descent, and practical considerations.

Monte Carlo: Optimization and Estimation

Explores optimization and estimation in Monte Carlo methods, emphasizing Bayes-optimal groups and estimators.

Data Representations and Processing in Machine Learning

Covers data representations and processing techniques essential for effective machine learning algorithms.

Review Session: Module 1

Introduces inferential statistics, covering sampling, central tendency, dispersion, histograms, z-scores, and the normal distribution.

Neural Networks: Training and Optimization

Explores the training and optimization of neural networks, addressing challenges like non-convex loss functions and local minima.

Untitled

Linear Models: Basics

Introduces linear models in machine learning, covering basics, parametric models, multi-output regression, and evaluation metrics.

Hypothesis Testing and Confidence Intervals: Key Concepts

Provides an overview of hypothesis testing and confidence intervals in statistics, including practical examples and key concepts.

Approximate Query Processing: BlinkDB

Introduces BlinkDB, a framework for approximate query processing using sampling techniques.

Natural Language Generation: Decoding & Training

Explores challenges in natural language generation, decoding algorithms, training issues, and reward functions.

Generative Models: Self-Attention and Transformers

Covers generative models with a focus on self-attention and transformers, discussing sampling methods and empirical means.