Course

CS-526: Learning theory

Lectures in this course (15)

PAC learning framework

Covers the PAC learning framework, including domain, feature set, classifier, and empirical risk.

Uniform convergence and No-Free-Lunch theorem

Explores uniform convergence, loss functions, and the No-Free-Lunch theorem in machine learning.

Tensor Decomposition: Jennrich's Theorem

Covers tensor decomposition and Jennrich's theorem, focusing on tensor rank and the uniqueness of tensor decomposition.

Matricizations and Alternating Least Squares

Covers matricizations, ALS method, and tensor decomposition, emphasizing the alignment of fibers in matrices.

Growth rate and uniform convergence

Explores growth rate, uniform convergence, PAC learning, and distribution learning challenges.

VC dimension

Covers the concept of VC dimension and its implications on PAC learning.

Tucker Decomposition: Multilinear rank and applications in data compression

Covers the Tucker decomposition and its applications in data compression, explaining the notion of multilinear rank and the HOSVD method.

Nonuniform Learnability and Structural Risk Minimization

Explores nonuniform learnability, PAC theorems, and structural risk minimization in machine learning.

Power Method and Applications

Explores the power method and its applications in topic models and document analysis, emphasizing iterative processes and probability distribution reconstruction.

Convex Optimization: Gradient Descent

Explores VC dimension, gradient descent, convex sets, and Lipschitz functions in convex optimization.

Stochastic Gradient Descent

Explores stochastic gradient descent optimization and the Mean-Field Method in neural networks.

SGD and Mean Field Analysis

Explores Stochastic Gradient Descent and Mean Field Analysis in two-layer neural networks, emphasizing their iterative processes and mathematical foundations.

Neural networks under SGD

Explores neural networks trained under Stochastic Gradient Descent, discussing hidden layers, square loss function, and particle evolution.

Neural networks under SGD

Explores the optimization of neural networks using Stochastic Gradient Descent (SGD) and the concept of dual risk versus empirical risk.

Tensors: Motivations and Introduction

Covers the basics of tensors, including their definition, properties, and decomposition, starting with a motivating example involving Gaussian distributions.