Course

EE-556: Mathematics of data: from theory to computation

Lectures in this course (118)

Introduces proximal operators and conditional gradient methods for composite convex minimization problems in data optimization.

Untitled

Variance Reduction Techniques

Covers variance reduction techniques in optimization, focusing on gradient descent and stochastic gradient descent methods.

Parametric Models: Estimation and Optimization

Explores parametric models, estimation techniques, regression models, and score-based classifiers in data analysis.

Graphical Model Learning: M-Estimator Examples

Explores graphical model learning with M-estimators, Gaussian process regression, Google PageRank modeling, density estimation, and generalized linear models.

Generalization in Deep Learning

Explores generalization in deep learning, covering model complexity, implicit bias, and the double descent phenomenon.

Optimality of Convergence Rates: Accelerated/Stochastic Gradient Descent

Covers the optimality of convergence rates in accelerated and stochastic gradient descent methods for non-convex optimization problems.

Structures in Non-Convex Optimization

Covers non-convex optimization, deep learning training problems, stochastic gradient descent, adaptive methods, and neural network architectures.

Optimization Basics: Linear Algebra, Analysis, Convexity

Introduces optimization basics, covering linear algebra, analysis, and convexity principles.

Deep Learning Building Blocks

Covers the fundamental building blocks of deep learning, including tensors, backpropagation, and PyTorch.

Adversarial Machine Learning

Explores adversarial machine learning, generative adversarial networks, and the challenges of adversarial examples in data optimization.

Convexity and Jacobians

Explores convexity, Jacobians, subdifferentials, and convergence rates in optimization and function analysis.

Deep Learning Building Blocks

Covers tensors, loss functions, autograd, and convolutional layers in deep learning.

Diffusion Models and Robustness

Explores diffusion models, GAN training challenges, SDE-based generative models, and robustness in deep learning.

Concise Signal Models and Compressive Sensing

Explores concise signal models, compressive sensing, sparsity, atomic norms, and non-smooth minimization using subgradient descent.

Primal-dual optimization: Theory and Computation

Explores primal-dual optimization, conjugation of functions, strong duality, and quadratic penalty methods in data mathematics.

Primal-dual Optimization II

Explores primal-dual optimization methods, focusing on Lagrangian approaches and various methods like penalty, augmented Lagrangian, and splitting techniques.

Iterative Descent Methods: Optimization Principles

Explores iterative descent methods in optimization, gradient descent, local minima, and convergence principles.

Optimization Methods: Convexity and Gradient Descent

Explores optimization methods, including convexity, gradient descent, and non-convex minimization, with examples like maximum likelihood estimation and ridge regression.

Language Models: From Theory to Computation

Explores the mathematics of language models, covering architecture design, pre-training, and fine-tuning, emphasizing the importance of pre-training and fine-tuning for various tasks.