Course

EE-566: Adaptation and learning

Lectures in this course (28)

Vector Differentiation and Convex Functions

Covers the concepts of vector differentiation and convex functions.

Explores subgradients in convex functions, emphasizing non-differentiable yet convex scenarios and properties of subdifferentials.

Naive Bayes Classifier

Introduces the Naive Bayes classifier, covering independence assumptions, conditional probabilities, and applications in document classification and medical diagnosis.

Proximal Operator and Gradient Descent

Covers proximal operators, gradient descent, optimality conditions, and convergence analysis in optimization problems.

Gradient Descent: Proximal Operator and Step-Size Strategies

Explores proximal operator, gradient descent, and step-size strategies for risk function minimization.

Linear Discriminant Analysis: Generative Methods

Covers Linear Discriminant Analysis (LDA) as a generative method for classification.

Stochastic Optimization: Algorithms and Methods

Explores stochastic optimization algorithms and methods for convex problems with smooth and nonsmooth risks.

Principal Component Analysis: Dimensionality Reduction

Explores Principal Component Analysis for dimensionality reduction and unsupervised feature selection.

Stochastic Optimization and Adaptive Gradient Methods

Explores stochastic optimization, adaptive gradient methods, recommender systems, and matrix factorization in user-item rating matrices.

Logistic Regression: Part 1

Introduces logistic regression for binary classification and explores multiclass classification using OvA and OvO strategies.

Perceptron: Part 2

Covers the Perceptron algorithm and its application to binary classification problems, including the Pocket Perceptron algorithm.

Adaptive Gradient Methods: Part 1

Explores adaptive gradient methods and their impact on optimization scenarios, including AdaGrad, ADAM, and RMSprop.

Gradient Noise: Smooth and Nonsmooth Risks

Explores the impact of gradient noise on optimization algorithms, focusing on smooth and nonsmooth risk functions and the derivation of gradient noise moments.

Support Vector Machines: SVM Basics

Covers the basics of Support Vector Machines, focusing on hard-margin and soft-margin formulations.

Kernel Methods: Nonlinear Separation Surfaces

Explores kernel methods for nonlinear separation surfaces using polynomial and Gaussian kernels in Perceptron and SVM algorithms.

Convergence Analysis: Stochastic Gradient Algorithms

Explores the convergence analysis of stochastic gradient algorithms under various operational modes and step-size sequences.

Mean-Square-Error Inference

Covers the concept of mean-square-error inference and optimal estimators for inference problems using different design criteria.

Generalization Theory

Explores generalization theory in machine learning, addressing challenges in higher-dimensional spaces and the bias-variance tradeoff.

Bayesian Inference: Gaussian Variables

Explores Bayesian inference for Gaussian random variables, covering joint distribution, marginal pdfs, and the Bayes classifier.

Feedforward Neural Networks: Activation Functions and Backpropagation

Introduces feedforward neural networks, activation functions, and backpropagation for training, addressing challenges and powerful methods.