Lectures related to Shrinkage (statistics)

Estimation, Shrinkage and Penalization

Covers estimation, shrinkage, and penalization in statistics for data science, emphasizing the importance of balancing bias and variance in model estimation.

Bias-Variance Tradeoff in Ridge Estimation

Explores the bias-variance tradeoff in ridge estimation, showcasing how a bit of bias can enhance mean squared error by reducing variance.

Shrinkage Estimation of Large Covariance Matrices

Explores shrinkage estimation of high-dimensional covariance matrices, comparing linear and nonlinear approaches for improved accuracy.

Ridge Regression: Penalised Least Squares

Explores Ridge Regression for handling multicollinearity and the LASSO method for model selection.

The Stein Phenomenon and Superefficiency

Explores the Stein Phenomenon, showcasing the benefits of bias in high-dimensional statistics and the superiority of the James-Stein Estimator over the Maximum Likelihood Estimator.

Functional Linear Regression: Sparse Estimation and Adaptive Methods

By Angelina Roche covers adaptive and sparse estimation in functional linear regression models.

Optimal Estimation and Bias in Statistics

Delves into optimal estimation, bias role in finite samples, and the delicate tradeoff between bias and variance in statistical estimation.

Maximum Likelihood Estimation

Covers Maximum Likelihood Estimation, focusing on ML Estimation-Distribution, Shrinkage Estimation, and Loss functions.

Penalization in Ridge Regression

Covers penalization in ridge regression, emphasizing the trade-off between bias and variance in regression models.

Model Selection and Evaluation: Bias-Variance Dilemma, Ridge Estimation

Explores over-learning, generalization, and under-learning in machine learning models.

Understanding Generalization: Implicit Bias & Optimization

Explores the trade-off between complexity and risk in machine learning models, the benefits of overparametrization, and the implicit bias of optimization algorithms.

Adaptive Gradient Methods: Theory and Applications

Explores adaptive gradient methods, their properties, convergence, and comparison with traditional optimization algorithms.

SVMs and Feature Maps

Explores SVMs, feature maps, and the importance of finding the maximum margin solution for classification problems.

Data Representations & Processing

Explores data representations, overfitting, model selection, cross-validation, and imbalanced data challenges.

Gradient Descent: Optimization and Regularization

Covers the algorithm of gradient descent for optimization, including early stopping and regularization techniques.

Maximum Likelihood Estimation

Explores Maximum Likelihood Estimation, covering assumptions, properties, distribution, shrinkage estimation, and loss functions.

Data Representations and Processing

Discusses overfitting, model selection, cross-validation, regularization, data representations, and handling imbalanced data in machine learning.

Mathematics of Data: Models and Estimators

Covers the Mathematics of Data, focusing on models, estimators, and practical issues in data analysis.

Statistical Estimation: Gaussian Linear Model

Delves into statistical estimation, highlighting the Gaussian linear model and the limitations of ML estimators.

Generalized Linear Models: Theory and Applications

Covers the theory and applications of Generalized Linear Models, including MLE, measures of fit, shrinkage, and special examples.