Lectures related to Second-Order Model Compression

Provable Benefits of Overparameterization in Model Compression

Explores the provable benefits of overparameterization in model compression, emphasizing the efficiency of deep neural networks and the importance of retraining for improved performance.

Model Compression: Techniques for Efficient NLP Models

Explores model compression techniques in NLP, discussing pruning, quantization, weight factorization, knowledge distillation, and attention mechanisms.

Model Compression Techniques: Enhancing Neural Networks

Covers model compression techniques to enhance the efficiency of large language models in production settings.

Neural Networks: Training and Activation

Explores neural networks, activation functions, backpropagation, and PyTorch implementation.

Neural Signal Compression

Explores analog-to-digital conversion, neural signal optimization, multichannel architectures, and on-chip compression techniques in neuroengineering.

Dimensionality Reduction: PCA and Autoencoders

Introduces artificial neural networks, CNNs, and dimensionality reduction using PCA and autoencoders.

Neural Networks: Two Layers Neural Network

Covers the basics of neural networks, focusing on the development from two layers neural networks to deep neural networks.

Convolutional Neural Networks

Covers Convolutional Neural Networks, including layers, training strategies, standard architectures, tasks like semantic segmentation, and deep learning tricks.

Deep Learning Fundamentals

Introduces deep learning fundamentals, covering data representations, neural networks, and convolutional neural networks.

Deep Learning: Convolutional Neural Networks

Covers Convolutional Neural Networks, standard architectures, training techniques, and adversarial examples in deep learning.

Neural Networks for NLP

Covers modern Neural Network approaches to NLP, focusing on word embeddings, Neural Networks for NLP tasks, and future Transfer Learning techniques.

Auto-encoder and GANs

Covers auto-encoders for data compression and GANs for data generation.

Deep Learning: Convolutional Networks

Explores convolutional neural networks, backpropagation, and stochastic gradient descent in deep learning.

Deep Learning: Data Representations and Neural Networks

Covers data representations, Bag of Words, histograms, data pre-processing, and neural networks.

Optical Computing: All-Analog Photoelectronic Chip for Vision Tasks

Presents an all-analog photoelectronic chip for high-speed vision tasks, addressing challenges in classical computation and proposing a hybrid optical-electrical framework.

Deep Splines: Unifying Framework for Deep Neural Networks

Introduces a functional framework for deep neural networks with adaptive piecewise-linear splines, focusing on biomedical image reconstruction and the challenges of deep splines.

Neural Networks: Perceptron Model and Backpropagation Algorithm

Covers the perceptron model and backpropagation algorithm in neural networks.

Recurrent Neural Networks: Training and Challenges

Discusses recurrent neural networks, their training challenges, and solutions like LSTMs and GRUs.

Language Models: Fixed-context and Recurrent Neural Networks

Discusses language models, focusing on fixed-context neural models and recurrent neural networks.

Feed-forward Networks

Introduces feed-forward networks, covering neural network structure, training, activation functions, and optimization, with applications in forecasting and finance.