Concept

Automatic differentiation

Related lectures (15)

Covers the fundamental building blocks of deep learning, including tensors, backpropagation, and PyTorch.

Automatic Differentiation: BackProp revisited

Discusses automatic differentiation, emphasizing reverse mode differentiation for optimizing convolutional layer filters by gradient descent.

PyTorch and Convolutional Networks

Covers PyTorch tensor data structure and training a CNN to classify images.

Machine Learning on Earth System with Remote Sensing

Explores machine learning applications in Earth system analysis using remote sensing data, focusing on automatic image interpretation and explainable AI.

Vector Differentiation and Convex Functions

Covers the concepts of vector differentiation and convex functions.

BackProp Algorithm: Pseudocode and Processing Steps

Covers the BackProp algorithm, including initialization, signal propagation, error computation, weight updating, and complexity comparison with numerical differentiation.

Mathematics of Data: From Theory to Computation

Covers key concepts in data mathematics, including automatic differentiation, linear layers, and attention layers.

Quasi-Newton Methods

Introduces Quasi-Newton methods for optimization, explaining their advantages over traditional approaches like Gradient Descent and Newton's Method.

Recurrent Neural Networks: Neural Language Models

Explores fixed-context neural language models, RNNs, vanishing gradients, and sequence labeling in NLP.

Hands on with Manopt

Explores practical optimization using Manopt for manifolds, covering gradient checks, approximation errors, and Hessian computations.

Deep Learning Building Blocks

Covers tensors, loss functions, autograd, and convolutional layers in deep learning.

Vector Analysis: Scalar Fields

Covers the analysis of scalar fields, including divergence, gradient, and Laplacian.

Recurrent Neural Networks: Training and Challenges

Discusses recurrent neural networks, their training challenges, and solutions like LSTMs and GRUs.

Natural Language Generation: Evaluating Text Quality

Covers the evaluation methods for natural language generation systems, including metrics and human assessments of generated text quality.

State Space Models: Expressivity and Transformers

Covers state space models and their expressivity compared to transformers, focusing on attention mechanisms and computational efficiency.