Concept

Attention (machine learning)

Related lectures (31)

Explores the evolution from attention mechanisms to transformers in modern NLP, emphasizing the significance of self-attention and cross-attention.

Generative Models: Self-Attention and Transformers

Covers generative models with a focus on self-attention and transformers, discussing sampling methods and empirical means.

Seq2Seq Models: Attention vs. No Attention

Explores Seq2Seq models with and without attention mechanisms, covering encoder-decoder architecture, context vectors, decoding processes, and different types of attention mechanisms.

Transformers in Vision

Explores the evolution of visual intelligence models, focusing on Transformers and their applications in computer vision and natural language processing.

Transformers: Pretraining and Decoding Techniques

Covers advanced transformer concepts, focusing on pretraining and decoding techniques in NLP.

Transformer Architecture: The X Gomega

Delves into the Transformer architecture, self-attention, and training strategies for machine translation and image recognition.

Foundations of Deep Learning: Transformer Architecture Overview

Covers the foundational concepts of deep learning and the Transformer architecture, focusing on neural networks, attention mechanisms, and their applications in sequence modeling tasks.

Transformers: Self-Attention and MLP

Explores transformers, emphasizing self-attention and MLP mechanisms for efficient sequence processing.

Cognitive Maps in Rats and Men

Explores cognitive maps, reward systems, latent learning, attention mechanisms, and transformers in visual intelligence and machine learning.

Transformer: Pre-Training

Explores the Transformer model, from recurrent models to attention-based NLP, highlighting its key components and significant results in machine translation and document generation.

Sequence to Sequence Models: Overview and Attention Mechanisms

Explores sequence to sequence models, attention mechanisms, and their role in addressing model limitations and improving interpretability.

Deep Learning for NLP

Explores deep learning for NLP, covering word embeddings, context representations, learning techniques, and challenges like vanishing gradients and ethical considerations.

Transformers: Overview and Self-Attention

Provides an overview of Transformers, self-attention, multi-headed attention, and the Transformer decoder and encoder.

Machine Translation: Attention Mechanism

Explores the attention mechanism in machine translation, addressing the bottleneck problem and improving NMT performance significantly.

Neural Networks: Perceptron Model and Backpropagation Algorithm

Covers the perceptron model and backpropagation algorithm in neural networks.

Sequence to Sequence Models: Overview and Applications

Covers sequence to sequence models, their architecture, applications, and the role of attention mechanisms in improving performance.

Model Compression: Techniques for Efficient NLP Models

Explores model compression techniques in NLP, discussing pruning, quantization, weight factorization, knowledge distillation, and attention mechanisms.

Transformers in Visual Intelligence

Explores transformers in visual intelligence, focusing on object detection, image synthesis, and feature fusion.

Transformers: Unifying Machine Learning Communities

Covers the role of Transformers in unifying various machine learning fields.

Machine Learning in Human Rights: HURIDOCS

Explores machine learning in human rights, focusing on defining goals, handling false positives and negatives, and ensuring transparency and trust.