Lecture

Contextual Representations: ELMo & BERT

Related lectures (29)

Contextual Representations: ELMO and BERT Overview

Covers contextual representations in NLP, focusing on ELMO and BERT architectures and their applications in various tasks.

Pre-Training: BiLSTM and Transformer

Delves into pre-training BiLSTM and Transformer models for NLP tasks, showcasing their effectiveness and applications.

Deep Learning for NLP

Explores deep learning for NLP, covering word embeddings, context representations, learning techniques, and challenges like vanishing gradients and ethical considerations.

Deep Learning for NLP

Introduces deep learning concepts for NLP, covering word embeddings, RNNs, and Transformers, emphasizing self-attention and multi-headed attention.

Pretraining: Transformers & Models

Explores pretraining models like BERT, T5, and GPT, discussing their training objectives and applications in natural language processing.

Language Models: From Theory to Computation

Explores the mathematics of language models, covering architecture design, pre-training, and fine-tuning, emphasizing the importance of pre-training and fine-tuning for various tasks.

Neural Networks for NLP

Covers modern Neural Network approaches to NLP, focusing on word embeddings, Neural Networks for NLP tasks, and future Transfer Learning techniques.

Natural Language Processing: Understanding Transformers and Tokenization

Provides an overview of Natural Language Processing, focusing on transformers, tokenization, and self-attention mechanisms for effective language analysis and synthesis.

Transformers: Pretraining and Decoding Techniques

Covers advanced transformer concepts, focusing on pretraining and decoding techniques in NLP.

Sequence to Sequence Models: Overview and Applications

Covers sequence to sequence models, their architecture, applications, and the role of attention mechanisms in improving performance.

Deep Learning for NLP

Delves into Deep Learning for Natural Language Processing, exploring Neural Word Embeddings, Recurrent Neural Networks, and Attentive Neural Modeling with Transformers.

Binary Sentiment Classifier Training

Covers the training of a binary sentiment classifier using an RNN.

Pretraining Sequence-to-Sequence Models: BART and T5

Covers the pretraining of sequence-to-sequence models, focusing on BART and T5 architectures.

Recurrent Neural Networks: Training and Challenges

Discusses recurrent neural networks, their training challenges, and solutions like LSTMs and GRUs.

Seq2Seq Models: Attention vs. No Attention

Explores Seq2Seq models with and without attention mechanisms, covering encoder-decoder architecture, context vectors, decoding processes, and different types of attention mechanisms.

Model Analysis

Explores neural model analysis in NLP, covering evaluation, probing, and ablation studies to understand model behavior and interpretability.

Language Models: Fixed-context and Recurrent Neural Networks

Discusses language models, focusing on fixed-context neural models and recurrent neural networks.

Long Short-Term Memory Networks

Introduces Long Short-Term Memory (LSTM) networks as a solution to vanishing and exploding gradients in recurrent neural networks.

Transformers: Revolutionizing Attention Mechanisms in NLP

Covers the development of transformers and their impact on attention mechanisms in NLP.

Neural Word Embeddings: Learning Representations for Natural Language

Covers neural word embeddings and methods for learning word representations in natural language processing.