From Attention to Transformers

About
Privacy
Disclaimer

Graph Chatbot

Related lectures (31)

Contextual Representations: ELMO and BERT Overview

Covers contextual representations in NLP, focusing on ELMO and BERT architectures and their applications in various tasks.

Pretraining: Transformers & Models

Explores pretraining models like BERT, T5, and GPT, discussing their training objectives and applications in natural language processing.

Pre-Training: BiLSTM and Transformer

Delves into pre-training BiLSTM and Transformer models for NLP tasks, showcasing their effectiveness and applications.

Neural Networks for NLP

Covers modern Neural Network approaches to NLP, focusing on word embeddings, Neural Networks for NLP tasks, and future Transfer Learning techniques.

Cognitive Maps in Rats and Men

Explores cognitive maps, reward systems, latent learning, attention mechanisms, and transformers in visual intelligence and machine learning.

Neural Networks: Two Layers Neural Network

Covers the basics of neural networks, focusing on the development from two layers neural networks to deep neural networks.

Recurrent Neural Networks: Training and Challenges

Discusses recurrent neural networks, their training challenges, and solutions like LSTMs and GRUs.

Graph-to-Graph Transformers: Syntax-aware Graph Encoding

Introduces the Syntax-aware Graph-to-Graph Transformer architecture for effective conditioning on syntactic dependency graphs.

Transformer Architecture: Subquadratic Attention Mechanisms

Covers transformer architecture, focusing on encoder-decoder models and subquadratic attention mechanisms for efficient processing of input sequences.

Chemical Reactions: Transformer Architecture

Explores atom mapping in chemical reactions and the transition to reaction grammar using the transformer architecture.

Pretraining Sequence-to-Sequence Models: BART and T5

Covers the pretraining of sequence-to-sequence models, focusing on BART and T5 architectures.

Page 2 of 2