Concept

Vanishing gradient problem

Related lectures (31)

Introduces feed-forward networks, covering neural network structure, training, activation functions, and optimization, with applications in forecasting and finance.

Multilayer Perceptron: Training and Optimization

Explores the multilayer perceptron model, training, optimization, data preprocessing, activation functions, backpropagation, and regularization.

Recurrent Neural Networks: Neural Language Models

Explores fixed-context neural language models, RNNs, vanishing gradients, and sequence labeling in NLP.

Language Modelling and Recurrent Neural Networks

Explores language modelling, RNNs, n-gram models, LSTMs, and bidirectional RNNs.

Deep Learning: Backward Propagation and Vanishing Gradient

Delves into backward propagation in deep learning, addressing the vanishing gradient challenge and the need for effective hidden units.

Long Short-Term Memory Networks

Introduces Long Short-Term Memory (LSTM) networks as a solution to vanishing and exploding gradients in recurrent neural networks.

Recurrent Neural Networks: LSTMs & GRUs

Explores LSTM and GRU variants in recurrent neural networks, addressing challenges and advantages.

Recurrent Neural Networks: Training and Challenges

Discusses recurrent neural networks, their training challenges, and solutions like LSTMs and GRUs.

Vanishing Gradient Problem: Deep Learning

Discusses the vanishing gradient problem in deep neural networks and its solutions.

Batch Normalization: Why It Works

Explores the aim and process of batch normalization in deep neural networks, emphasizing its importance in stabilizing mean input and solving the vanishing gradient problem.

Machine Translation: Attention Mechanism

Explores the attention mechanism in machine translation, addressing the bottleneck problem and improving NMT performance significantly.

Deep Learning

Covers the fundamentals of deep learning, including data representations, bag of words, data pre-processing, artificial neural networks, and convolutional neural networks.

Deep Learning for NLP

Explores deep learning for NLP, covering word embeddings, context representations, learning techniques, and challenges like vanishing gradients and ethical considerations.

Neural Networks: Training and Optimization

Explores neural network training, optimization, and environmental considerations, with insights into PCA and K-means clustering.

Sequence to Sequence Models: Overview and Applications

Covers sequence to sequence models, their architecture, applications, and the role of attention mechanisms in improving performance.

Deep Learning Fundamentals

Introduces deep learning, from logistic regression to neural networks, emphasizing the need for handling non-linearly separable data.

Deep Learning: Multilayer Perceptron and Training

Covers deep learning fundamentals, focusing on multilayer perceptrons and their training processes.

Stochastic Models for Communications

Covers stochastic models for communication systems and their impact on system performance.

NFNets: Removing BatchNorm for High-Performance Image Recognition

Explores NFNets as an alternative to BatchNorm in ResNets, achieving high performance on ImageNet.

Weight Update: Mean Input and Bias Problem

Discusses the mean input shift and bias problem in weight updates for neural networks, highlighting the importance of correct initialization to prevent gradient issues.