Lecture

Subquadratic Attention Mechanisms: State Space Models Overview

Related lectures (26)

Covers state space models and their expressivity compared to transformers, focusing on attention mechanisms and computational efficiency.

Covers state space models and the expressivity of transformers in sequence copying tasks.

Explores practical aspects of solving parity games, including winning strategies, algorithms, complexity, determinism, and heuristic approaches.

Covers classical and quantum computational complexity concepts and implications.

Explores generalization in deep learning, covering model complexity, implicit bias, and the double descent phenomenon.

Explores the history and efficiency of the fast Fourier transform algorithm, demonstrating its practical applications and computational advantages.

Explores the Fast Fourier Transform algorithm for efficient signal processing applications, including dividing signals and analyzing complexity.

Explores dynamic programming for the Knapsack problem, discussing strategies, algorithms, NP-hardness, and time complexity analysis.

Covers optimization algorithms, convergence properties, and time complexity of sequences and functions.

Explores deep and convolutional networks, covering generalization, optimization, and practical applications in machine learning.

Explores algorithmic complexity, visualization of functions, and algorithm efficiency analysis using Python.

Explores critical phenomena in complex systems, including stochastic objects, percolation, and combinatorial optimization.

Explains the Fast Fourier Transform, a fast algorithm for calculating the Discrete Fourier Transform.

Explores double descent curves and overparametrization in machine learning models, highlighting the risks and benefits.

Explores linear programming optimization with constraints, Dijkstra's algorithm, and LP formulations for finding feasible solutions.

Explores complexity classes P and NP, highlighting solvable and verifiable problems, including NP-complete challenges.

Delves into the trade-off between model complexity and risk, generalization bounds, and the dangers of overfitting complex function classes.

Covers dynamic programming techniques for solving the rod cutting and matrix chain multiplication problems.

Covers the analysis of algorithm efficiency and optimizing code for faster execution.

Explores algorithm complexity, analyzing efficiency and worst-case scenarios of sorting algorithms.