Course

COM-516: Markov chains and algorithmic applications

Lectures in this course (34)

Markov Chains: Definition and Examples

Covers the definition and properties of Markov chains, including transition matrix and examples.

Markov Chains: Transition Probabilities

Explores Markov chains, transition matrices, distribution, and random walks.

Markov Chains: States Classification

Explores the classification of states in Markov chains, emphasizing equivalence classes and periodicity.

Recurrence & Transience

Explores the concepts of recurrence and transience in Markov chains, distinguishing between them and discussing their implications.

Expected Number of Visits in State

Covers the criterion for recurrence in infinite chains based on the expected number of visits in a state.

Positive and Null-Recurrence

Explains positive and null-recurrence in Markov chains and the classification of states in equivalence classes.

Stationary Distribution in Markov Chains

Explores the concept of stationary distribution in Markov chains, discussing its properties and implications, as well as the conditions for positive-recurrence.

Stationary Distribution in Markov Chains

Explores examples of stationary distribution in Markov chains, including cyclic random walks and the implications of irreducibility.

Limiting Distribution and Ergodic Theorem

Explores limiting distribution in Markov chains and the implications of ergodicity and aperiodicity on stationary distributions.

Ergodic Theorem: Basic Tools

Explores the proof of the ergodic theorem using total variation distance and coupling concepts.

Coupling of Markov Chains: Ergodic Theorem

Explores the coupling of Markov chains and the proof of the ergodic theorem, emphasizing distribution convergence and chain properties.

Ergodic Theorem: Proof and Applications

Explains the proof of the ergodic theorem and the concept of positive-recurrence in Markov chains.

Reversible Chains & Detailed Balance

Explores reversible chains, detailed balance, transition probabilities, and numbered balls in urns.

Eigenvalues and Eigenvectors of Markov Chains

Explores eigenvalues and eigenvectors of Markov chains, focusing on convergence rates and matrix properties.

Spectral Gap and Mixing Time

Explores spectral gap and mixing time in Markov chains, including their definitions and behavior.

Convergence Rate Theorem: Part 1

Delves into the proof of the convergence rate theorem for an ergodic Markov chain, emphasizing eigenvalues and detailed balance properties.

Convergence Rate Theorem: Part 2

Covers the proof of the convergence rate theorem, emphasizing detailed balance equations and ergodic Markov chains.

Proof of Convergence Rate Theorem

Covers the proof of the convergence rate theorem, emphasizing the correction of a missing factor sqrt{pi_j} in the proof.

Spectral Gap in Markov Chains

Explores the spectral gap in Markov chains and its impact on convergence speed.

Lower Bound on Total Variation Distance

Explores the lower bound on total variation distance in Markov chains and its implications on mixing time.