Concept

Andrey Markov

Related lectures (31)

Markov Chains: Recurrence and Transience

Explores first passage times, strong Markov property, and state recurrence/transience in Markov chains.

Quantum Entropy

Covers quantum entropy, density matrices, and entanglement in quantum information theory.

Markov Chains and Applications

Explores Markov chains, Ising Model, Metropolis algorithm, and Glauber dynamics.

MCMC Examples and Error Estimation

Covers Markov Chain Monte Carlo examples and error estimation methods.

Reactive Agents: Decision Processes

Explores reactive agents and decision processes, covering topics such as Markov Decision Processes and real-time agents.

Runtime Enforcement for Reactive Systems: Shield Synthesis

Explores shield synthesis for runtime enforcement in reactive systems, ensuring correct behavior with minimal deviation, illustrated with examples like traffic light controllers and Pac-Man.

Probability Inequalities

Explores probability inequalities, convergence types, and moment generating functions for distribution approximation.

Recurrence and transience: proofs

Covers the proof of the equivalence between AQ=0 and XP(s)=X.

Invariant Distributions: Markov Chains

Explores invariant distributions, recurrent states, and convergence in Markov chains, including practical applications like PageRank in Google.

Interactive Lecture: Reinforcement Learning

Explores advanced reinforcement learning topics, including policies, value functions, Bellman recursion, and on-policy TD control.

Inequalities: Cauchy-Schwarz, Jensen, Chebyshev

Explores inequalities for random variables, including Cauchy-Schwarz, Jensen, and Chebyshev.

Neurobiological Signals: Processing and Classification

Explores neurobiological signal processing, including spike modeling, de-noising, and data classification techniques.

Recurrence and Transience: Markov Chains

Explores recurrence and transience in Markov chains, discussing the strong Markov property and state classifications.

Markov Chains: Stopping Times

Explores stopping times in Markov Chains, illustrating their properties and the Strong Markov Property.

Markov Chains: Theory and Applications

Covers the theory and applications of Markov chains in modeling random phenomena and decision-making under uncertainty.

Dependability Evaluation in Industrial Automation

Explores dependability evaluation, preventive maintenance, reliability, Markov models, FMEA, FTA, and software safety integrity in industrial automation.

Markov Chains and Applications

Explores Markov chains, their properties, and algorithmic applications, emphasizing information quantification and state monotonicity.

Advanced Machine Learning: Discrete Reinforcement Learning

Introduces the basics of Reinforcement Learning, covering discrete states, actions, policies, value functions, MDPs, and optimal policies.

Replica and Symmetry Breaking

Explores the Nobel Prize-winning discovery of replica and cavity methods in complex systems, focusing on the random energy model and the application of probability theory.

Optimal Marketing Strategy

Covers decision-making in marketing based on customer behavior for optimal strategies.