Lecturer

Yuansi Chen

Lecture Time

Tue 10:15-12:00 (from 16.04.2024 to 28.05.2024), HG G 26.5

Course Number

401-4634-24L/401-4634-DRL

Office Hours

Tue 3:00-4:00 (from 16.04.2024 to 28.05.2024), HG G 15.1

Content

Selected topics on Markov chain Monte Carlo sampling algorithms, diffusion generative models and related proof techniques.
The official course catalogue page can be found here.

Prerequisites

We assume basic knowledge of linear algebra, introduction to probability and statistics.
Familarity with convex optimization, stochastic process and stochastic calculus would help.

Lecture notes

(tentative, please refresh page)

Suggested readings

Lecture 1

• Introduction of Sampling, Diffusions, and Stochastic Localization , Montanari 2023
• Nonparametric density estimation course notes from Computational Statistics

Lecture 2

• Geometric Random Walks: A Survey, Vempala 2005

Lecture 3

• Chapter 1 on Langevin Diffusion in Continuous Time in Log-Concave sampling, Chewi 2024
• Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices, Vempala and Wibisono 2022
• Log-concave sampling: Metropolis-Hastings algorithms are fast, Dwivedi, Chen, Wainwright and Yu 2018
• A simple proof of the mixing of metropolis-adjusted langevin algorithm under smoothness and isoperimetry, Chen and Gatmiry 2023

Lecture 4

• Denoising Diffusion Probabilistic Models, Ho, Jain and Abbeel 2020
• Score-Based Generative Modeling through Stochastic Differential Equations , Song et al. 2020
• Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions, Chen et al. 2022

Lecture 5

• Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions, Chen et al. 2022
• Sampling, Diffusions, and Stochastic Localization , Montanari 2023

Lecture 6

• Generative Adversarial Networks, Goodfellow 2014
• An Introduction to Variational Autoencoders , Kingma and Welling 2019
• Stochastic Interpolants: A Unifying Framework for Flows and Diffusions , Albergo, Boffi and Vanden-Eijnden 2023

Lecture 7

• Notes on the use of stochastic localization in high dimensional probability, Eldan 2021
• Sampling, Diffusions, and Stochastic Localization , Montanari 2023
• An Information-Theoretic View of Stochastic Localization , El Alaoui and Montanari 2021

Literature

• Stochastic Integration and Differential Equations, Protter 2005
• Stochastic Differential Equations : An Introduction with Applications, Øksendal 2003
• Mathematical Aspects of Mixing Times in Markov Chains, Montenegro and Tetali 2006
• Partial Differential Equations, Jost 2007, mainly Chapter 7 The Heat Equation, Semigroups, and Brownian Motion
• Log-Concave sampling, Chewi 2024

Acknowledgement

This course contains materials such as lecture notes/slides that were developed or adapted from many other courses. Especially,

• Convergence of Markov Chains, by Ravi Montenegro
• Continuous Algorithms, by Kevin Tian