Lecturer

Martin Schweizer

Coordinator

Zhouyi Tan

Time and Location

Thu. 14-16, HG E 21

First meeting date

23.09.2021

Content

The aim of this seminar is to give an introduction to some of the mathematical ideas behind reinforcement learning. This includes stochastic optimisation and convergence analysis. The emphasis is on mathematical theory, not on developing and testing algorithms.

Prerequisites

The underlying textbook mostly works with stochastic control problems for discrete-time Markov chains with a finite state space. But for a proper understanding, students should be familiar with measure-theoretic probability theory as well as stochastic processes in discrete time, and in particular with the construction of Markov chains on the canonical path space via the Ionescu-Tulcea theorem.

Rules and conditions

To obtain the credit points, everyone must give a 90-minute talk and attend the seminar regularly. During the first meeting, the talks will be assigned and the decision about the participants will be finalised.

The coordinator will hold a weekly office hour to answer questions from the students.

Office hour

Tu., 11-12, HG G 47.1

Materials

We mainly discuss materials from the book Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning, Second Edition by Abhijit Gosavi. All references below are to that book.

The last two talks will be based on a journal article.

List of Talks

Literature

• A. Gosavi, Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , Second Edition, Springer 2015.
• V.S. Borkar, S.P. Meyn, The ODE method for convergence of stochastic approximation and reinforcement learning, SIAM J. Control Optim. 2000.
• V. Dixon, I. Halperin, P. Bilokon, Machine Learning in Finance, Springer 2020. (This book is indicated only for extra background information and will not be used in the seminar.)