Empirical process theory provides a rich toolbox for studying the properties of empirical risk minimizers, such as least squares and maximum likelihood estimators, support vector machines, etc.
In this series of lectures, we will start with considering exponential inequalities, including concentration inequalities, for the deviation of averages from their mean. We furthermore present some notions from approximation theory, because this enables us to assess the modulus of continuity of empirical processes. We introduce e.g., Vapnik Chervonenkis dimension: a combinatorial concept (from learning theory) of the "size" of a collection of sets or functions. As statistical applications, we study consistency and exponential inequalities for empirical risk minimizers, and asymptotic normality in semi-parametric models. We moreover examine regularization and model selection.
The main content is adapted from the previous ETH courses taught by Prof. van de Geer. We are very grateful for her lecture notes, which provide a student-friendly version of the book "Empirical Processes in M-Estimation", van de Geer, 2010 [Geer10].
The official course catalogue page can be found here.
This course is designed to be accessible for first-year master student in mathematics. A solid background on undergraduate probability and ``Fundamentals of Mathematical Statistics''' is required.
| Date | Content | Notes |
|---|---|---|
| Mon 22.09. | Introduction, motivation | Lecture 01 |
| Mon 29.09. | Glivenko-Cantelli classes | Lecture 02 |
| Mon 06.10. | Exponential probability inequalities | Lecture 03 |
| Mon 13.10. | Covering number, metric entropy, bracketing number, | Lecture 04 |
| Mon 20.10. | Symmetrization, ULLN based on symmetrization and entropy | Lecture 05 |
| Mon 27.10. | Symmetrization, VC-classes | Lecture 06 & 07 |
| Mon 03.11. | (continued) | - |
| Mon 10.11. | M-estimators, consistency examples | Lecture 08 |
| Mon 17.11. | Uniform central limit theorem | Lecture 09 |
| Mon 24.11. | Chaining, Dudley's entropy integral and asymptotic equicontinuity | Lecture 10 |
| Mon 01.12. | Application to VC graph classes, asymptotic normality of M-estimators | Lecture 11 |
| Mon 08.12. | Application to least-square estimators / regularized LSE | Lecture 12 |
| Mon 15.12. | Guest lecture by Antonio Di Noia |