Information Theory and Signal Processing

Instructor Michael Gastpar Instructor Emre Telatar Instructor Ruediger Urbanke
Office INR 130 Office INR 117 Office INR 116
Email [email protected] Email [email protected] Email [email protected]
Office Hours By appointment Office Hours By appointment Office Hours By appointment
Teaching Assistant Su Li Email [email protected] Office INR034
Admin Assistant Muriel Bardet Email [email protected] Office INR137
Lectures Monday 09:15 – 11:00  Room: INM200
Friday 08:15 – 10:00  Room: INM201
Exercises Friday 10:15 – 12:00 Room:  INM201
Language: English
Credits : 6 ECTS

Lecture notes (PDF file)

Official Prerequisites: COM-300 Modèles stochastiques pour les communications (or equivalent)

Here is a link to official coursebook information).

Some Homework will be graded…

If you do not hand in your final exam your overall grade will be NA. Otherwise, your grade will be determined based on the following weighted average:
10 % for the Homework, 90 % for the Final Exam.

Special Announcements

Last year’s Final Exam and Solutions.

Detailed Schedule

(tentative, subject to changes)

Date Topics Covered Lectures Exercises
Review HW0 Sol0
21/9 General Introduction ; Review Linear Algebra, Probability
Exercise: Review Session (Linear Algebra, Probability)   (MG)
Chapter 2 Handout
Information Measures
24/9 Basic Information Measures (ET) Handout
28/9 (ET) HW1
01/10 (ET)
05/10 (ET) Sol1
Estimation and Detection
08/10 Optimum Detection and Estimation ; MMSE (MG)
12/10 Parameter estimation ; Fisher information ; Cram`er-Rao bound (MG) HW2
15/10 Exploration bias and generalization guarantees (II) Handout1
19/10 via information measures (II) Handout2 Sol2
Exponential Families
22/10 Exponential families ; Max Entropy problems (RU) Chapter 4
26/10 Boltzmann distribution ; Exponential families (RU) HW3 Sol3
Compression and Quantization
29/10 Compression and Quantization (ET)
02/11 Compression and Quantization (ET) HW4
05/11 Compression and Quantization (ET)
09/11 Compression and Quantization (ET) Sol4
Multi-Arm Bandits
12/11 Multi-armed Bandits : Explore & Exploit (RU) Chapter 7
16/11 Multi-armed Bandits : UCB algorithm (RU) HW5
19/11 Multi-armed Bandits : Converse bound (RU) Sol5
23/11 Multi-armed Bandits : Variations (RU)
Signal Representations
Signal Representation : Fourier, Sparse Fourier, (MG)
Chapter 6
30/11 Wavelets, Compressed sensing? (MG) HW6
03/12 Wiener Filter (MG)
07/12 LMS Adaptive Filter (MG) Sol6
Distribution Estimation
10/12 Distribution Estimation ; Property Testing and Estimation (RU) Chapter 8
14/12 Distribution Estimation ; Property Testing and Estimation (RU) HW7
17/12 Distribution Estimation ; Property Testing and Estimation (RU)
21/12 Distribution Estimation ; Property Testing and Estimation (RU) Sol7