This is the OLD 2019 course website. For the current one, see here.
This course is offered jointly with the Information Processing Group.
Previous year’s website: ML 2018
Contact us: Use the moodle discussion forum, or some of the contact details below:
| Instructor | Martin Jaggi | Instructor | Ruediger Urbanke |
| Office | INJ 341 | Office | INR 116 |
| [email protected] | [email protected] | ||
| Office Hours | By appointment | Office Hours | By appointment |
| Teaching Assistants
|
Student Assistants
|
| Lectures | Tuesday | 17:15 – 19:00 | Room: SG1 (first week in rolex forum) |
| Thursday | 16:15 – 18:00 | Room: SG1 (first week in rolex forum) |
|
| Exercises | Thursday | 14:15 – 16:00 | Rooms: INF119, INF2, INJ218, INM202, INR219 |
| Language: | English | |
| Credits : | 7 ECTS |
For a summary of the logistics of this course, see the course info sheet here (PDF).
(and also here is a link to official coursebook information).
Special Announcements
- The final exam takes place Wednesday 15.01.2020 from 16h15 to 19h15 in STCC08328
- The first week lectures (both tue+thur) will take place in the Rolex learning center for capacity reasons.
-
Projects: There will be two group projects during the course.
-
Project 1 counts 10% and is due Oct 28th.
-
Project 2 counts 30% and is due Dec 19th.
-
-
The new exercise sheet, as well as the solutions for the previous week will typically be made available each tuesday (here and on github). Labs and projects will be in Python. See Lab 1 to get started.
-
Code Repository for Labs, Projects, Lecture notes: github.com/epfml/ML_course
-
Link to additional AriML online app.
-
the exam is closed book but you are allowed one crib sheet (A4 size paper, both sides can be used), either handwritten or 11 point minimum font size; bring a pen and white eraser; you find the exams from the past three years with solutions here:
Detailed Schedule
Annotated lecture notes from each class are made available on github here.
| Date | Topics Covered | Lectures | Exercises | Projects |
|---|---|---|---|---|
| 17/9 | Introduction, Linear Regression | 01a,01b | ||
| 19/9 | Cost functions | 01c,01d | Lab 1 | |
| 24/9 | Optimization | 02a | ||
| 26/9 | Optimization | Lab 2 | Project 1 start | |
| 01/10 | Least Squares, Max Likelihood | 03a,03b | ||
| 03/10 | Overfitting, Ridge Regression, Lasso | 03c,03d | Lab 3 | |
| 08/10 | Generalization, Model Selection, and Validation | 04a | ||
| 10/10 | Bias-Variance decomposition | 04b | Lab 4 | |
| 15/10 | Classification | 05a | ||
| 17/10 | Logistic Regression | 05b | Lab 5 | |
| 22/10 | Generalized Linear Models | 06a | ||
| 24/10 | K-Nearest Neighbor | 06b | Lab 6 | |
| 29/10 | Support Vector Machines | 07a | Proj. 1 due 28.10. | |
| 31/10 | Kernel Regression | 07b | Lab 7 | |
| 05/11 | Unsupervised Learning, K-Means | 08a,08b | Project 2 start | |
| 07/11 | Gaussian Mixture Models | 08c | Lab 8 | |
| 12/11 | EM algorithm | 09a | ||
| 14/11 | SVD and PCA | 09b | Lab 9 | |
| 19/11 | Matrix Factorizations | 10a | ||
| 21/11 | Text Representation Learning | 10b | Lab 10 | |
| 26/11 | Neural Networks – Basics, Representation Power | |||
| 28/11 | Neural Networks – Backpropagation, Activation Functions | Lab 11 & Q&A | ||
| 03/12 | Neural Networks – CNN, Regularization, Data Augmentation, Dropout | 12a, 12b | ||
| 05/12 | Adversarial ML | 12c | Lab 12 | |
| 10/12 | Adversarial ML | |||
| 12/12 | Graphical Models — Bayes Nets | 13b | Lab 13 | |
| 17/12 | Graphical Models – Factor Graphs and the Sum-Product Algorithm | 14a | ||
| 19/12 | Graphical Models – Factor Graphs and the Sum-Product Algorithm | 14b | Proj. 2 due 19.12. |
Textbooks
(not mandatory)
Gilbert Strang, Linear Algebra and Learning from Data
Christopher Bishop, Pattern Recognition and Machine Learning
Shai Shalev-Shwartz, Shai Ben-David, Understanding Machine Learning
Michael Nielsen, Neural Networks and Deep Learning
Projects
Final projects last year were done among 5 options.
The interdisciplinary ‘ML for Science’ projects performed last year across campus were:
Quality of Life in Swiss Cities based on OpenStreetMap
Compressive Sensing MRI using Deep Learning
Predicting the density of lightning activity from atmospheric and geographic features
Machine Learning for Science: Quantum Machine Learning
Solar Panel Recognition and Segmentation on Swiss Map using Convolutional Neural Networks
ML for crystal structure determination as an alternative to NMR spectroscopy
Human Behavior Modelling
Domain-invariant defect detection using deep learning
Comparing classification techniques for metabolic kinetic models selection
Spatially-Inferred Graphical Models for fMRI Data Mining
Crowded enzyme kinetics using simulation and machine learning
Quality of life in Swiss Cities
Ultrathin section segmentation
Correlations between cognitive performance and sensory stimuli in the work environment
Autism Diagnostic based on Machine Learning
Human performance modelling according to indoor temperature and light (quantity and colour)
Automatic Harmonization using Recurrent Neural Networks
Chord recognition on Beethoven string quartets
Machine Learning Privacy
Quality of Life Clustering of Swiss Cities from Insurance and Demographic Data
Machine learning for air quality measurement and modeling
The Case for Bagged Neural Networks: Evidence from Outlier Detection using Autoencoder Ensembles
Chord Prediction with The Annotated Beethoven Corpus
Predicting Forces on a Flapping Wing Model using Machine Learning
Brain Tissue Segmentation
Clustering and Predicting Swiss cities based on Insurance Data
Predicting the material properties of the arterial wall of a mouse
Predicting organic carbon with infrared spectra
3D Pointclouds Super-resolution for Digital humanities
Segmentation Of Silicon Wafers For Electron Microscopy Using Mask-RCNN
Classifying Nanopore Readings with Deep Learning
Wind Profile Prediction in an Urban Canyon: a Machine Learning Approach
A Stem Cell Classifier for Single Cell RNA Sequencing Data
Deep Convolutional Neural Networks for Cell Segmentation in Bright-Field Microscopy Images
Predicting Aerosol Particles: Sulfate, Nitrate and PM2.5
Healthy aging: age group prediction from chunking strategies during motor sequence learning
Multi-lingual text classifier for social media data
Implementation of an Improved Model for the Prediction of Effective Rate Constants in the Presence of Crowders
Architecture of Feelings
Evaluating the quality of videos through machine learning
Classifying segmentation defects in mutant zebrafish embryo
Fingerprinting DNS-over-HTTPS traffic
Statistics of Turkish researchers after the 2016 Coup d’Etat attempt
Analysis of the dismissal of Turkish researchers after the 2016 Coup d’Etat attempt
Evaluating the risk of relapse in melanoma
Compressive Sensing MRI using Deep Learning
Predicting the density of lightning activity from atmospheric and geographic features
Machine Learning for Science: Quantum Machine Learning
Solar Panel Recognition and Segmentation on Swiss Map using Convolutional Neural Networks
ML for crystal structure determination as an alternative to NMR spectroscopy
Human Behavior Modelling
Domain-invariant defect detection using deep learning
Comparing classification techniques for metabolic kinetic models selection
Spatially-Inferred Graphical Models for fMRI Data Mining
Crowded enzyme kinetics using simulation and machine learning
Quality of life in Swiss Cities
Ultrathin section segmentation
Correlations between cognitive performance and sensory stimuli in the work environment
Autism Diagnostic based on Machine Learning
Human performance modelling according to indoor temperature and light (quantity and colour)
Automatic Harmonization using Recurrent Neural Networks
Chord recognition on Beethoven string quartets
Machine Learning Privacy
Quality of Life Clustering of Swiss Cities from Insurance and Demographic Data
Machine learning for air quality measurement and modeling
The Case for Bagged Neural Networks: Evidence from Outlier Detection using Autoencoder Ensembles
Chord Prediction with The Annotated Beethoven Corpus
Predicting Forces on a Flapping Wing Model using Machine Learning
Brain Tissue Segmentation
Clustering and Predicting Swiss cities based on Insurance Data
Predicting the material properties of the arterial wall of a mouse
Predicting organic carbon with infrared spectra
3D Pointclouds Super-resolution for Digital humanities
Segmentation Of Silicon Wafers For Electron Microscopy Using Mask-RCNN
Classifying Nanopore Readings with Deep Learning
Wind Profile Prediction in an Urban Canyon: a Machine Learning Approach
A Stem Cell Classifier for Single Cell RNA Sequencing Data
Deep Convolutional Neural Networks for Cell Segmentation in Bright-Field Microscopy Images
Predicting Aerosol Particles: Sulfate, Nitrate and PM2.5
Healthy aging: age group prediction from chunking strategies during motor sequence learning
Multi-lingual text classifier for social media data
Implementation of an Improved Model for the Prediction of Effective Rate Constants in the Presence of Crowders
Architecture of Feelings
Evaluating the quality of videos through machine learning
Classifying segmentation defects in mutant zebrafish embryo
Fingerprinting DNS-over-HTTPS traffic
Statistics of Turkish researchers after the 2016 Coup d’Etat attempt
Analysis of the dismissal of Turkish researchers after the 2016 Coup d’Etat attempt
Evaluating the risk of relapse in melanoma
Links to the 3 official project competitions we offered on AIcrowd:
The following teams participated in the ICLR reproducibility challenge:
Improving Generalization and Stability of Generative Adversarial Networks
Meta-learning with differentiable closed-form solvers
A Resizable Mini-batch Gradient Descent based on a Multi-Armed Bandit
Detecting Adversarial Examples via Neural Fingerprinting
Learning Neural PDE Solvers with Convergence Guarantees
AutoLoss: Learning Discrete Schedules for Alternate Optimization
MAE : Mutual Posterior-Divergence Regularization for Variational Autoencoders
Hyper-Regularization: An Adaptive Choice for the Learning Rate in Gradient Descent