Machine Learning CS-433

This course is offered jointly by the TML and MLO groups.
Previous year’s website: ML 2019

Contact us: Use the discussion forum, or some of the contact details below:

Instructor Nicolas Flammarion Instructor Martin Jaggi
Office INJ 336 Office INJ 341
Email [email protected] Email [email protected]
Office Hours By appointment Office Hours By appointment
Teaching Assistants

 

  • Maksym Andriushchenko
  • Arnout Devos
  • Semih Günel
  • Prakhar Gupta
  • Mahdi Hajibabaei
  • Sai Praneeth Karimireddy
  • Anastasiia Koloskova
  • Tao Lin
  • Zohreh Mostaani
  • Guillermo Ortiz Jimenez
  • Matteo Pagliardini
  • Scott Pesme
  • Aswin Suresh
  • Thijs Vogels
Student Assistants

 

  • Mohamed Ridha Chahed
  • Paul Griesser
  • Haitham Hammami
  • Wei Jiang
  • Stanislas Jouven
  • Maja Stamenkovic
  • Robin Zbinden
Lectures Tuesday 2x45mins youtube recordings
  Thursday 2x45mins youtube recordings
Q&A  Thursday 16:15 – 17:00 short live Q&A on zoom, about lecture contents
Exercises Thursday 14:15 – 16:00

Rooms: live on discord, or INF119INF2INJ218INM202INR219

Exercises Solutions Tuesday 17:15 – 18:00 live on zoom
Language:   English
Credits :   7 ECTS

For a summary of the logistics of this course, see the course info sheet here (PDF).
(and also here is a link to official coursebook information).

Special Announcements

  • Please register on moodle asap so we can contact you. You can change registration later if needed.
  • The zoom links for Q&A and exercises (discord), the discussion forum, and the youtube playlist are on moodle.
  • Projects: There will be two group projects during the course.
    • Project 1 counts 10% and is due Oct 26th.
    • Project 2 counts 30% and is due Dec 17th.
  • The videos of the lectures for each week, new exercise sheet, as well as the solutions for the previous week will typically be made available each tuesday. Labs and projects will be in Python. See Lab 1 to get started.
  • Code Repository for Labs, Projects, Lecture notes: github.com/epfml/ML_course
  • Link to additional AriML online app (or iphone / android).
  • the exam is closed book but you are allowed one crib sheet (A4 size paper, both sides can be used), either handwritten or 11 point minimum font size; bring a pen and white eraser; you find the exams from the past three years with solutions here:

Detailed Schedule

Annotated lecture notes from each class are made available on github here.

Date Topics Covered Lectures Exercises Projects
15/9 Introduction, Linear Regression 01a,01b
01c,01d
   
17/9 Cost functions   Lab 1  
22/9 Optimization 02a    
24/9 Optimization   Lab 2 Project 1 start
29/10 Least Squares, Max Likelihood 03a,03b    
01/10 Overfitting, Ridge Regression, Lasso 03c,03d Lab 3  
06/10 Generalization, Model Selection, and Validation 04a    
08/10 Bias-Variance decomposition 04b Lab 4  
13/10 Classification 05a    
15/10 Logistic Regression 05b Lab 5  
20/10 Generalized Linear Models 06a    
22/10 K-Nearest Neighbor 06b Lab 6  
27/10 Support Vector Machines     Proj. 1 due 26.10.
29/10 Kernel Regression   Lab 7  
03/11 Neural Networks – Basics, Representation Power     Project 2 start
07/11 Neural Networks – Backpropagation, Activation Functions   Lab 8  
10/11 Neural Networks – CNN, Regularization, Data Augmentation, Dropout      
12/11 Adversarial ML   Lab 9  
17/11 Adversarial ML      
19/11 GANs   Lab 10  
24/11 Unsupervised Learning, K-Means      
26/11 Gaussian Mixture Models   Lab 11 & Q&A  
01/12 EM algorithm      
03/12 SVD and PCA   Lab 12  
08/12 Matrix Factorizations      
10/12 Text Representation Learning   Lab 13  
15/12 Invited Guest Lecture      
17/12 projects     Proj. 2 due 17.12.

Textbooks

(not mandatory)

Gilbert Strang, Linear Algebra and Learning from Data
Christopher Bishop, Pattern Recognition and Machine Learning
Shai Shalev-Shwartz, Shai Ben-David, Understanding Machine Learning
Michael Nielsen, Neural Networks and Deep Learning

Projects

Final projects last year were done among 5 options.

The interdisciplinary ‘ML for Science’ projects performed last year across campus were:

Fall 2019:

Machine Learning for Invisibility Cloak Architecture
Predicting PV Array Power Output Using an All-Sky Camera
Predicting bacteria evolution during cell cycle observations using a Recurrent Neural Network
Searching for Similar YouTube Channels
Stress Classification from Biosignals using Neural Networks
Crystal-Structure Descriptor for Binary Materials Based on Coordination Numbers
Polymers identification using nanopores
ML for Science – Generate images of Mesopotamian artifacts
Machine Learning for Toxicological Testing
Global gene expression analysis: determine hormone signalling activation in human breast cancer samples
Yeast Cell Segmentation with U_Net: Effects of Weight Maps and Attention Gates
Machine Learning for LHC performance optimization
Product Life Cycle Prediction Using Machine Learning
Clustering estrogen receptor-positive breast cancer tumors based on hormonal response type
Identifying the shape of the worms based on the hand-annotated images dataset
Applause Recognition of Live Concert Recordings for the MetaMedia Center
Machine Learning for EDS Data Decomposition
Twitter Astroturfing Detection
Removing Noise From Microscope Images Without Ground Truth
U-Net for Yeast Cell Segmentation
C elegans segmentation
FCN for neuronal semantic segmentation
Machine Learning Project 2 – Optimal Option Exit Strategies with Neural Networks
Bio Product Classifier
Segmenting yeast cells from microscope images
Instance Segmentation of Yeast Cells using Mask R-CNN
A machine learning approach to determine chemical shifts in NMR spectroscopy data
Detecting rooftop solar PV installations using CNN
A Physical-Interactive Pac-Man Game for Stroke Rehabilitation
Logistic Hairdressing
Prediction Forces for a Flapping Wing system using Linear Regression and Neural Networks Methods
Exploratory Analysis with MOOC data used for Blended Learning
Predicting Energy Building Consumption
Anomaly detection for energy consumption time series
Segmentation of Cracks on Laboratory Images
The _place_ (physical and conceptual) of advertising in video game magazines
Image Classification: Distinguish Murine from Human Cells in PDXs Models
PLIER for single-cell RNA sequencing data
Machine Learning project #2: Identifying Outer Divertor Legs On Images From Tokamak Experiments
Machine Learning: Gravitational Lens Finding
VNAV – No GPS, No Problem
Earthquake Detection from Seismological Data
Detecting Astroturfing Bots on Twitter
Feature identification of MANTIS data
ObsBox Instability Clustering
Photovoltaic Power Production in Swiss Communes
Detection of Strong Gravitational Lenses with Convolutional Neural Networks
Nano Manufacturing with ML and Raman Spectroscopy
Motion Prediction on Drosophila Using a Seq2seq Model
Segmentation of spinal cord images
Avalanche Intelligence
Pac-Man Error Project
Building Classification using Google Street-View
U-Net application for yeast cell segmentation
Landmark Based Visual Navigation for Drones
Classification: Crashes and Disruptions in Plasma Experiments
Automating Route Setting in Climbing via Deep Generative Models
Leading Edge Suction Prediction of a Dynamically Pitching Airfoil with Trailing Edge Flap
Aerodynamical parameters estimation from velocity, pressure and noise sensors data using temporal convolutional neural network
Machine Learning for Science: Diffraction detection in rock structures
Classifying amino acid modifications using nanopore sequencing data
Microscopical Image Restoration of C. elegans using Noise2Noise
ML challenge : Strong Gravitational lensing
Instance segmentation
Invisibility cloak
Pacman collaboration with CHILI lab
Enchordings – Harmony Embeddings

Fall 2018:

Quality of Life in Swiss Cities based on OpenStreetMap
Compressive Sensing MRI using Deep Learning
Predicting the density of lightning activity from atmospheric and geographic features
Machine Learning for Science: Quantum Machine Learning
Solar Panel Recognition and Segmentation on Swiss Map using Convolutional Neural Networks
ML for crystal structure determination as an alternative to NMR spectroscopy
Human Behavior Modelling
Domain-invariant defect detection using deep learning
Comparing classification techniques for metabolic kinetic models selection
Spatially-Inferred Graphical Models for fMRI Data Mining
Crowded enzyme kinetics using simulation and machine learning
Quality of life in Swiss Cities
Ultrathin section segmentation
Correlations between cognitive performance and sensory stimuli in the work environment
Autism Diagnostic based on Machine Learning
Human performance modelling according to indoor temperature and light (quantity and colour)
Automatic Harmonization using Recurrent Neural Networks
Chord recognition on Beethoven string quartets
Machine Learning Privacy
Quality of Life Clustering of Swiss Cities from Insurance and Demographic Data
Machine learning for air quality measurement and modeling
The Case for Bagged Neural Networks: Evidence from Outlier Detection using Autoencoder Ensembles
Chord Prediction with The Annotated Beethoven Corpus
Predicting Forces on a Flapping Wing Model using Machine Learning
Brain Tissue Segmentation
Clustering and Predicting Swiss cities based on Insurance Data
Predicting the material properties of the arterial wall of a mouse
Predicting organic carbon with infrared spectra
3D Pointclouds Super-resolution for Digital humanities
Segmentation Of Silicon Wafers For Electron Microscopy Using Mask-RCNN
Classifying Nanopore Readings with Deep Learning
Wind Profile Prediction in an Urban Canyon: a Machine Learning Approach
A Stem Cell Classifier for Single Cell RNA Sequencing Data
Deep Convolutional Neural Networks for Cell Segmentation in Bright-Field Microscopy Images
Predicting Aerosol Particles: Sulfate, Nitrate and PM2.5
Healthy aging: age group prediction from chunking strategies during motor sequence learning
Multi-lingual text classifier for social media data
Implementation of an Improved Model for the Prediction of Effective Rate Constants in the Presence of Crowders
Architecture of Feelings
Evaluating the quality of videos through machine learning
Classifying segmentation defects in mutant zebrafish embryo
Fingerprinting DNS-over-HTTPS traffic
Statistics of Turkish researchers after the 2016 Coup d’Etat attempt
Analysis of the dismissal of Turkish researchers after the 2016 Coup d’Etat attempt
Evaluating the risk of relapse in melanoma
 
 
Links to the 3 official project competitions we offered on AIcrowd:
 
The following teams participated in the ICLR reproducibility challenge:

Improving Generalization and Stability of Generative Adversarial Networks
Meta-learning with differentiable closed-form solvers
A Resizable Mini-batch Gradient Descent based on a Multi-Armed Bandit
Detecting Adversarial Examples via Neural Fingerprinting
Learning Neural PDE Solvers with Convergence Guarantees
AutoLoss: Learning Discrete Schedules for Alternate Optimization
MAE : Mutual Posterior-Divergence Regularization for Variational Autoencoders
Hyper-Regularization: An Adaptive Choice for the Learning Rate in Gradient Descent