Slides 2017

Outline

The 2017 course consists of the following topics

                       
Lecture 1 Introduction to convex optimization and iterative methods.
   
Lecture 2 Review of basic probability theory.
  Maximum likelihood, M-estimators, and empirical risk minimization as a motivation for convex optimization.
   
Lecture 3 Fundamental concepts in convex analysis.
  Basics of complexity theory.
   
Lecture 4 Unconstrained smooth minimization I:
  Concept of an iterative optimization algorithm.
  Convergence rates.
  Characterization of functions.
   
Lecture 5
Unconstrained smooth minimization II:
  Gradient and accelerated gradient methods.
   
Lecture 6 Unconstrained smooth minimization III:
  The quadratic case.
  The conjugate gradient method.
  Variable metric algorithms.
   
Lecture 7 Stochastic gradient methods.
   
Lecture 8 Composite convex minimization I: 
  Subgradient method.
  Proximal and accelerated proximal gradient methods.
   
Lecture 9 Composite convex minimization II:
  Proximal Newton-type methods.
  Stochastic proximal gradient methods.
   
Lecture 10 Constrained convex minimization I:
  The primal-dual approach.
  Smoothing approaches for non-smooth convex minimization.
   
Lecture 11 Constrained convex minimization II:
  The Frank-Wolfe method.
  The universal primal-dual gradient method.
  The alternating direction method of multipliers (ADMM).
   
Lecture 12 Disciplined convex programming.