Credits

6 ECTS, CTD 33h, TP 16.5h

• Lecture: Numerical optimisation
  • ECTS: 6
  • Course code: GBX8AM02 (UGA) or WMM8AM02 (INP)

Instructors

• Quoc-Tung Le

Description

This program provides the mathematical and numerical backgrounds for solving standard optimisation problems using (mostly) first-order methods, with a thorough understanding of which algorithm to choose when, how to tune the parameters, and what the theory behind them is. Concrete examples will be investigated, and in particular the training of machine learning models.

Content

1. Introduction, classification, examples.
2. Theoretical results: convexity and compactness, optimality conditions, duality
3. Algorithms for unconstrained optimisation (gradient descent, line search, stochastic methods)
4. Algorithms for non-differentiable problems (prox, subgradient).

This course includes practical sessions in Python.

Prerequisites

Basic algebra (linear spaces, matrix computation). Basic calculus (norm, Banach spaces, Hilbert spaces, basic differential calculus). The students should be able to compute the gradient and the Hessian of real functions on IR^n and also differentials of simple functions such as quadratic forms. Knowing how to code in Python is required to pass the course, as there will be graded practical sessions.