Numerical optimisation

Credits

6 ECTS, CTD 33h, TP 16.5h

Instructors

Hadrien Hendrikx and Ieva Petrulyonite

Description

This program provides the mathematical and numerical backgrounds for solving standard optimisation problems using (mostly) first-order methods, with a thorough understanding of which algorithm to choose when, how to tune the parameters, and what is the theory behind. Concrete examples will be investigating, and in particular the training of machine learning models.

Content

1. Introduction, classification, examples.
2. Theoretical results: convexity and compacity, optimality conditions, duality
3. Algorithmic for unconstrained optimisation (gradient descent, line search, stochastic methods)
4. Algorithms for non differentiable problems (prox, subgradient).

This course includes practical sessions in Python.

Prerequisites

Basic algebra (linear spaces, matrix computation) Basic calculus (Norm, Banach spaces, Hilbert spaces, basic differential calculus) The students should be able to compute the gradient and the Hessian of real functions on IR^n and also differentials of simple functions such as quadratic forms. Knowing how to code in Python is required to pass the course, as there will be graded practical sessions.