CS 295 - Optimization for Machine Learning

Below we will be posting slides and lecture notes for the class. The syllabus can be found here.

NEW! Lecture notes are available here.

Lecture 1 slides: Definitions and Gradient Descent
Lecture 2 slides: Gradient Descent (cont.)
Lecture 3 slides: Stochastic Gradient Descent
Lecture 4 slides: Stochastic Gradient Descent (examples)
Lecture 5 slides: Intro to Online optimization and Online Learning
Lecture 6 slides: Online optimization and Online Learning (cont.)
Lecture 7 slides: Accelerated Methods
Lecture 8 slides: Intro to non-convex optimization
Lecture 9 slides: Non-convex optimization: GD + noise avoids saddles
Lecture 10 slides: Introduction to min-max
Lecture 11 slides: Min-max and optimism
Lecture 12 slides: Intro to Multi-Armed Bandits
Lecture 13 slides: Multi-Armed Bandits (part 2)
Lecture 14 slides: Intro to MDPs
Lecture 17 slides: Stochastic Games and multi-agent RL
Lecture 15 slides: Intro to Perceptron and ERM
Lecture 16 slides: VC dimension and Learnability