Optimization for Machine Learning

This repository includes all codes implementation of a graduate-level optimization for machine learning course.
It focuses on unconstrained first-order methods which are frequently used in area of machine learning and data science.

Topics include:

1. Gradient Descent for Linear Regression.
2. Projected Gradient Descent Methods
3. Amijio Line Search for logistic regression (in matlab version)
4. mirror descent algorithm
5. proximal gradient descent
6. accelerated gradient descent (heavy-ball methods and Nesterov acceleration)
7. ADMM algorithm (rPCA for video object detection)
8. SGD algorithm: sample with/without replacement, average SGD, momentum acceleration.

Note

For any college faculties or students who want to use these codes during class or homework, you can directly use them for reference or demo.