Numerical Optimization for Machine Learning
This repository contains the intuition behind basic numerical optimization using multivariate calculus and vector operations. More importantly, it contains python example code for numerical optimization.
Line search is quickly glossed over in study guide, but I put in some example code to help better illustrate how to use it. However, sometimes instead of trying to write code it might just be easier to use the current libraries such as scipy for optimization purposes.
I put in the folder first order and second order algorithms for just using Line Search.
Included a package called mlrose, which was the the libraries for several algorithms including Hill Climbing. Very useful to use. Also included is a tutorial on how to use them.
Please let me know if any of my explanations are not clear or need improvement. I am happy to correct and edit.
Thank you Qian Ge for your explanation on line search using python and how to go out it.
Thank you Alex Gonchar for your code on optimization algorithms.
Hayes, G. (2019). mlrose: Machine Learning, Randomized Optimization and SEarch package for Python. https://github.com/gkhayes/mlrose. Accessed: 19, March 2019.