Reimplementing NMF algorithm proposed in C. Lin, "Projected Gradient Methods for Nonnegative Matrix Factorization," in Neural Computation, vol. 19, no. 10, pp. 2756-2779, Oct. 2007. doi: 10.1162/neco.2007.19.10.2756
Non-Negative Matrix Factorization (NMF) is a group of unsupervised machine learning techniques which factors a multivariate dataset as the approximate product of two lower dimension matrices. NMF is widely applied in various fields such as astronomy, computer and recommender systems. In this project we are interested in implementing NMF algorithm and analyze the robustness of the algorithm when the dataset is contaminated by different types of noise.
Our objective is to find two non-negative matrix factors W and H which multiply to give as close as possible a representation of our original image V.
Where W contains a basis optimized for the linear approximation of data in V, weighted by the components in H, i.e. Each column of W is a basis element. If V is our proverbial cake, then W and H are our list of core ingredients and their respective amounts.
More in depth information on NMF can be found at Wikipedia.
With NMF, we are hoping that we can learn representation/latent factor of the images that we can reconstruct faces from any situation, in this case it will be reconstructing noisy face or even reconstruct face image in dark lighting condition.
By altering the existing alternating non-negative least squares algorithm where it will alternatively fixes one matrix and improves the other.
Applying projected gradient method into alternating non-negative least squares algorithm we could get a faster convergence rate. Proposed projected gradient method can be seen in the pseudo code below:
For more information on the proofs and intuition of the algorithm, please kindly refer to the research paper.
As shown in the figure below, we manage to get a decent reconstuction(denoise) from the noisy images.
From the figure above, though we have managed to reconstruct the image but it seems to over-smooth the image. YaleB dataset has better reconstruction result, as I believe this is due to the dataset have perfectly centered face compared to ORL dataset. This suggests that we need some pre-processing on ORL to obatain a better result.
Part of the W(learned dictionary) for YaleB dataset is shown below.
Example on how to use program can be found at PG-NMF_example.ipynb.
Implement other different algorithms so that we can compare the robustness of different proposed approach.
https://angms.science/doc/NMF/nmf_pgd.pdf
latex generated with https://www.codecogs.com/latex/eqneditor.php