My implementation of the Support Vector Machine (SVM) focused on classifying two classes from the CIFAR-10 dataset: airplane (label 0) and dog (label 5). As part of my college project, I built the SVM from scratch, exploring both primal and dual approaches.
My model supports the four different kernels that are found in the Wikipedia article:
- Polynomial (homogeneous) kernel:: k(xi,xj)=(xi⋅xj) d . When d=1, it becomes the linear kernel.
- Polynomial (inhomogeneous) kernel:: k(xi,xj)=(xi⋅xj+r) d .
- Gaussian (Radial Basis Function - RBF) kernel: k(xi,xj)=exp(−γ∥xi−xj∥ 2 )k(xi,xj)=exp(−γ ∥ xi−xj ∥ 2 ), where γ>0.
- Sigmoid (Hyperbolic tangent) kernel: k(xi,xj)=tanh(κxi⋅xj+c) for some (but not all) κ>0 και c<0.
Below we see the results of the SVM (training times, training/test accuracies) after many trials and parameter modifications. Due to computational restraints, only tests on primal SVM were taken:(
