This repository contains numpy implementation of MLP and trained on MNIST dataset.
- Code
2 Hidden Layers with 256 hidden units
The model is trained with single batch.
Activation function: Sigmoid
Loss function: Squared Loss
Training Loss: 0.497138709845
Training Loss: 0.35123461672
Training Loss: 0.29073212762
Training Loss: 0.25287852721
Training Loss: 0.228611725677
Training Loss: 0.210472892538
epoch 1: training loss 0.20387900301012948
epoch 1, 39.9 secs, lr = 1.0000, train accuracy 94.63, val accuracy 94.32
Training Loss: 0.093845447349
Training Loss: 0.0939409568035
Training Loss: 0.0950943133785
Training Loss: 0.0923214857969
Training Loss: 0.0910271367438
Training Loss: 0.0891776134181
epoch 2: training loss 0.08799078829060497
epoch 2, 40.7 secs, lr = 1.0000, train accuracy 96.22, val accuracy 95.17
Training Loss: 0.0525298033834
Training Loss: 0.0553080336578
Training Loss: 0.0550518232716
Training Loss: 0.0528660089821
Training Loss: 0.0527412062536
Training Loss: 0.0514901406598
epoch 1: training loss 0.050354947315775396
epoch 1, 39.9 secs, lr = 0.5000, train accuracy 97.70, val accuracy 96.50
Training Loss: 0.0390803788902
Training Loss: 0.0407655826777
Training Loss: 0.0397739693009
Training Loss: 0.0381463515907
Training Loss: 0.0385929397959
Training Loss: 0.037993033945
epoch 2: training loss 0.03723274103350594
epoch 2, 40.3 secs, lr = 0.5000, train accuracy 98.22, val accuracy 96.81
Training Loss: 0.0276963877574
Training Loss: 0.0281844048542
Training Loss: 0.0277066549976
Training Loss: 0.0263793106951
Training Loss: 0.0265859680621
Training Loss: 0.026233011265
epoch 1: training loss 0.025580692733962265
epoch 1, 35.5 secs, lr = 0.2000, train accuracy 98.78, val accuracy 97.04