This GitHub repository contains all code created for my bachelor thesis. It is a combination of already existing pruning algorithms and sparse neural network analysis methods, as well as new approaches discussed in my thesis.
- All Results - Contains all sparse networks trained and obtained
- Configs - Contains configuration files of models for better repruceabilty of results
- Data - Contains the dataset that were used
- Experiments - Contains the main files of both pruning algorithms, which get called from the main.py
- Figures - Contains the plots created from the notebooks
- Layers - Contains the unique implementations of layers that the pruning algorithms require to work properly
- Models - Contains the implementations of the Fully-Connected and ResNet-20 models
- Notebooks - Contains all the notebooks created for analysis of each sparsity level and model
- Pruners - Contains parts of the pruning algorithms
- Trainers - Contains training loops from the pruning algorithms
- Utils - Contains most of the functions necessary to make the creation, the pruning and the training of models possible
- thesis_notebook - Contains implementations of all plots used in the thesis
- README - Contains the words you are currently reading
- gem_miner_args_helper - File for processing the configfiles of Gem-Miner
- loader - File for loading functions, such as models or test accuracies
- main - Is the file calling the main files of the pruning algorithms, depending on the experiment choosen in the configfile
- synflow_args_helper - File for processing the configfiles of SynFlow
- syngem_utils - Contains all functions and analysis methods created for this thesis
- run - the file being called to run the main plus config file
- environment - Contains the libraries and packages used for making the code in this repo work
To reproduce the results of this thesis choose your experiment by un-commenting it in the run.sh file and then executing it or alternatively run the main file manually such as :
python main.py --config Configs/synflow_fc.yml
Depending on the experiment you want to run, you may need to adjust the configfile in the Configs directory accordingly, such as setting the random seed, pruning algorithm or desired sparsity.
With the sparse neural network obtained, you can choose an analysis notebook to reproduce the results, but the loading functions may need to be adjusted, if the networks are not stored in the respective directories. Also the notebook works best when run in this directory, thus being able to access all models and functions properly.
For any problems or questions feel free to contact me.
The pruning algorithms used can be found at the GitHub repositories of SynFlow and Gem-Miner and can also be found in an adjusted version in the respective directories in this repo. The adjustments include usabilty aspects as well as crucial adaptions, for example, letting Gem-Miner prune Fully-Connected models.
The original analysis methods can be found as implementations in their respective GitHub repositories as well.