Neural networks behave as hash encoders: An empirical study

This repository contains the code for the paper "Neural networks behave as hash encoders: An empirical study" by Fengxiang He, Shiye Lei, Jianmin Ji, and Dacheng Tao.

Experimental results

• Model capacity

• Training time

• Sample size

Instructions

Dependencies

• Python3.6
• Tensorflow1.9
• Keras2.2
• MNIST dataset
• CIFAR-10 dataset

Training

1. Model capacity

• For MNIST: python run_layer_width_train_mlp.py --dataset mnist --depth 1 --begin_repeat 1 --repeat 10
• For CIFAR-10: python run_layer_width_train_mlp.py --dataset cifar10 --depth 5 --begin_repeat 1 --repeat 10

2. Sample size

• For MNIST: python run_sample_size_train_mlp.py --dataset mnist --depth 1 --begin_repeat 1 --repeat 10
• For CIFAR-10: python run_sample_size_train_mlp.py --dataset cifar10 --depth 5 --begin_repeat 1 --repeat 10

3. Training time

• For MNIST: python run_training_time_train_mlp.py --dataset mnist --depth 1 --begin_repeat 1 --repeat 10
• For CIFAR-10: python run_training_time_train_mlp.py --dataset cifar10 --depth 5 --begin_repeat 1 --repeat 10

Compute and evaluate encoding properties

1. Model capacity

• For MNIST: python run_layer_width_encoding_properties.py --dataset mnist --depth 1 --begin_repeat 1 --repeat 10
• For CIFAR-10: python run_layer_width_encoding_properties.py --dataset cifar10 --depth 5 --begin_repeat 1 --repeat 10

2. Sample size

• For MNIST: python run_sample_size_encoding_properties.py --dataset mnist --depth 1 --begin_repeat 1 --repeat 10
• For CIFAR-10: python run_sample_size_encoding_properties.py --dataset cifar10 --depth 5 --begin_repeat 1 --repeat 10

3. Training time

• For MNIST: python run_training_time_encoding_properties.py --dataset mnist --depth 1 --begin_repeat 1 --repeat 10
• For CIFAR-10: python run_training_time_encoding_properties.py --dataset cifar10 --depth 5 --begin_repeat 1 --repeat 10

Well-trained model

For the well-trained models in our paper, please kindly contact Shiye Lei at [email protected].

Citation

@article{he2021neural,
  title={Neural networks behave as hash encoders: An empirical study},
  author={He, Fengxiang and Lei, Shiye and Ji, Jianmin and Tao, Dacheng},
  journal={arXiv preprint arXiv:2101.05490},
  year={2020}
}

Contact

For any issue, please kindly contact

Fengxiang He: [email protected]
Shiye Lei: [email protected]
Jianmin Ji: [email protected]
Dacheng Tao: [email protected]

--

Last update: Fri 15 Jan 2021