Pre-training on ImageNet-1K

Installation

Please follow the installation instructions in DINOv2 and install timm==0.9.16 as well.

Dataset

We prepare ImageNet-1K following the instructions in DeiT.

Training

1. Specify the directory of datasets with data-path in the training script run_pretrain.sh.

2. Specify the teacher and student choices with teacher-model and target_model.

3. Specify the model choices with model to choose from DINOv2, SynCLR, CLIP.

4. Specify the pre-trained teacher path with teacher-path for SynCLR and CLIP training.

5. Simply run the training script as follows:

   bash run_pretrain.sh
   

Acknowledgment

This part is heavily build upon DeiT, DINOv2, SynCLR. We gratefully thank the authors for their wonderful works.