InsightFace Training Note

For two months, I have been training face recognition model using Deep Insight’s open source project InsightFace (https://github.com/deepinsight/insightface.git). All my experiments were conducted on the Tesla P40 GPU. All models I got from experiments are now kept by Shenzhen Sunwin Intelligent.

1. Selecting the network and loss function

InsightFace provides a variety of network and loss function choices, but according to the author, training ArcFace with LresNet100E-IR yields the most accurate model which achieved LFW 99.8%+ and MegaFace 98.0%+. Unfortunately, the training process was too slow so I jumped to the second best option – training LrestNet50E-IR with CosineFace. I used the given cleaned ms1m dataset as my training set and after about 400000 iterations the most accurate model I got yields lfw 99.7% and agedb_30 97.6%.

2. Saving the logging, standard output and model

I set the parameter ckpt to 2 in order to save all models (else only models that achieved lfw 99%+ will be saved). In order to know which saved model is the best one in the quickest way, I saved the logging and redirected the standard output to be appended to the log.

** Command "| tee -a " can save the log and std out to a file and still print to console.

3. Creating private training dataset

Alignment

To detect, align and crop the faces, use /src/align/align_megaface.py with argument "--name webface". However the codes are for photos which have attributes bbox and landmark, so I added the following codes:

Now landmark is a 10x1 numpy array, containing x and y coordinates of 5 facial landmarks. But only 3 points are needed for estimation, so I changed the line after similarity transform to

Change file format and rename

After running /src/align/align_megaface.py I got a dataset containing only cropped faces. It also generates a .lst file which can be directly used for the next step. However, if names of photos contain elements other than letters of English alphabet or numbers, there will be an exception during training. Also, the default raw photo format for the training is .jpg. To solve these two problems, I wrote a script which uses PIL to convert all photos to RGB, rename and save them as .jpg files.

Generate .lst file

I modified /src/data/glint2lst.py so that it writes all photo names to a .lst file in the format 1 ADDRESS LABEL.

Notice that the folders' structure should look like the following:

dataset

folders of different label each represents an individual

photos of one individual

Write the property file and use face2rec2.py to generate .idx and .rec

The property file has the format <TOTAL NUMBER OF IDENTITIES,112,112>. The codes in /src/face2rec2.py directly generate a new property file when merging two datasets.

Merge the private dataset with ms1m

I used /src/data/dataset_merge.py to merge the two datasets.

4. Using triplet to fine-tune

Using train_triplet.py to fine-tune the model can sometimes improve the accuracy by about 0.1%. All the parameters are given in insightface’s readme page.

5. Verifying accuracy

I used /src/eval/verification.py to verify the accuracy of my model.

Result

Verification datasets are from InsightFace.

	LFW(%)	CFP-FF(%)	CFP-FP(%)	AgeDB-30(%)	Vgg2-fp(%)
R50 (CosineFace)	99.717	99.814	92.714	97.600
R50 (triplet, Cos)	99.717	99.800	93.114	97.783
MobileFaceNet(ArcFace)	99.483	99.429	90.043	95.550
MobileFaceNet(triplet, Arc)	99.583	99.671	95.357	96.533	94.320

MobileFaceNet training process

The first two steps are from: deepinsight/insightface#214. The dataset I used combined the ms1m-v1 dataset from InsightFace and a private dataset. The private dataset is provided by Shenzhen Sunwin Intelligent and contains 1,900,000 raw photos of 50,000 identities collected from Chinese social media Weibo, QQ, Wechat, Tik Tok, etc. No data overlap with ms1m, lfw and agedb-30 is detected yet. The merged dataset contains around 135k individuals.

After 140k iteration the highest accuracy on agedb-30 is 89.333%. I used the 89.3% model as the pretrained model and trained with argument "--lr_steps='100000,140000,160000'". After 400k iteration the highest accuracy on agedb-30 is 94.817%. I used the 94.8% model as the pretrained model and trained it on ms1m-v1 from InsightFace. After 600k iteration the highest accuracy on agedb-30 is 95.767%. I then fine-tuned the 95.7% model using /src/train_triplet.py and InsightFace's ms1m-v2 dataset; after 30k iteration I got agedb-30 96.317%. Then I fine-tuned it with lr 0.004 and finally got the above agedb_30 96.533% result.

Acknowledgement

@article{deng2018arcface,
title={ArcFace: Additive Angular Margin Loss for Deep Face Recognition},
author={Deng, Jiankang and Guo, Jia and Zafeiriou, Stefanos},
journal={arXiv:1801.07698},
year={2018}
}