Group Normalization for Mask R-CNN

Introduction

This file provides Mask R-CNN baseline results and models trained with Group Normalization:

@article{GroupNorm2018,
  title={Group Normalization},
  author={Yuxin Wu and Kaiming He},
  journal={arXiv:1803.08494},
  year={2018}
}

Note: This code uses the GroupNorm op implemented in CUDA, included in the Caffe2 repo. When writing this document, Caffe2 is being merged into PyTorch, and the GroupNorm op is located here. Make sure your Caffe2 is up to date.

Pretrained Models with GN

These models are trained in Caffe2 on the standard ImageNet-1k dataset, using GroupNorm with 32 groups (G=32).

• R-50-GN.pkl: ResNet-50 with GN, 24.0% top-1 error (center-crop).
• R-101-GN.pkl: ResNet-101 with GN, 22.6% top-1 error (center-crop).

Results

Baselines with BN

case	type	lr schd	im/ gpu	train mem (GB)	train time (s/iter)	train time total (hr)	inference time (s/im)	box AP	mask AP	model id
R-50-FPN, BN*	Mask R-CNN	2x	2	8.6	0.897	44.9	0.099 + 0.018	38.6	34.5	35859007
R-101-FPN, BN*	Mask R-CNN	2x	2	10.2	0.993	49.7	0.126 + 0.017	40.9	36.4	35861858

Notes:

• This table is copied from Detectron Model Zoo.
• BN^* means that BatchNorm (BN) is used for pre-training and is frozen and turned into a per-channel linear layer when fine-tuning. This is the default of Faster/Mask R-CNN and Detectron.

Mask R-CNN with GN

Standard Mask R-CNN recipe

case	type	lr schd	im/ gpu	train mem (GB)	train time (s/iter)	train time total (hr)	inference time (s/im)	box AP	mask AP	model id	download links
R-50-FPN, GN	Mask R-CNN	2x	2	10.5	1.017	50.8	0.146 + 0.017	40.3	35.7	48616381	model  |  boxes  |  masks
R-101-FPN, GN	Mask R-CNN	2x	2	12.4	1.151	57.5	0.180 + 0.015	41.8	36.8	48616724	model  |  boxes  |  masks

Notes:

• GN is applied on: (i) ResNet layers inherited from pre-training, (ii) the FPN-specific layers, (iii) the RoI bbox head, and (iv) the RoI mask head.
• These GN models use a 4conv+1fc RoI box head. The BN^* counterpart with this head performs similarly with the default 2fc head: using this codebase, R-50-FPN BN^* with 4conv+1fc has 38.8/34.4 box/mask AP.
• 2x is the default schedule (180k) in Detectron.

Longer training schedule

case	type	lr schd	im/ gpu	train mem (GB)	train time (s/iter)	train time total (hr)	inference time (s/im)	box AP	mask AP	model id	download links
R-50-FPN, GN	Mask R-CNN	3x	2	10.5	1.033	77.4	0.145 + 0.015	40.8	36.1	48734751	model  |  boxes  |  masks
R-101-FPN, GN	Mask R-CNN	3x	2	12.4	1.171	87.9	0.180 + 0.014	42.3	37.2	48734779	model  |  boxes  |  masks

Notes:

• 3x is a longer schedule (270k). GN can improve further when using the longer schedule, but its BN^* counterpart remains similar (R-50-FPN BN^*: 38.9/34.3) with the longer schedule.
• These models are without any scale augmentation that can further improve results.

Explorations

Training Mask R-CNN from scratch

GN enables to train Mask R-CNN from scratch without ImageNet pre-training, despite the small batch size.

case	type	lr schd	im/ gpu	train mem (GB)	train time (s/iter)	train time total (hr)	inference time (s/im)	box AP	mask AP	model id
R-50-FPN, GN, scratch	Mask R-CNN	3x	2	10.8	1.087	81.5	0.140 + 0.019	39.5	35.2	56421872
R-101-FPN, GN, scratch	Mask R-CNN	3x	2	12.7	1.243	93.2	0.177 + 0.019	41.0	36.4	56421911

Notes:

• To reproduce these results, see the config yaml files starting with scratch .
• These are results using freeze_at=0. See this commit about the related issue.

 

R-50-FPN, GN, scratch	Mask R-CNN	3x	2	10.5	0.990	74.3	0.146 + 0.020	36.2	32.5	49025460
R-101-FPN, GN, scratch	Mask R-CNN	3x	2	12.4	1.124	84.3	0.180 + 0.019	37.5	33.3	49024951

Notes:

• These are early results that followed the default training using freeze_at=2. This means the layers of conv1 and res2 were simply random weights in the case of training from-scratch. See this commit about the related issue.