Skip to content

Latest commit

 

History

History
304 lines (277 loc) · 16.7 KB

README.md

File metadata and controls

304 lines (277 loc) · 16.7 KB
Raw

Group Normalization for Mask R-CNN

Introduction

This file provides Mask R-CNN baseline results and models trained with Group Normalization:

@article{GroupNorm2018,
  title={Group Normalization},
  author={Yuxin Wu and Kaiming He},
  journal={arXiv:1803.08494},
  year={2018}
}

Note: This code uses the GroupNorm op implemented in CUDA, included in the Caffe2 repo. When writing this document, Caffe2 is being merged into PyTorch, and the GroupNorm op is located here. Make sure your Caffe2 is up to date.

Pretrained Models with GN

These models are trained in Caffe2 on the standard ImageNet-1k dataset, using GroupNorm with 32 groups (G=32).

  • R-50-GN.pkl: ResNet-50 with GN, 24.0% top-1 error (center-crop).
  • R-101-GN.pkl: ResNet-101 with GN, 22.6% top-1 error (center-crop).

Results

Baselines with BN

         case           type lr
schd		 im/
gpu		 train
mem
(GB)		 train
time
(s/iter)		 train
time
total
(hr)		 inference
time
(s/im)		 box
AP		 mask
AP		 model id
R-50-FPN, BN* Mask R-CNN 2x 2 8.6 0.897 44.9 0.099 + 0.018 38.6 34.5 35859007
R-101-FPN, BN* Mask R-CNN 2x 2 10.2 0.993 49.7 0.126 + 0.017 40.9 36.4 35861858

Notes:

  • This table is copied from Detectron Model Zoo.
  • BN* means that BatchNorm (BN) is used for pre-training and is frozen and turned into a per-channel linear layer when fine-tuning. This is the default of Faster/Mask R-CNN and Detectron.

Mask R-CNN with GN

Standard Mask R-CNN recipe

         case           type lr
schd		 im/
gpu		 train
mem
(GB)		 train
time
(s/iter)		 train
time
total
(hr)		 inference
time
(s/im)		 box
AP		 mask
AP		 model id download
links
R-50-FPN, GN Mask R-CNN 2x 2 10.5 1.017 50.8 0.146 + 0.017 40.3 35.7 48616381 model  |  boxes  |  masks
R-101-FPN, GN Mask R-CNN 2x 2 12.4 1.151 57.5 0.180 + 0.015 41.8 36.8 48616724 model  |  boxes  |  masks

Notes:

  • GN is applied on: (i) ResNet layers inherited from pre-training, (ii) the FPN-specific layers, (iii) the RoI bbox head, and (iv) the RoI mask head.
  • These GN models use a 4conv+1fc RoI box head. The BN* counterpart with this head performs similarly with the default 2fc head: using this codebase, R-50-FPN BN* with 4conv+1fc has 38.8/34.4 box/mask AP.
  • 2x is the default schedule (180k) in Detectron.

Longer training schedule

         case           type lr
schd		 im/
gpu		 train
mem
(GB)		 train
time
(s/iter)		 train
time
total
(hr)		 inference
time
(s/im)		 box
AP		 mask
AP		 model id download
links
R-50-FPN, GN Mask R-CNN 3x 2 10.5 1.033 77.4 0.145 + 0.015 40.8 36.1 48734751 model  |  boxes  |  masks
R-101-FPN, GN Mask R-CNN 3x 2 12.4 1.171 87.9 0.180 + 0.014 42.3 37.2 48734779 model  |  boxes  |  masks

Notes:

  • 3x is a longer schedule (270k). GN can improve further when using the longer schedule, but its BN* counterpart remains similar (R-50-FPN BN*: 38.9/34.3) with the longer schedule.
  • These models are without any scale augmentation that can further improve results.

Explorations

Training Mask R-CNN from scratch

GN enables to train Mask R-CNN from scratch without ImageNet pre-training, despite the small batch size.

         case           type lr
schd		 im/
gpu		 train
mem
(GB)		 train
time
(s/iter)		 train
time
total
(hr)		 inference
time
(s/im)		 box
AP		 mask
AP		 model id
R-50-FPN, GN, scratch Mask R-CNN 3x 2 10.8 1.087 81.5 0.140 + 0.019 39.5 35.2 56421872
R-101-FPN, GN, scratch Mask R-CNN 3x 2 12.7 1.243 93.2 0.177 + 0.019 41.0 36.4 56421911

Notes:

  • To reproduce these results, see the config yaml files starting with scratch .
  • These are results using freeze_at=0. See this commit about the related issue.

 

R-50-FPN, GN, scratch Mask R-CNN 3x 2 10.5 0.990 74.3 0.146 + 0.020 36.2 32.5 49025460
R-101-FPN, GN, scratch Mask R-CNN 3x 2 12.4 1.124 84.3 0.180 + 0.019 37.5 33.3 49024951

Notes:

  • These are early results that followed the default training using freeze_at=2. This means the layers of conv1 and res2 were simply random weights in the case of training from-scratch. See this commit about the related issue.