Faster R-CNN

By Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun at Microsoft Research

Introduction

Faster R-CNN is a framework for object detection with deep CNNs including a Region Proposal Network (RPN) and an Object Detection Network. Both networks are trained for sharing convolutional layers for fast testing.

Faster R-CNN was initially described in an arXiv tech report.

This repo contains a MATLAB re-implementation of Fast R-CNN. Details about Fast R-CNN are in: rbgirshick/fast-rcnn.

License

Faster R-CNN is released under the MIT License (refer to the LICENSE file for details).

Citing Faster R-CNN

If you find Faster R-CNN useful in your research, please consider citing:

@article{ren15fasterrcnn,
    Author = {Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun},
    Title = {{Faster R-CNN}: Towards Real-Time Object Detection with Region Proposal Networks},
    Journal = {arXiv preprint arXiv:1506.01497},
    Year = {2015}
}

Main resutls

           | training data | test data | mAP | time/img

-------------- |:------------:|:------------:|:------------:|:------------: Faster RCNN, VGG-16 | VOC 2007 trainval | VOC 2007 test | 69.9% | 196ms Faster RCNN, VGG-16 | VOC 2007 trainval + 2012 trainval | VOC 2007 test | 73.2% | 196ms Faster RCNN, VGG-16 | VOC 2012 trainval | VOC 2012 test | 67.0% | 196ms Faster RCNN, VGG-16 | VOC 2007 trainval&test + 2012 trainval | VOC 2012 test | 70.4% | 196ms

Contents

0. Requirements: software
1. Requirements: hardware
2. Preparation for Testing
3. Testing Demo
4. Preparation for Training
5. Training
6. Resources

Requirements: software

0. Caffe build for Faster R-CNN (included in this repository)
   • If you are using Windows, you may download a compiled mex file by running fetch_data/fetch_caffe_library.m
   • If you are using Linux or you want to compile for Windows, please follow the instructions on our Caffe branch.
1. MATLAB

Requirements: hardware

GPU memory

0. Region Proposal Network (RPN)
   • 2GB GPU memory for ZF net
   • 5GB GPU memory for VGG-16 net
1. Ojbect Detection Network (Fast R-CNN)
   • 3GB GPU memory for ZF net
   • 8GB GPU memory for VGG-16 net

Preparation for Testing:

0. Run fetch_data/fetch_caffe_library.m to download a compiled Caffe mex (for Windows only).
1. Run faster_rcnn_build.m
2. Run startup.m

Testing Demo:

0. Run fetch_data/fetch_model_trained.m to download our trained models.
1. Run experiments/script_faster_rcnn_demo.m to test a single demo image.
   • The first run might be slower due to memory load.
   • The running time on K40 of this code is about 220ms/image, 10% more than we reported in the paper. This is because of unknown issues when we switch from our older version of Caffe to the newer one.
   • The speed on Titan X is about 2x of on K40.

Preparation for Training:

0. Run fetch_data/fetch_model_ZF.m to download an ImageNet-pre-trained ZF net.
1. Run fetch_data/fetch_model_VGG16.m to download an ImageNet-pre-trained VGG-16 net.
2. Download VOC 2007 and 2012 data to ./datasets

Training:

0. Run experiments/script_faster_rcnn_VOC2007_ZF.m to train a model with ZF net. It runs four steps as follows:
   • Train RPN with conv layers tuned; compute RPN results on the train/test sets.
   • Train Fast R-CNN with conv layers tuned using step-1 RPN proposals; evaluate detection mAP.
   • Train RPN with conv layers fixed; compute RPN results on the train/test sets.
   • Train Fast R-CNN with conv layers fixed using step-3 RPN proposals; evaluate detection mAP.
   • Note: the entire training time is ~12 hours on K40.
1. Run experiments/script_faster_rcnn_VOC2007_VGG16.m to train a model with VGG net.
   • Note: the entire training time is ~2 days on K40.

Resources

0. Experiment logs: OneDrive, DropBox, BaiduYun
1. Regions proposals of our trained RPN:
   • ZF net trained on VOC 07 trainval [To Add]
   • ZF net trained on VOC 07/12 trainval [To Add]
   • VGG net trained on VOC 07 trainval [To Add]
   • VGG net trained on VOC 07/12 trainval [To Add]

If the automatic "fetch_data" fails, you may manually download resouces from:

0. Pre-complied caffe mex:
   • Windows-based mex complied with VS2013 and Cuda6.5: OneDrive, DropBox, BaiduYun
1. ImageNet-pretrained networks:
   • Zeiler & Fergus (ZF) net OneDrive, DropBox, BaiduYun
   • VGG-16 net OneDrive, DropBox, BaiduYun
2. Final RPN+FastRCNN models: OneDrive, DropBox, BaiduYun