Table of contents

• Table of contents
• Deep Sort with PyTorch(support yolo series)
  • Introduction
  • Model
    • Object detection
    • ReID
  • Project structure
  • Dependencies
  • Quick Start
  • The Object model
    • Training the Object model
    • Detect
    • Validate
  • Training the RE-ID model
  • Demo videos and images
• C ++
• References

Deep Sort with PyTorch(support yolo series)

Introduction

This is an implement of MOT tracking algorithm deep sort. Currently, the project supports models of the mainstream yolo series model. This project add the existing yolo detection model algorithm (YOLOv3, YOLOV4, YOLOV4Scaled, YOLOV5, YOLOV6, YOLOV7, YOLOV8, YOLOX, YOLOR, PPYOLOE). Therefore, it is sometimes called the following name:

• YOLOv3-DeepSort
• YOLOV4-DeepSort
• YOLOV4Scaled-DeepSort
• YOLOV5-DeepSort
• YOLOV6-DeepSort
• YOLOV7-DeepSort
• YOLOV8-DeepSort
• YOLOX-DeepSort
• YOLOR-DeepSort
• PPYOLOE-DeepSort

Also, this project is compatible with the original object detection model structure. Individual training and inference of the detection model can be performed.

Model

Object detection

• MMDet
• YOLOv3
• YOLOV4
• YOLOV4Scaled
• YOLOV5
• YOLOV6
• YOLOV7
• YOLOV8
• YOLOX
• YOLOR
• PPYOLOE

ReID

• deepsort-reid
• fast-reid

Project structure

Expand

yolovx_deepsort_pytorch/
├── 001.avi
├── checkpoint
├── configs
│   ├── deep_sort.yaml
│   ├── fastreid.yaml
│   ├── mmdet.yaml
│   ├── ppyoloe.yaml
│   ├── yolor.yaml
│   ├── yolov3_tiny.yaml
│   ├── yolov3.yaml
│   ├── yolov4Scaled.yaml
│   ├── yolov4.yaml
│   ├── yolov5.yaml
│   ├── yolov6.yaml
│   ├── yolov7.yaml
│   └── yolox.yaml
├── deep_sort
│   ├── deep
│   ├── deep_sort.py
│   ├── __init__.py
│   ├── __pycache__
│   ├── README.md
│   └── sort
├── deepsort.py
├── demo
│   ├── 1.jpg
│   ├── 2.jpg
│   └── demo.gif
├── detector
│   ├── __init__.py
│   ├── MMDet
│   ├── PPYOLOE
│   ├── __pycache__
│   ├── YOLOR
│   ├── YOLOv3
│   ├── YOLOV4
│   ├── YOLOV4Scaled
│   ├── YOLOV5
│   ├── YOLOV6
│   ├── YOLOV7
│   └── YOLOX
├── LICENSE
├── models
│   ├── deep_sort_pytorch
│   ├── ppyoloe
│   ├── readme.md
│   ├── yolor
│   ├── yolov3
│   ├── yolov4
│   ├── yolov4-608
│   ├── yolov4Scaled
│   ├── yolov5
│   ├── yolov6
│   ├── yolov7
│   └── yolox
├── output
│   ├── ppyoloe
│   ├── README.MD
│   ├── yolor
│   ├── yolov3
│   ├── yolov4
│   ├── yolov4Scaled
│   ├── yolov5
│   ├── yolov6
│   ├── yolov7
│   └── yolox
├── ped_det_server.py
├── README.md
├── requirements.txt
├── results_analysis
│   └── analysis.py
├── scripts
│   ├── yoloe.sh
│   ├── yolor.sh
│   ├── yolov3_deepsort.sh
│   ├── yolov3_tiny_deepsort.sh
│   ├── yolov4_deepsort.sh
│   ├── yolov4Scaled_deepsort.sh
│   ├── yolov5_deepsort.sh
│   ├── yolov6_deepsort.sh
│   ├── yolov7_deepsort.sh
│   └── yolox_deepsort.sh
├── thirdparty
│   ├── fast-reid
│   └── mmdetection
├── train.jpg
├── tutotial
│   ├── Hungarian_Algorithm.ipynb
│   ├── kalman_filter.ipynb
│   └── kalman_filter.py
├── utils
│   ├── asserts.py
│   ├── draw.py
│   ├── evaluation.py
│   ├── __init__.py
│   ├── io.py
│   ├── json_logger.py
│   ├── log.py
│   ├── parser.py
│   ├── __pycache__
│   └── tools.py
├── webserver
│   ├── config
│   ├── images
│   ├── __init__.py
│   ├── readme.md
│   ├── rtsp_threaded_tracker.py
│   ├── rtsp_webserver.py
│   ├── server_cfg.py
│   └── templates
└── yolov3_deepsort_eval.py

Dependencies

Expand

See this requirements.txt for more detail.

• python 3 (python2 not sure)
• numpy
• scipy
• opencv-python
• sklearn
• torch >= 0.4
• torchvision >= 0.1
• pillow
• vizer
• edict

Quick Start

Expand

0. Check all dependencies installed

pip install -r requirements.txt

for user in china, you can specify pypi source to accelerate install like:

pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

1. Clone this repository

git clone https://github.com/xuarehere/yolovx_deepsort_pytorch.git

2. Download YOLOv3 parameters

cd detector/YOLOv3/weight/
wget https://pjreddie.com/media/files/yolov3.weights
wget https://pjreddie.com/media/files/yolov3-tiny.weights
cd ../../../

3. Download deepsort parameters ckpt.t7

cd deep_sort/deep/checkpoint
# download ckpt.t7 from
https://drive.google.com/drive/folders/1xhG0kRH1EX5B9_Iz8gQJb7UNnn_riXi6 to this folder
cd ../../../

4. Compile nms module

cd detector/YOLOv3/nms
sh build.sh
cd ../../..

Notice: If compiling failed, the simplist way is to **Upgrade your pytorch >= 1.1 and torchvision >= 0.3" and you can avoid the troublesome compiling problems which are most likely caused by either gcc version too low or libraries missing.

5. (Optional) Prepare third party submodules

fast-reid

This library supports bagtricks, AGW and other mainstream ReID methods through providing an fast-reid adapter.

to prepare our bundled fast-reid, then follow instructions in its README to install it.

Please refer to configs/fastreid.yaml for a sample of using fast-reid. See Model Zoo for available methods and trained models.

MMDetection

This library supports Faster R-CNN and other mainstream detection methods through providing an MMDetection adapter.

to prepare our bundled MMDetection, then follow instructions in its README to install it.

Please refer to configs/mmdet.yaml for a sample of using MMDetection. See Model Zoo for available methods and trained models.

Run

git submodule update --init --recursive

6. Run demo

usage: deepsort.py [-h]
                   [--fastreid]
                   [--config_fastreid CONFIG_FASTREID]
                   [--mmdet]
                   [--config_mmdetection CONFIG_MMDETECTION]
                   [--config_detection CONFIG_DETECTION]
                   [--config_deepsort CONFIG_DEEPSORT] [--display]
                   [--frame_interval FRAME_INTERVAL]
                   [--display_width DISPLAY_WIDTH]
                   [--display_height DISPLAY_HEIGHT] [--save_path SAVE_PATH]
                   [--cpu] [--camera CAM]
                   VIDEO_PATH         

# yolov3 + deepsort
python deepsort.py [VIDEO_PATH]

# yolov3_tiny + deepsort
python deepsort.py [VIDEO_PATH] --config_detection ./configs/yolov3_tiny.yaml

# yolov3 + deepsort on webcam
python3 deepsort.py /dev/video0 --camera 0

# yolov3_tiny + deepsort on webcam
python3 deepsort.py /dev/video0 --config_detection ./configs/yolov3_tiny.yaml --camera 0

# fast-reid + deepsort
python deepsort.py [VIDEO_PATH] --fastreid [--config_fastreid ./configs/fastreid.yaml]

# MMDetection + deepsort
python deepsort.py [VIDEO_PATH] --mmdet [--config_mmdetection ./configs/mmdet.yaml]


# yolov4 + deepsort on video
python3 deepsort.py ./001.avi --save_path ./output/yolov4/001 --config_detection ./configs/yolov4.yaml --detect_model yolov4


# yolov4Scaled + deepsort on video
python3 deepsort.py ./001.avi --save_path ./output/yolov4Scaled/001 --config_detection ./configs/yolov4Scaled.yaml --detect_model yolov4Scaled

# yolov5 + deepsort on video
python3 deepsort.py ./001.avi --save_path ./output/yolov5/001 --config_detection ./configs/yolov5.yaml --detect_model yolov5

# yolov6 + deepsort on video
python3 deepsort.py ./001.avi --save_path ./output/yolov6/001 --config_detection ./configs/yolov6.yaml --detect_model yolov6

# yolov7 + deepsort on video
python3 deepsort.py ./001.avi --save_path ./output/yolov7/001 --config_detection ./configs/yolov7.yaml --detect_model yolov7

# yolov8 + deepsort on video
python3 deepsort.py ./001.avi --save_path ./output/yolov8/001 --detect_model yolov8 --config_detection ./configs/yolov8.yaml 

# yolox + deepsort on video
python deepsort.py  ./001.avi --save_path ./output/yolox/001  --config_detection ./configs/yolox.yaml  --detect_model yolox 

# yolor + deepsort on video
python deepsort.py  ./001.avi --save_path ./output/yolor/001  --config_detection ./configs/yolor.yaml  --detect_model yolor 

# ppyoloe + deepsort on video
python deepsort.py  ./001.avi --save_path ./output/ppyoloe/001  --config_detection ./configs/ppyoloe.yaml  --detect_model ppyoloe 

Use --display to enable display.
Results will be saved to ./output/results.avi and ./output/results.txt.

All files above can also be accessed from BaiduDisk!
linker：BaiduDisk passwd：fbuw

The Object model

Expand

Switch to model directory, see README.MD for more details.

For example:

cd detector/YOLOV5
# see detector/YOLOV5/README.md for more details

Training the Object model

see detector/{model}/README.md for more details.

For example:

cd detector/YOLOV5
python train.py --img 640 --data VOC.yaml --cfg yolov5s.yaml --weight ../../models/yolov5/yolov5s.pt --batch-size 64 --epochs 100

or See this link for more detail

Detect

see detector/{model}/README.md for more details.

cd detector/YOLOV5/
python  detect.py --source ../../001.avi --weights ../../models/yolov5/v4.0/yolov5s.pt --device 1

Validate

Validate a model's accuracy on datasets.

see detector/{model}/README.md for more details.

cd detector/YOLOV5/

python val.py --weights  ../../models/yolov5/v4.0/yolov5s.pt  --data coco.yaml  --img 640 --half

Training the RE-ID model

Expand

The original model used in paper is in original_model.py, and its parameter here original_ckpt.t7.

To train the model, first you need download Market1501 dataset or Mars dataset.

Then you can try train.py to train your own parameter and evaluate it using test.py and evaluate.py.

Dataset

python prepare_data.py --data_dir {data_path}/Market-1501-v15.09.15

Train

$ cd ./deep_sort/deep/train.py

$ python train.py --data-dir /workspace/dataset/Market-1501/Market-1501-v15.09.15/pytorch/ --interval 10  --gpu-id 0

See this link for more detail

Demo videos and images

C ++

yolov5, v7

• yolov7_deepsort_tensorrt:

References

Expand

• ultralytics/ultralytics
• isLinXu/YOLOv8_Efficient
• nwojke/deep_sort
• Joseph Redmon/yolov3
• ZQPei/deep_sort_pytorch
• WongKinYiu/PyTorch_YOLOv4
• WongKinYiu/ScaledYOLOv4
• ultralytics/yolov5
• meituan/YOLOv6
• WongKinYiu/yolov7
• Megvii-BaseDetection/YOLOX
• WongKinYiu/yolor
• Nioolek/PPYOLOE_pytorch