STGCN

Introduction

STGCN is one of the first algorithms that adopt Graph Convolution Neural Networks for skeleton processing. We provide STGCN trained on NTURGB+D with 2D skeletons (HRNet) and 3D skeletons in both the original training setting and the PYSKL training setting. We provide checkpoints for four modalities: Joint, Bone, Joint Motion, and Bone Motion. The accuracy of each modality links to the weight file.

Citation

@inproceedings{yan2018spatial,
  title={Spatial temporal graph convolutional networks for skeleton-based action recognition},
  author={Yan, Sijie and Xiong, Yuanjun and Lin, Dahua},
  booktitle={Thirty-second AAAI conference on artificial intelligence},
  year={2018}
}
# If you use the STGCN with PYSKL practices in your work
@misc{duan2022pyskl,
    title={PYSKL: a toolbox for skeleton-based video understanding},
    author={PYSKL Contributors},
    howpublished = {\url{https://github.com/kennymckormick/pyskl}},
    year={2022}
}

Model Zoo

We release numerous checkpoints trained with various modalities, annotations on NTURGB+D and NTURGB+D 120. The accuracy of each modality links to the weight file.

Dataset	Practice	Annotation	GPUs	Training Epochs	Joint Top1 Config Link: Weight Link	Bone Top1 Config Link: Weight Link	Joint Motion Top1 Config Link: Weight Link	Bone-Motion Top1 Config Link: Weight Link	Two-Stream Top1	Four Stream Top1
NTURGB+D XSub	Vanilla	Official 3D Skeleton	8	80	joint_config: 81.5	bone_config: 81.0	joint_motion_config: 79.9	bone_motion_config: 81.2	84.3	86.6
NTURGB+D XSub	Vanilla	HRNet 2D Skeleton	8	80	joint_config: 85.7	bone_config: 85.8	joint_motion_config: 81.6	bone_motion_config: 83.9	88.8	90.1
NTURGB+D XSub	PYSKL	Official 3D Skeleton	8	80	joint_config: 87.8	bone_config: 88.6	joint_motion_config: 85.8	bone_motion_config: 86.2	90.0	90.7
NTURGB+D XSub	PYSKL	HRNet 2D Skeleton	8	80	joint_config: 89.0	bone_config: 91.2	joint_motion_config: 86.7	bone_motion_config: 87.8	92.0	92.4
NTURGB+D XView	Vanilla	Official 3D Skeleton	8	80	joint_config: 90.1	bone_config: 87.7	joint_motion_config: 88.8	bone_motion_config: 88.3	91.4	93.2
NTURGB+D XView	Vanilla	HRNet 2D Skeleton	8	80	joint_config: 92.4	bone_config: 90.0	joint_motion_config: 92.0	bone_motion_config: 86.5	93.8	95.1
NTURGB+D XView	PYSKL	Official 3D Skeleton	8	80	joint_config: 95.5	bone_config: 95.0	joint_motion_config: 93.7	bone_motion_config: 92.8	96.2	96.5
NTURGB+D XView	PYSKL	HRNet 2D Skeleton	8	80	joint_config: 98.0	bone_config: 96.5	joint_motion_config: 95.6	bone_motion_config: 95.4	98.2	98.3
NTURGB+D 120 XSub	PYSKL	Official 3D Skeleton	8	80	joint_config: 82.1	bone_config: 83.7	joint_motion_config: 80.3	bone_motion_config: 80.6	85.6	86.2
NTURGB+D 120 XSub	PYSKL	HRNet 2D Skeleton	8	80	joint_config: 80.1	bone_config: 83.4	joint_motion_config: 78.6	bone_motion_config: 79.8	84.0	84.7
NTURGB+D 120 XSet	PYSKL	Official 3D Skeleton	8	80	joint_config: 84.5	bone_config: 85.8	joint_motion_config: 82.7	bone_motion_config: 83.0	87.5	88.4
NTURGB+D 120 XSet	PYSKL	HRNet 2D Skeleton	8	80	joint_config: 84.2	bone_config: 87.7	joint_motion_config: 82.5	bone_motion_config: 83.5	88.3	89.0

Note

1. We use the linear-scaling learning rate (Initial LR ∝ Batch Size). If you change the training batch size, remember to change the initial LR proportionally.
2. For Two-Stream results, we adopt the 1 (Joint):1 (Bone) fusion. For Four-Stream results, we adopt the 2 (Joint):2 (Bone):1 (Joint Motion):1 (Bone Motion) fusion.

Training & Testing

You can use the following command to train a model.

bash tools/dist_train.sh ${CONFIG_FILE} ${NUM_GPUS} [optional arguments]
# For example: train STGCN on NTURGB+D XSub (3D skeleton, Joint Modality) with 8 GPUs, with validation, with PYSKL practice, and test the last and the best (with best validation metric) checkpoint.
bash tools/dist_train.sh configs/stgcn/stgcn_pyskl_ntu60_xsub_3dkp/j.py 8 --validate --test-last --test-best

You can use the following command to test a model.

bash tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${NUM_GPUS} [optional arguments]
# For example: test STGCN on NTURGB+D XSub (3D skeleton, Joint Modality) with metrics `top_k_accuracy`, and dump the result to `result.pkl`.
bash tools/dist_test.sh configs/stgcn/stgcn_pyskl_ntu60_xsub_3dkp/j.py checkpoints/SOME_CHECKPOINT.pth 8 --eval top_k_accuracy --out result.pkl