Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
root caf8280fe6 | 1 year ago | |
---|---|---|
.ipynb_checkpoints | 1 year ago | |
.vscode | 2 years ago | |
assets | 4 years ago | |
cython_bbox-0.1.3 | 2 years ago | |
demos | 1 year ago | |
exp/mot/fairmot_hrnet_w18 | 1 year ago | |
experiments | 2 years ago | |
models | 1 year ago | |
src | 1 year ago | |
track | 1 year ago | |
tracking | 2 years ago | |
videos | 2 years ago | |
.gitignore | 4 years ago | |
README.md | 2 years ago | |
ffmpeg-4.4.1-essentials_build.zip | 2 years ago | |
models.zip | 2 years ago | |
requirements.txt | 2 years ago | |
run.py | 1 year ago | |
text.py | 1 year ago |
A simple baseline for one-shot multi-object tracking:
FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking,
Yifu Zhang, Chunyu Wang, Xinggang Wang, Wenjun Zeng, Wenyu Liu,
IJCV2021 (arXiv 2004.01888)
There has been remarkable progress on object detection and re-identification in recent years which are the core components for multi-object tracking. However, little attention has been focused on accomplishing the two tasks in a single network to improve the inference speed. The initial attempts along this path ended up with degraded results mainly because the re-identification branch is not appropriately learned. In this work, we study the essential reasons behind the failure, and accordingly present a simple baseline to addresses the problems. It remarkably outperforms the state-of-the-arts on the MOT challenge datasets at 30 FPS. We hope this baseline could inspire and help evaluate new ideas in this field.
Dataset | MOTA | IDF1 | IDS | MT | ML | FPS |
---|---|---|---|---|---|---|
2DMOT15 | 60.6 | 64.7 | 591 | 47.6% | 11.0% | 30.5 |
MOT16 | 74.9 | 72.8 | 1074 | 44.7% | 15.9% | 25.9 |
MOT17 | 73.7 | 72.3 | 3303 | 43.2% | 17.3% | 25.9 |
MOT20 | 61.8 | 67.3 | 5243 | 68.8% | 7.6% | 13.2 |
All of the results are obtained on the MOT challenge evaluation server under the “private detector” protocol. We rank first among all the trackers on 2DMOT15, MOT16, MOT17 and MOT20. The tracking speed of the entire system can reach up to 30 FPS.
conda create -n FairMOT
conda activate FairMOT
conda install pytorch==1.7.0 torchvision==0.8.0 cudatoolkit=10.2 -c pytorch
cd ${FAIRMOT_ROOT}
pip install cython
pip install -r requirements.txt
git clone -b pytorch_1.7 https://github.com/ifzhang/DCNv2.git
cd DCNv2
./make.sh
crowdhuman
|——————images
| └——————train
| └——————val
└——————labels_with_ids
| └——————train(empty)
| └——————val(empty)
└------annotation_train.odgt
└------annotation_val.odgt
If you want to pretrain on CrowdHuman (we train Re-ID on CrowdHuman), you can change the paths in src/gen_labels_crowd_id.py and run:
cd src
python gen_labels_crowd_id.py
If you want to add CrowdHuman to the MIX dataset (we do not train Re-ID on CrowdHuman), you can change the paths in src/gen_labels_crowd_det.py and run:
cd src
python gen_labels_crowd_det.py
MOT15
|——————images
| └——————train
| └——————test
└——————labels_with_ids
└——————train(empty)
MOT20
|——————images
| └——————train
| └——————test
└——————labels_with_ids
└——————train(empty)
Then, you can change the seq_root and label_root in src/gen_labels_15.py and src/gen_labels_20.py and run:
cd src
python gen_labels_15.py
python gen_labels_20.py
to generate the labels of 2DMOT15 and MOT20. The seqinfo.ini files of 2DMOT15 can be downloaded here [Google], [Baidu],code:8o0w.
DLA-34 COCO pretrained model: DLA-34 official.
HRNetV2 ImageNet pretrained model: HRNetV2-W18 official, HRNetV2-W32 official.
After downloading, you should put the pretrained models in the following structure:
${FAIRMOT_ROOT}
└——————models
└——————ctdet_coco_dla_2x.pth
└——————hrnetv2_w32_imagenet_pretrained.pth
└——————hrnetv2_w18_imagenet_pretrained.pth
Our baseline FairMOT model (DLA-34 backbone) is pretrained on the CrowdHuman for 60 epochs with the self-supervised learning approach and then trained on the MIX dataset for 30 epochs. The models can be downloaded here:
crowdhuman_dla34.pth [Google] [Baidu, code:ggzx ] [Onedrive].
fairmot_dla34.pth [Google] [Baidu, code:uouv] [Onedrive]. (This is the model we get 73.7 MOTA on the MOT17 test set. )
After downloading, you should put the baseline model in the following structure:
${FAIRMOT_ROOT}
└——————models
└——————fairmot_dla34.pth
└——————...
sh experiments/crowdhuman_dla34.sh
sh experiments/mix_ft_ch_dla34.sh
sh experiments/mix_dla34.sh
sh experiments/mot17_dla34.sh
sh experiments/mot15_ft_mix_dla34.sh
#np.clip(xy[:, 0], 0, width, out=xy[:, 0])
#np.clip(xy[:, 2], 0, width, out=xy[:, 2])
#np.clip(xy[:, 1], 0, height, out=xy[:, 1])
#np.clip(xy[:, 3], 0, height, out=xy[:, 3])
Then, we can train on the mix dataset and finetune on MOT20:
sh experiments/crowdhuman_dla34.sh
sh experiments/mix_ft_ch_dla34.sh
sh experiments/mot20_ft_mix_dla34.sh
The MOT20 model 'mot20_fairmot.pth' can be downloaded here: [Google] [Baidu, code:jmce].
sh experiments/mix_mot17_half_dla34.sh
sh experiments/mix_mot17_half_hrnet18.sh
sh experiments/mix_mot17_half_res34.sh
sh experiments/mix_mot17_half_res34fpn.sh
sh experiments/mix_mot17_half_res50.sh
The ablation study model 'mix_mot17_half_dla34.pth' can be downloaded here: [Google] [Onedrive] [Baidu, code:iifa].
Training Data | MOTA | IDF1 | IDS |
---|---|---|---|
MOT17 | 69.8 | 69.9 | 3996 |
MIX | 72.9 | 73.2 | 3345 |
CrowdHuman + MIX | 73.7 | 72.3 | 3303 |
sh experiments/all_yolov5s.sh
The pretrained model of yolov5s on the COCO dataset can be downloaded here: [Google] [Baidu, code:wh9h].
The model of the light version 'fairmot_yolov5s' can be downloaded here: [Google] [Baidu, code:2y3a].
cd src
python track.py mot --load_model ../models/fairmot_dla34.pth --conf_thres 0.6
to see the tracking results (76.5 MOTA and 79.3 IDF1 using the baseline model). You can also set save_images=True in src/track.py to save the visualization results of each frame.
cd src
python track_half.py mot --load_model ../exp/mot/mix_mot17_half_dla34.pth --conf_thres 0.4 --val_mot17 True
If you use our pretrained model 'mix_mot17_half_dla34.pth', you can get 69.1 MOTA and 72.8 IDF1.
cd src
python track.py mot --test_mot17 True --load_model ../models/fairmot_dla34.pth --conf_thres 0.4
python track.py mot --test_mot16 True --load_model ../models/fairmot_dla34.pth --conf_thres 0.4
cd src
python track.py mot --test_mot17 True --load_model ../models/fairmot_yolov5s.pth --conf_thres 0.4 --arch yolo --reid_dim 64
and send the txt files to the MOT challenge evaluation server to get the results. (You can get the SOTA results 73+ MOTA on MOT17 test set using the baseline model 'fairmot_dla34.pth'.)
cd src
python track.py mot --test_mot15 True --load_model your_mot15_model.pth --conf_thres 0.3
python track.py mot --test_mot20 True --load_model your_mot20_model.pth --conf_thres 0.3
Results of the test set all need to be evaluated on the MOT challenge server. You can see the tracking results on the training set by setting --val_motxx True and run the tracking code. We set 'conf_thres' 0.4 for MOT16 and MOT17. We set 'conf_thres' 0.3 for 2DMOT15 and MOT20.
You can input a raw video and get the demo video by running src/demo.py and get the mp4 format of the demo video:
cd src
python demo.py mot --load_model ../models/fairmot_dla34.pth --conf_thres 0.4
You can change --input-video and --output-root to get the demos of your own videos.
--conf_thres can be set from 0.3 to 0.7 depending on your own videos.
You can train FairMOT on custom dataset by following several steps bellow:
A large part of the code is borrowed from Zhongdao/Towards-Realtime-MOT and xingyizhou/CenterNet. Thanks for their wonderful works.
@article{zhang2021fairmot,
title={Fairmot: On the fairness of detection and re-identification in multiple object tracking},
author={Zhang, Yifu and Wang, Chunyu and Wang, Xinggang and Zeng, Wenjun and Liu, Wenyu},
journal={International Journal of Computer Vision},
pages={1--19},
year={2021},
publisher={Springer}
}
No Description
Text Wavefront Object Python C other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》