Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
root 5068c9c470 | 1 year ago | |
---|---|---|
scripts | 1 year ago | |
src | 1 year ago | |
README.md | 1 year ago | |
eval_cls.py | 1 year ago | |
eval_seg.py | 1 year ago | |
export.py | 1 year ago | |
requirements.txt | 1 year ago | |
train_cls.py | 1 year ago | |
train_seg.py | 1 year ago |
Point Transformer was proposed in 2021.It design a self-attention layers for point clouds and use these to construct
self-attention networks for tasks such as semantic scene segmentation, object part segmentation, and object classification.
The networks are based purely on self-attention and pointwise operations
Paper: Zhao H, Jiang L, Jia J, et al. Point transformer[C]
Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 16259-16268.
The hierarchical structure of Point Transformer is composed by a number of point transformer block transition down block
and transition up block. Point Transformer Block is a residual block with the point transformer layer at its core.The input
is a set of feature vectors $x$ with associated 3D coordinates $p$. The point transformer block facilitates information exchange
between these localized feature vectors, producing new feature vectors for all data points as its output. The transition down block
is to reduce the cardinality of the point set as required, for example from N to N/4 in the transition from the first to the second stage.
transition up block is to map features from the downsampled input point set P2 onto its superset. The feature encoder in point transformer
networks for semantic segmentation and classification has five stages that operate on progressively downsampled point sets.
The downsampling rates for the stages are [1, 4, 4, 4, 4], thus the cardinality of the point set produced by each stage is [N, N/4, N/16, N/64, N/256],
where N is thenumber of input points.
Dataset used: alignment ModelNet40
Dataset size:6.48G,Each point cloud contains 2048 points uniformly sampled from a shape surface. Each cloud is
zero-mean and normalized into a unit sphere.
Data format:txt files
Dataset used: alignment ShapeNetCore
After installing MindSpore via the official website, you can start training and evaluation as follows:
If you want to train and evaluate the classification ability :
# Run training for classification with ModelNet40
bash scripts/run_train_cls.sh [DATASET_PATH] [LOG_PATH] [DEVICE_NUM] [VISIABLE_DEVICES(0,1,2,3,4,5,6,7)] [RANK_TABLE](optional)
# example Run stand-alone training:
bash scripts/run_train_cls.sh path-contain-modelnet40_normal_resampled log_cls 1 0
# Run distributed training
bash scripts/run_train_cls.sh path-contain-modelnet40_normal_resampled log_cls 8 0,1,2,3,4,5,6,7 hccl_8p_01234567_127.0.0.1.json
# Evaluate
bash scripts/run_eval_cls.sh [DATA_PATH] [CKPT_NAME]
# example:
bash scripts/run_eval_cls.sh path-contain-modelnet40_normal_resampled best_model.ckpt
If you want to train and evaluate the segmentation ability :
# Run training for segmentation with ShapeNetCore
bash scripts/run_train_seg.sh [DATASET_PATH] [LOG_PATH] [DEVICE_NUM] [VISIABLE_DEVICES(0,1,2,3,4,5,6,7)] [RANK_TABLE](optional)
# example Run stand-alone training:
bash scripts/run_train_seg.sh path-contain-shapenetcore_partanno_segmentation_benchmark_v0_normal log_seg 1 0
# Run distributed training
bash scripts/run_train_seg.sh path-contain-shapenetcore_partanno_segmentation_benchmark_v0_normal log_seg 8 0,1,2,3,4,5,6,7 hccl_8p_01234567_127.0.0.1.json
# Evaluate
bash scripts/run_eval_seg.sh [DATA_PATH] [CKPT_NAME]
# example:
bash scripts/run_eval_seg.sh path-contain-shapenetcore_partanno_segmentation_benchmark_v0_normal best_model.ckpt
├── PointTransformer
├── eval_cls.py # eval ModelNet40
├── eval_seg.py # eval ShapeNetCore
├── export.py
├── scripts
│ ├── run_eval_cls.sh # launch evaluating for ModelNet40
│ ├── run_eval_seg.sh # launch evaluating for ShapeNetCore
│ ├── run_train_cls.sh # launch training for ModelNet40
│ └── run_train_seg.sh # launch training for ShapeNetCore
├── src
│ ├── config
│ │ ├── config_cls.yaml # config file for ModelNet40
│ │ ├── config_seg.yaml # config file for ShapeNetCore
│ │ └── default.py # parse configuration
│ ├── dataset
│ │ ├── ModelNet.py # data preprocessing
│ │ └── ShapeNet.py # data preprocessing
│ ├── model
│ │ ├── point_helper.py # network definition utils
│ │ ├── pointTransfomrer_cls.py # network for classification
│ │ ├── pointTransformer.py # point transformer core
│ │ └── pointTransformerSeg.py # network for segmentation
│ └── utils
│ ├── callback.py # network for segmentation
│ ├── common.py # common preprocessing
│ ├── local_adapter.py # warp modelarts processing
│ ├── lr_scheduler.py # dynamic learning rate
│ └── metric.py # IOU for pointcloud segmentation
├── train_cls.py # train net
└── train_seg.py # train net
Major parameters in train.py are as follows:
--config_path # Path to Configuration yaml
--batch_size # Training batch size.
--epoch_size # Total training epochs.
--learning_rate # Training learning rate.
--weight_decay # Optimizer weight decay.
--dataset_path # The path to the train and evaluation datasets.
--save_checkpoint_path # The path to save files generated during training.
--use_normals # Whether to use normals data in training.
--pretrained_ckpt # The file path to load checkpoint.
--is_modelarts # Whether to use modelarts.
# Run training for classification with ModelNet40
bash scripts/run_train_cls.sh [DATASET_PATH] [LOG_PATH] [DEVICE_NUM] [VISIABLE_DEVICES(0,1,2,3,4,5,6,7)] [RANK_TABLE](optional)
# example Run stand-alone training:
bash scripts/run_train_cls.sh path-contain-modelnet40_normal_resampled log_cls 1 0
# Run distributed training
bash scripts/run_train_cls.sh path-contain-modelnet40_normal_resampled log_cls 8 0,1,2,3,4,5,6,7 hccl_8p_01234567_127.0.0.1.json
# Run training for segmentation with ShapeNetCore
bash scripts/run_train_seg.sh [DATASET_PATH] [LOG_PATH] [DEVICE_NUM] [VISIABLE_DEVICES(0,1,2,3,4,5,6,7)] [RANK_TABLE](optional)
# example Run stand-alone training:
bash scripts/run_train_seg.sh path-contain-shapenetcore_partanno_segmentation_benchmark_v0_normal log_seg 1 0
# Run distributed training
bash scripts/run_train_seg.sh path-contain-shapenetcore_partanno_segmentation_benchmark_v0_normal log_seg 8 0,1,2,3,4,5,6,7 hccl_8p_01234567_127.0.0.1.json
After training, the loss value will be achieved as follows:
# train log
epoch: 1 step: 94, loss is 2.88671875
epoch time: 951506.524 ms, per step time: 10122.410 ms
epoch: 2 step: 94, loss is 1.85546875
epoch time: 490416.995 ms, per step time: 5217.202 ms
...
The model checkpoint will be saved in the '[LOG_PATH/ckpt]' directory.
Before running the command below, please check the checkpoint path used for evaluation.
# Evaluate
bash scripts/run_eval_cls.sh [DATA_PATH] [CKPT_NAME]
# example:
bash scripts/run_eval_cls.sh path-contain-modelnet40_normal_resampled best_model.ckpt
# Evaluate
bash scripts/run_eval_seg.sh [DATA_PATH] [CKPT_NAME]
# example:
bash scripts/run_eval_seg.sh path-contain-shapenetcore_partanno_segmentation_benchmark_v0_normal best_model.ckpt
You can view the results through the file "eval.log". The accuracy of the test dataset will be as follows:
# classification
'Accuracy': 0.9254
# segmentation
ins. mIoU is 0.83, cat. mIoU is {cls_mIou}
Parameters | Ascend | GPU(Pytorch) |
---|---|---|
Model Version | Point Transformer | Point Transformer |
Resource | Ascend 910; CPU 24cores; Memory 256G; OS Euler2.8 | RTX 3090; GPU Memory 24268MiB; Ubuntu |
uploaded Date | ||
MindSpore Version | 1.5.0 | |
Dataset | ModelNet40 | ModelNet40 |
Training Parameters | epoch=200, steps=7600, batch_size=32, lr=0.05 | epoch=200, steps=82000, batch_size=24, lr=0.001 |
Optimizer | SGD | SGD |
Loss Function | CrossEntropy | CrossEntropy |
outputs | probability | probability |
Loss | 0.05 | 0.03 |
Speed | 3.8 s/step (8p) | 1.4 s/step (8p) |
Total time | 18.3 h (8p) | |
Checkpoint for Fine tuning | 81MB (.ckpt file) | 81MB (.pth file) |
Parameters | Ascend | GPU(Pytorch) |
---|---|---|
Model Version | Point Transformer | Point Transformer |
Resource | Ascend 910; CPU 24cores; Memory 256G; OS Euler2.8 | RTX 3090; GPU Memory 24268MiB; Ubuntu |
uploaded Date | ||
MindSpore Version | 1.5.0 | |
Dataset | ShapeNetCore | |
Training Parameters | epoch=200, steps=7600, batch_size=32, lr=0.05 | |
Optimizer | SGD | SGD |
Loss Function | CrossEntropy | CrossEntropy |
outputs | probability | probability |
Loss | 0.01 | |
Speed | 5.2 s/step (8p) | |
Total time | 1days | |
Checkpoint for Fine tuning | 136MB (.ckpt file) |
Parameters | Ascend | GPU |
---|---|---|
Model Version | Point Transformer | Point Transformer |
Resource | Ascend 910; CPU 24cores; Memory 256G; OS Euler2.8 | RTX 3090; GPU Memory 24268MiB; Ubuntu |
Uploaded Date | ||
MindSpore Version | 1.5.0 | 1.5.0 |
Dataset | ModelNet40 | ModelNet40 |
Batch_size | 32 | 32 |
Outputs | probability | probability |
Accuracy | 92.5% (1p) | |
Total time |
Parameters | Ascend | GPU |
---|---|---|
Model Version | Point Transformer | Point Transformer |
Resource | Ascend 910; CPU 24cores; Memory 256G; OS Euler2.8 | RTX 3090; GPU Memory 24268MiB; Ubuntu |
Uploaded Date | ||
MindSpore Version | 1.5.0 | 1.5.0 |
Dataset | ModelNet40 | ModelNet40 |
Batch_size | 32 | 32 |
Outputs | probability | probability |
mIoU | 84% (1p) | |
Total time |
Please check the official homepage.
No Description
Python Shell
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》