https://github.com/NJUVISION/PCGCv1
Learned Point Cloud Geometry Compression
Jianqiang Wang, Hao Zhu, Haojie Liu, Zhan Ma. [arXiv]
PCGCv2: Multiscale Point CLoud Geometry Compression [arXiv] [github]
(a)
(b)
An illustrative overview in (a) and detailed diagram in (b) for point cloud geometry compression consisting of a pre-processing for PCG voxelization, scaling & partition, a compression network for compact PCG representation, and metadata signaling, and post-processing for PCG reconstruction and rendering. “Q” stands for “Quantization”, “AE” and “AD” are Arithmetic Encoder and Decoder respectively. “Conv” denotes the convolution layer with the number of the output, channels, and kernel size.
Requirements
Usage
Encoding
python test.py compress "testdata/8iVFB/longdress_vox10_1300.ply" \
--ckpt_dir="checkpoints/hyper/a6b3/"
下面是叶华的:
python test.py compress "/userhome/PCGCv1/testdata/8iVFB/redandblack_vox10_1550.ply"
--ckpt_dir="/userhome/PCGCv1/checkpoints/hyper/a10b3/"
python test.py compress "/userhome/PCGCv1/testdata/8iVFB/redandblack_vox10_1550.ply"
--ckpt_dir="/userhome/PCGCv1/checkpoints/factorized/a2b3/" --mode=factorized
python test.py compress "/userhome/PCGCv1/pytorch_eval/28_airplane_0270.ply"
--ckpt_dir="/userhome/PCGCv1/checkpoints/hyper/a10b3/"
Decoding
python test.py decompress "compressed/longdress_vox10_1300" \
--ckpt_dir="checkpoints/hyper/a6b3/"
下面是叶华的:
python test.py decompress "compressed/28_airplane_0270"
--ckpt_dir="/userhome/PCGCv1/checkpoints/hyper/a10b3/"
python test.py decompress "compressed/redandblack_vox10_1550"
--ckpt_dir="/userhome/PCGCv1/checkpoints/factorized/a2b3/" --mode=factorized
Other default options: --scale=1, --cube_size=64, --rho=1.0, --mode='hyper', --modelname='models.model_voxception'
Examples
Please refer to demo.ipynb
for each step.
Evaluation
python eval.py --input "testdata/8iVFB/longdress_vox10_1300.ply" \
--rootdir="results/hyper/" \
--cfgdir="results/hyper/8iVFB_vox10.ini" \
--res=1024
下面是叶华的:
注意修改--cfgdir文件里ckpt_dir的路径,否则会报错。
python eval.py --input "/userhome/PCGCv1/testdata/8iVFB/redandblack_vox10_1550.ply"
--rootdir="results/hyper/"
--cfgdir="/userhome/PCGCv1/results/hyper/8iVFB_vox10.ini"
--res=1024
打印出现这个不用管,它代码就是这么写的:decoding error on gpu !!!!!!!!!!!!!!!!!!!! 116.00733 !!!!!!!!!!!!!!!!!!!!
python eval.py --input "/userhome/PCGCv1/testdata/8iVFB/redandblack_vox10_1550.ply"
--rootdir="results/factorized/"
--cfgdir="/userhome/PCGCv1/results/factorized/cfg/redandblack_vox10_1550.ini"
--res=1024 --mode=factorized
Different parameters are required for different dataset, for example:
python eval.py --input "testdata/Sparse/House_without_roof_00057_vox12.ply" \
--rootdir "results/hyper/" \
--cfgdir "results/hyper/House_without_roof_00057_vox12.ini" \
--res=4096
下面是叶华的:
python eval.py --input "/userhome/PCGCv1/testdata/Sparse/House_without_roof_00057_vox12.ply"
--rootdir "/userhome/PCGCv1/results/hyper/"
--cfgdir "/userhome/PCGCv1/results/hyper/House_without_roof_00057_vox12.ini"
--res=4096
The detailed cfgs and results can be downloaded in https://box.nju.edu.cn/f/b78aeedc0453442aafe5/
And several examples of decoded point clouds can be download in https://box.nju.edu.cn/d/f6a6f8ae61c94cea9248/
Training
Generating training dataset
sampling points from meshes, here we use pyntcloud (pip install pyntcloud)
cd dataprocess
python mesh2pc.py
The output point clouds can be download in http://yun.nju.edu.cn/d/227493a5bd/ (todo)
python generate_dataset.py
the output training dataset can be download in http://yun.nju.edu.cn/d/604927e275/ (todo)
Training
python train_hyper.py --alpha=0.75 \
--prefix='hyper_' --batch_size=8 --init_ckpt_dir='checkpoints/hyper/a0.75b3' --reset_optimizer=1
下面是叶华的:
python train_hyper.py --alpha=10
--prefix='hyper_' --batch_size=8 --init_ckpt_dir='/userhome/PCGCv1/checkpoints/hyper/a10b3' --reset_optimizer=1
执行成功
or
python train_factorized.py --alpha=2 \
--prefix='voxception_' --batch_size=8 --init_ckpt_dir='./checkpoints/factorized/a2b3' --reset_optimizer=1
Comparison
Objective Comparison
results.ipynb
Qualitative Evaluation
Update
- 2019.10.09 initial smbmission.
- 2019.10.22 submit demos, several pretrained models and training datasets.
- 2019.10.27 submit all pretrained models and evaluate on 8i voxelized full bodies.
- 2019.11.14 check bug and start testing on AVS PCC Cat3.
- 2019.11.15 test point cloud sequences using avs metric tool. (thanks for the help from Wei Yan)
- 2019.11.19 finish testing AVS Cat3.
- 2019.11.20 test AVS Cat2.
- 2019.11.26 test AVS single frame.
- 2019.11.31 doc.
- 2020.06.27 python3 & clean up.
- 2020.10.03 open source.
- 2020.12.09 ablation studies & experiment configuration.
- 2020.12.16 add some examples of decoded point clouds.
Todo
- pytorch version & tensorflow2.0 version.
- training again.
Issues
- Error on GPU: sometimes the point clouds may fail to decode correctly due to the randomness on GPU.
You can test on CPU by setting os.environ['CUDA_VISIBLE_DEVICES']="" in "evel.py".
Or encode and decode at the same time, see "compress_hyper" in "transform.py".
Authors
These files are provided by Nanjing University Vision Lab. And thanks for the help from SJTU Cooperative Medianet Innovation Center. Please contact us (wangjq@smail.nju.edu.cn) if you have any questions.