post_process
Point cloud compression, point cloud post-processing, geometry refinement
This is a novel learning-based post-processing method to enhance the decompressed point clouds, which can deal with decompressed point clouds with huge variety of distortions using a single model. The source code uses pytorch as deeplearning framework, here we transplant to TensorFlow.
Paper Citation:
Xiaoqing Fan, Ge Li, Dingquan Li, Yurui Ren, Wei Gao, Thomas H. Li, “Deep Geometry Post-Processing for Decompressed Point Clouds,” IEEE International Conference on Multimedia and Expo (ICME), 2022.
our contributions
1.transplant from pytorch to TensorFlow.
2.benchmark tests on many PCs, including those not tested by author.
3.compare performances between pytorch and tensorflow.
files
1.Deep Geometry Post Processing for Decompressed Point Clouds.pdf: Paper
2.postprocess_pytorch: source code and models on pytorch
3.tensorflow: source code and models on TensorFlow
4.train-sets: go to dataset
environment
- python 3.6.9
- cuda 10.2
- pytorch 1.8.1 or tensorflow-gpu 2.3.1
command
1.pytorch
training:
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 --master_port 12345 train.py --config ./config/multi_scale.yaml --name baseline
test:
CUDA_VISIBLE_DEVICES=0 python -m torch.distributed.launch --nproc_per_node=1 --master_port 6789 test.py --config ./config/multi_scale_all.yaml --name baseline --calculate_psnr --test_list ./test_78/test_10bit.txt --single_gpu
2.tensorflow
training:
on single GPU
python train.py
on multi GPUs
python train_multiGPUs.py
test:
python test.py
performance
We perform tests on PC files from 3 publicly available datasets(for more details, pls go to: https://git.openi.org.cn/OpenPointCloud/PCC_benchmark_testsets).
For these PCs, we first carry out compression and decompression on GPCC octree and trisoup on different rates. And then impletement post-process to improve quality. From Table 1 below, we can see that post-process method works in variable compression rates and PC bit-widths of 9bit, 10bit, 11bit. It remarkably improves the quality of reconstructed PCs, except for those from lossless compression.
For D1 and D2, TensorFlow and Pytorch nearly have same performances. But Pytorch uses much more time than TensorFlow, 4-8 times. One reason is that the 3D convolution operation in TensorFlow is faster than Pytorch. Another reason is that the version in TensorFlow is baseline, as there is no 3D interpolation operators in it, which can save time but with little influence on performance.
Table 1.
file |
compression |
rate |
ori d1_PSNR |
ori d2_PSNR |
TF_d1_psnr |
TF_d2_psnr |
TF_time |
PT_d1_psnr |
PT_d2_psnr |
PT_time |
basketball_player_vox11_00000200_dec.ply |
octree |
r01 |
52.88 |
55.089 |
59.971 |
60.605 |
32.68 |
59.891 |
60.257 |
194.92 |
basketball_player_vox11_00000200_dec.ply |
octree |
r02 |
58.814 |
61.529 |
69.285 |
70.69 |
31.62 |
69.716 |
70.85 |
190.69 |
basketball_player_vox11_00000200_dec.ply |
octree |
r03 |
64.459 |
68.141 |
76.147 |
78.685 |
32.59 |
76.394 |
78.916 |
203.65 |
basketball_player_vox11_00000200_dec.ply |
octree |
r04 |
69.234 |
74.351 |
80.956 |
84.325 |
39.62 |
81.078 |
84.226 |
240.14 |
basketball_player_vox11_00000200_dec.ply |
octree |
r05 |
74.002 |
79.191 |
83.586 |
87.142 |
59.93 |
83.699 |
87.078 |
279.02 |
basketball_player_vox11_00000200_dec.ply |
octree |
r06 |
inf |
inf |
67.19 |
67.756 |
59.93 |
64.766 |
65.205 |
291.68 |
basketball_player_vox11_00000200_dec.ply |
trisoup |
r01 |
59.441 |
64.181 |
53.847 |
54.156 |
52.06 |
51.581 |
51.876 |
307.68 |
basketball_player_vox11_00000200_dec.ply |
trisoup |
r02 |
67.874 |
71.649 |
59.878 |
60.233 |
53.77 |
54.929 |
55.227 |
434.38 |
basketball_player_vox11_00000200_dec.ply |
trisoup |
r03 |
72.442 |
75.537 |
64.846 |
65.274 |
55.24 |
57.912 |
58.203 |
418.1 |
basketball_player_vox11_00000200_dec.ply |
trisoup |
r04 |
73.673 |
76.653 |
67.163 |
67.703 |
58.2 |
61.555 |
61.887 |
420.93 |
dancer_vox11_00000001_dec.ply |
octree |
r01 |
52.897 |
55.263 |
59.396 |
59.749 |
27.68 |
59.343 |
59.695 |
263.17 |
dancer_vox11_00000001_dec.ply |
octree |
r02 |
58.808 |
61.653 |
68.742 |
69.85 |
27.11 |
69.066 |
70.003 |
382.99 |
dancer_vox11_00000001_dec.ply |
octree |
r03 |
64.462 |
68.282 |
75.691 |
78.059 |
28.78 |
75.998 |
77.977 |
533.88 |
dancer_vox11_00000001_dec.ply |
octree |
r04 |
69.232 |
74.556 |
80.703 |
83.834 |
34.46 |
80.809 |
83.823 |
608.91 |
dancer_vox11_00000001_dec.ply |
octree |
r05 |
74.004 |
79.204 |
83.336 |
86.521 |
42.2 |
83.581 |
86.717 |
644.06 |
dancer_vox11_00000001_dec.ply |
octree |
r06 |
inf |
inf |
66.823 |
67.356 |
52.83 |
63.996 |
64.411 |
639.12 |
dancer_vox11_00000001_dec.ply |
trisoup |
r01 |
59.285 |
63.771 |
54.13 |
54.432 |
47.17 |
51.68 |
51.965 |
361.1 |
dancer_vox11_00000001_dec.ply |
trisoup |
r02 |
67.644 |
71.633 |
59.763 |
60.092 |
49.5 |
55.324 |
55.626 |
363.53 |
dancer_vox11_00000001_dec.ply |
trisoup |
r03 |
72.648 |
75.733 |
64.592 |
64.991 |
49.5 |
58.048 |
58.342 |
277.59 |
dancer_vox11_00000001_dec.ply |
trisoup |
r04 |
73.965 |
76.896 |
68.043 |
68.64 |
54.63 |
62.281 |
62.624 |
227.42 |
longdress_vox10_1300_dec.ply |
octree |
r01 |
52.777 |
55.653 |
61.836 |
63.68 |
9.92 |
62.467 |
64.285 |
47.22 |
longdress_vox10_1300_dec.ply |
octree |
r02 |
58.442 |
62.248 |
68.714 |
71.516 |
9.54 |
69.253 |
71.209 |
54.43 |
longdress_vox10_1300_dec.ply |
octree |
r03 |
63.212 |
68.463 |
74.12 |
77.468 |
11.73 |
74.253 |
77.452 |
60.5 |
longdress_vox10_1300_dec.ply |
octree |
r04 |
66.771 |
71.931 |
71.086 |
73.826 |
13.29 |
71.03 |
73.619 |
63.44 |
longdress_vox10_1300_dec.ply |
octree |
r05 |
69.679 |
74.743 |
78.782 |
82.482 |
16.02 |
79.015 |
82.538 |
69.31 |
longdress_vox10_1300_dec.ply |
octree |
r06 |
inf |
inf |
63.304 |
63.996 |
18.3 |
60.211 |
60.682 |
72.55 |
longdress_vox10_1300_dec.ply |
trisoup |
r01 |
51.562 |
54.717 |
47.043 |
47.279 |
16.2 |
45.306 |
45.582 |
65.45 |
longdress_vox10_1300_dec.ply |
trisoup |
r02 |
59.819 |
61.9 |
52.198 |
52.518 |
16.76 |
49.984 |
50.294 |
65.68 |
longdress_vox10_1300_dec.ply |
trisoup |
r03 |
65.995 |
68.91 |
59.923 |
60.472 |
16.59 |
54.914 |
55.285 |
74.31 |
longdress_vox10_1300_dec.ply |
trisoup |
r04 |
67.904 |
71.038 |
62.066 |
62.65 |
17.73 |
57.415 |
57.823 |
65.83 |
loot_vox10_1200_dec.ply |
octree |
r01 |
52.804 |
55.653 |
66.807 |
69.002 |
9.15 |
68.021 |
71.052 |
46.47 |
loot_vox10_1200_dec.ply |
octree |
r02 |
58.439 |
62.217 |
70.8 |
74.162 |
9.34 |
71.305 |
74.727 |
48.3 |
loot_vox10_1200_dec.ply |
octree |
r03 |
63.21 |
68.45 |
74.483 |
77.777 |
11.37 |
74.636 |
77.915 |
61.63 |
loot_vox10_1200_dec.ply |
octree |
r04 |
66.775 |
71.923 |
71.068 |
73.711 |
12.36 |
70.989 |
73.476 |
63.33 |
loot_vox10_1200_dec.ply |
octree |
r05 |
69.706 |
74.79 |
79.035 |
82.636 |
15.42 |
79.233 |
82.647 |
78.14 |
loot_vox10_1200_dec.ply |
octree |
r06 |
inf |
inf |
61.508 |
62.092 |
17.23 |
58.346 |
58.722 |
74.35 |
loot_vox10_1200_dec.ply |
trisoup |
r01 |
51.727 |
56.117 |
46.348 |
46.583 |
15.04 |
44.696 |
44.922 |
69.08 |
loot_vox10_1200_dec.ply |
trisoup |
r02 |
60.182 |
64.017 |
50.428 |
50.73 |
15.58 |
48.577 |
48.867 |
71.72 |
loot_vox10_1200_dec.ply |
trisoup |
r03 |
65.828 |
69.304 |
57.214 |
57.54 |
15.88 |
52.618 |
52.888 |
70.69 |
loot_vox10_1200_dec.ply |
trisoup |
r04 |
67.901 |
71.016 |
60.13 |
60.575 |
17.22 |
55.063 |
55.35 |
71.63 |
queen_vox10_0200_dec.ply |
octree |
r01 |
52.805 |
56.04 |
61.371 |
62.129 |
9.55 |
61.804 |
62.589 |
44.77 |
queen_vox10_0200_dec.ply |
octree |
r02 |
58.432 |
62.662 |
68.981 |
71.077 |
9.87 |
69.193 |
71.222 |
54.11 |
queen_vox10_0200_dec.ply |
octree |
r03 |
63.21 |
69.048 |
74.051 |
77.041 |
11.9 |
73.789 |
76.518 |
62.68 |
queen_vox10_0200_dec.ply |
octree |
r04 |
66.777 |
72.308 |
71.935 |
74.54 |
13.13 |
71.219 |
73.421 |
66.28 |
queen_vox10_0200_dec.ply |
octree |
r05 |
69.707 |
75.013 |
78.375 |
81.762 |
18.51 |
78.737 |
82.179 |
70.25 |
queen_vox10_0200_dec.ply |
octree |
r06 |
inf |
inf |
64.489 |
65.252 |
19.46 |
62.451 |
63.011 |
73.96 |
queen_vox10_0200_dec.ply |
trisoup |
r01 |
51.64 |
56.169 |
44.75 |
45.029 |
15.99 |
42.608 |
42.885 |
62.5 |
queen_vox10_0200_dec.ply |
trisoup |
r02 |
60.665 |
63.629 |
50.969 |
51.289 |
17.41 |
45.857 |
46.133 |
70.9 |
queen_vox10_0200_dec.ply |
trisoup |
r03 |
65.763 |
69.252 |
56.914 |
57.256 |
17.4 |
48.849 |
49.11 |
68.48 |
queen_vox10_0200_dec.ply |
trisoup |
r04 |
67.568 |
70.877 |
62.553 |
63.082 |
18.54 |
53.147 |
53.413 |
70 |
redandblack_vox10_1550_dec.ply |
octree |
r01 |
52.798 |
55.674 |
60.964 |
62.389 |
8.58 |
61.384 |
62.338 |
49.8 |
redandblack_vox10_1550_dec.ply |
octree |
r02 |
58.435 |
62.278 |
67.978 |
70.834 |
9.29 |
68.138 |
70.922 |
54.28 |
redandblack_vox10_1550_dec.ply |
octree |
r03 |
63.212 |
68.536 |
73.573 |
76.901 |
10.6 |
73.61 |
76.802 |
56.06 |
redandblack_vox10_1550_dec.ply |
octree |
r04 |
66.779 |
71.972 |
71.003 |
73.855 |
11.48 |
70.839 |
73.44 |
60.77 |
redandblack_vox10_1550_dec.ply |
octree |
r05 |
69.7 |
74.816 |
78.099 |
81.897 |
14.41 |
78.427 |
82.112 |
64.99 |
redandblack_vox10_1550_dec.ply |
octree |
r06 |
inf |
inf |
66.609 |
68.088 |
16.22 |
63.359 |
64.182 |
68.79 |
redandblack_vox10_1550_dec.ply |
trisoup |
r01 |
49.762 |
54.11 |
45.41 |
46.264 |
13.65 |
44.206 |
44.838 |
63.68 |
redandblack_vox10_1550_dec.ply |
trisoup |
r02 |
59.033 |
62.101 |
52.067 |
53.037 |
14.28 |
49.695 |
50.539 |
67.42 |
redandblack_vox10_1550_dec.ply |
trisoup |
r03 |
65.42 |
68.83 |
58.077 |
59.096 |
14.95 |
53.857 |
54.368 |
66.74 |
redandblack_vox10_1550_dec.ply |
trisoup |
r04 |
67.868 |
71.17 |
65.541 |
66.728 |
15.9 |
59.014 |
59.497 |
69.32 |
soldier_vox10_0690_dec.ply |
octree |
r01 |
52.799 |
55.572 |
61.148 |
62.507 |
12.33 |
61.431 |
62.456 |
61.97 |
soldier_vox10_0690_dec.ply |
octree |
r02 |
58.431 |
62.177 |
68.508 |
71.043 |
12.06 |
68.773 |
71.185 |
67.57 |
soldier_vox10_0690_dec.ply |
octree |
r03 |
63.209 |
68.517 |
72.608 |
74.624 |
14.83 |
71.194 |
72.542 |
79.22 |
soldier_vox10_0690_dec.ply |
octree |
r04 |
66.773 |
71.935 |
71.156 |
73.939 |
19.13 |
71.045 |
73.691 |
85.34 |
soldier_vox10_0690_dec.ply |
octree |
r05 |
69.701 |
74.814 |
78.789 |
82.38 |
20.45 |
79.008 |
82.468 |
88.68 |
soldier_vox10_0690_dec.ply |
octree |
r06 |
inf |
inf |
64.707 |
65.696 |
23.03 |
62.119 |
62.757 |
93.81 |
soldier_vox10_0690_dec.ply |
trisoup |
r01 |
51.197 |
55.015 |
47.648 |
47.89 |
26.02 |
46.034 |
46.246 |
82.97 |
soldier_vox10_0690_dec.ply |
trisoup |
r02 |
60.46 |
62.577 |
53.498 |
53.76 |
20.72 |
50.59 |
50.834 |
88.82 |
soldier_vox10_0690_dec.ply |
trisoup |
r03 |
65.872 |
68.97 |
60.278 |
60.715 |
21.7 |
55.345 |
55.625 |
85.01 |
soldier_vox10_0690_dec.ply |
trisoup |
r04 |
67.952 |
71.171 |
65.195 |
66.161 |
22.92 |
61.645 |
62.17 |
93.17 |
phil_vox9_0139_dec.ply |
octree |
r01 |
46.761 |
50.065 |
54.211 |
55.838 |
6.98 |
54.496 |
56.218 |
21.22 |
phil_vox9_0139_dec.ply |
octree |
r02 |
52.409 |
56.843 |
61.611 |
63.989 |
8.3 |
61.025 |
63.838 |
25.18 |
phil_vox9_0139_dec.ply |
octree |
r03 |
57.177 |
63.362 |
66.145 |
69.64 |
8.07 |
66.084 |
69.532 |
24.62 |
phil_vox9_0139_dec.ply |
octree |
r04 |
60.742 |
66.298 |
65.409 |
68.521 |
9.28 |
65.333 |
68.408 |
24.83 |
phil_vox9_0139_dec.ply |
octree |
r05 |
63.663 |
68.879 |
69.836 |
74.166 |
9.68 |
70.714 |
74.583 |
28.82 |
phil_vox9_0139_dec.ply |
octree |
r06 |
inf |
inf |
59.837 |
61.212 |
11.49 |
56.559 |
57.392 |
29.97 |
phil_vox9_0139_dec.ply |
trisoup |
r01 |
43.213 |
51.119 |
37.567 |
38.621 |
8.91 |
35.491 |
36.487 |
21.73 |
phil_vox9_0139_dec.ply |
trisoup |
r02 |
51.415 |
57.562 |
43.627 |
44.408 |
10.81 |
38.678 |
39.589 |
26.51 |
phil_vox9_0139_dec.ply |
trisoup |
r03 |
57.03 |
62.529 |
49.409 |
50.061 |
9.83 |
42.217 |
43.082 |
27.51 |
phil_vox9_0139_dec.ply |
trisoup |
r04 |
61.265 |
65.098 |
57.594 |
58.571 |
9.5 |
50.109 |
50.965 |
28.43 |
phil_vox10_0139_dec.ply |
octree |
r01 |
52.794 |
56.27 |
60.819 |
62.216 |
27.14 |
61.143 |
62.728 |
91.28 |
phil_vox10_0139_dec.ply |
octree |
r02 |
58.43 |
63.037 |
67.642 |
70.007 |
28.3 |
67.451 |
69.727 |
99.6 |
phil_vox10_0139_dec.ply |
octree |
r03 |
63.208 |
69.655 |
71.899 |
75.27 |
31.52 |
71.907 |
75.238 |
104.03 |
phil_vox10_0139_dec.ply |
octree |
r04 |
66.778 |
72.513 |
71.389 |
74.481 |
35.02 |
71.198 |
74.117 |
114.1 |
phil_vox10_0139_dec.ply |
octree |
r05 |
69.696 |
75.042 |
75.642 |
79.469 |
40.49 |
76.265 |
79.961 |
122.14 |
phil_vox10_0139_dec.ply |
octree |
r06 |
inf |
inf |
66.762 |
68.317 |
42.96 |
65.934 |
67.325 |
133.69 |
phil_vox10_0139_dec.ply |
trisoup |
r01 |
50.011 |
58.232 |
44.364 |
45.693 |
34.83 |
41.352 |
42.428 |
106.65 |
phil_vox10_0139_dec.ply |
trisoup |
r02 |
56.32 |
64.292 |
47.569 |
49.052 |
35.94 |
43.341 |
44.408 |
110.4 |
phil_vox10_0139_dec.ply |
trisoup |
r03 |
63.55 |
68.619 |
51.957 |
53.463 |
41.75 |
46.616 |
47.639 |
116.45 |
phil_vox10_0139_dec.ply |
trisoup |
r04 |
67.181 |
71.228 |
62.054 |
63.252 |
43.07 |
55.297 |
56.221 |
119.81 |
sarah_vox9_0023_dec.ply |
octree |
r01 |
46.787 |
50.143 |
54.418 |
56.486 |
11.77 |
54.944 |
56.545 |
20.84 |
sarah_vox9_0023_dec.ply |
octree |
r02 |
52.414 |
56.826 |
61.785 |
64.15 |
13.06 |
61.714 |
64.12 |
21.42 |
sarah_vox9_0023_dec.ply |
octree |
r03 |
57.173 |
63.116 |
66.292 |
69.845 |
8.64 |
66.358 |
69.797 |
23.39 |
sarah_vox9_0023_dec.ply |
octree |
r04 |
60.749 |
66.179 |
65.492 |
68.596 |
12.45 |
65.376 |
68.325 |
24.83 |
sarah_vox9_0023_dec.ply |
octree |
r05 |
63.689 |
68.876 |
70.472 |
74.557 |
10.92 |
70.626 |
74.882 |
26.86 |
sarah_vox9_0023_dec.ply |
octree |
r06 |
inf |
inf |
61.853 |
63.88 |
8.98 |
57.771 |
59.106 |
27.85 |
sarah_vox9_0023_dec.ply |
trisoup |
r01 |
42.602 |
51.165 |
37.077 |
37.952 |
7.44 |
35.177 |
36.009 |
21.66 |
sarah_vox9_0023_dec.ply |
trisoup |
r02 |
47.139 |
58.006 |
40.092 |
41.136 |
8.38 |
36.925 |
37.831 |
23.02 |
sarah_vox9_0023_dec.ply |
trisoup |
r03 |
57.591 |
62.562 |
44.735 |
46.068 |
17.33 |
39.549 |
40.31 |
24.06 |
sarah_vox9_0023_dec.ply |
trisoup |
r04 |
61.059 |
64.699 |
56.53 |
57.93 |
8.69 |
45.462 |
46.064 |
26.85 |
sarah_vox10_0023_dec.ply |
octree |
r01 |
52.796 |
56.255 |
61.064 |
62.571 |
24.32 |
61.137 |
62.621 |
78.18 |
sarah_vox10_0023_dec.ply |
octree |
r02 |
58.435 |
63.031 |
67.888 |
70.273 |
24.13 |
67.857 |
70.109 |
82.52 |
sarah_vox10_0023_dec.ply |
octree |
r03 |
63.208 |
69.386 |
71.765 |
75.237 |
26.4 |
70.193 |
72.414 |
92.46 |
sarah_vox10_0023_dec.ply |
octree |
r04 |
66.775 |
72.338 |
71.473 |
74.476 |
28.79 |
71.327 |
74.237 |
98.15 |
sarah_vox10_0023_dec.ply |
octree |
r05 |
69.698 |
74.954 |
75.891 |
79.678 |
33.64 |
76.499 |
80.122 |
106.9 |
sarah_vox10_0023_dec.ply |
octree |
r06 |
inf |
inf |
64.525 |
66.232 |
37.32 |
61.153 |
62.355 |
117.61 |
sarah_vox10_0023_dec.ply |
trisoup |
r01 |
46.004 |
58.526 |
43.974 |
45.45 |
29.53 |
41.23 |
42.25 |
90.23 |
sarah_vox10_0023_dec.ply |
trisoup |
r02 |
56.572 |
64.701 |
45.962 |
47.72 |
32.95 |
42.533 |
43.635 |
100.29 |
sarah_vox10_0023_dec.ply |
trisoup |
r03 |
63.609 |
68.535 |
50.849 |
52.515 |
32.86 |
45.525 |
46.534 |
102.58 |
sarah_vox10_0023_dec.ply |
trisoup |
r04 |
67.138 |
70.862 |
59.747 |
61.086 |
36.69 |
52.614 |
53.527 |
108.36 |
contributors
name: Ye Hua
email: yeh@pcl.ac.cn