RJXB软件学报Journal of Software1000-9825软件学报编辑部中国北京rjxb-34-4-196210.13328/j.cnki.jos.006683TP391计算机图形学与计算机辅助设计Computer Graphics and Computer Aided Design基于局部区域动态覆盖的3D点云分类方法3D Point Cloud Classification Method Based on Dynamic Coverage of Local Area王昌硕WANGChang-Shuo
The ability to describe local geometric shapes is very important for the representation of irregular point cloud. However, the existing network is still difficult to effectively capture accurate local shape information. This study simulates depthwise separable convolution calculation method in the point cloud and proposes a new type of convolution, namely dynamic cover convolution (DC-Conv), to aggregate local features. The core of DC-Conv is the space cover operator (SCOP), which constructs anisotropic spatial geometry in a local area to cover the local feature space to enhance the compactness of local features. DC-Conv achieves the capture of local shapes by dynamically combining multiple SCOPs in the local neighborhood. Among them, the attention coefficients of the SCOPs are adaptively learned from the point position in a data-driven way. Experiments on the 3D point cloud shape recognition benchmark dataset ModelNet40, ModelNet10, and ScanObjectNN show that this method can effectively improve the performance of 3D point cloud shape recognition and robustness to sparse point clouds even in the case of a single scale. Finally, sufficient ablation experiments are also provided to verify the effectiveness of the method. The open-source code is published at https://github.com/changshuowang/DC-CNN.
ReferencesDengWHuangKChenXSemantic RGB-D SLAM for rescue robot navigation2020822132022132910.1109/ACCESS.2020.3031867
Deng W, Huang K, Chen X, et al. Semantic RGB-D SLAM for rescue robot navigation. IEEE Access, 2020, 8: 221320-221329. [doi:10.1109/ACCESS.2020.3031867]
GulFRahimanWNazliAlhadySSA comprehensive study for robot navigation techniques201961163204610.1080/23311916.2019.1632046
Gul F, Rahiman W, NazliAlhady SS. A comprehensive study for robot navigation techniques. Cogent Engineering, 2019, 6(1): 1632046. [doi:10.1080/23311916.2019.1632046]
YurtseverELambertJCarballoAA survey of autonomous driving: Common practices and emerging technologies20208584435846910.1109/ACCESS.2020.2983149
Yurtsever E, Lambert J, Carballo A, et al. A survey of autonomous driving: Common practices and emerging technologies. IEEE Access, 2020, 8: 58443-58469. [doi:10.1109/ACCESS.2020.2983149]
GrigorescuSTrasneaBCociasTA survey of deep learning techniques for autonomous driving202037336238610.1002/rob.21918
Grigorescu S, Trasnea B, Cocias T, et al. A survey of deep learning techniques for autonomous driving. Journal of Field Robotics, 2020, 37(3): 362-386. [doi:10.1002/rob.21918]
ZhangJLinXAdvances in fusion of optical imagery and LiDAR point cloud applied to photogrammetry and remote sensing20178113110.1080/19479832.2016.1160960
Zhang J, Lin X. Advances in fusion of optical imagery and LiDAR point cloud applied to photogrammetry and remote sensing. Int'l Journal of Image and Data Fusion, 2017, 8(1): 1-31. [doi:10.1080/19479832.2016.1160960]
LiYMaJZhangYImage retrieval from remote sensing big data: A survey2021679411510.1016/j.inffus.2020.10.008
Li Y, Ma J, Zhang Y. Image retrieval from remote sensing big data: A survey. Information Fusion, 2021, 67: 94-115. [doi:10.1016/j.inffus.2020.10.008]
ZhuXXShahzadMFacade reconstruction using multiview spaceborne TomoSAR point clouds20135263541355210.1109/TGRS.2013.2273619
Zhu XX, Shahzad M. Facade reconstruction using multiview spaceborne TomoSAR point clouds. IEEE Trans. on Geoscience and Remote Sensing, 2013, 52(6): 3541-3552. [doi:10.1109/TGRS.2013.2273619]
Li L, Zhu S, Fu H, et al. End-to-end learning local multi-view descriptors for 3D point clouds. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2020. 1919-1928. [doi: 10.1109/cvpr42600.2020.00199]
Wei X, Yu R, Sun J. View-GCN: View-based graph convolutional network for 3D shape analysis. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2020. 1850-1859. [doi: 10.1109/cvpr42600.2020.00192]
Yang Z, Wang L. Learning relationships for multi-view 3D object recognition. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2019. 7505-7514. [doi: 10.1109/ICCV.2019.00760]
Maturana D, Scherer S. VoxNet: A 3D convolutional neural network for real-time object recognition. In: Proc. of the IEEE/RSJ Int'l Conf. on Intelligent Robots and Systems. 2015. 922-928. [doi: 10.1109/IROS.2015.7353481]
Riegler G, Ulusoy AO, Geiger A. Octnet: Learning deep 3D representations at high resolutions. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2017. 3577-3586. [doi: 10.1109/CVPR.2017.701]
Qi CR, Su H, Mo K, et al. Pointnet: Deep learning on point sets for 3D classification and segmentation. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2017. 652-660. [doi: 10.1109/CVPR.2017.16]
QiCRYiLSuHPointNet++: Deep hierarchical feature learning on point sets in a metric space20173050995108
Qi CR, Yi L, Su H, et al. PointNet++: Deep hierarchical feature learning on point sets in a metric space. Advances in Neural Information Processing Systems, 2017, 30: 5099-5108.
WangYSunYLiuZDynamic graph CNN for learning on point clouds201938511210.1145/3326362
Wang Y, Sun Y, Liu Z, et al. Dynamic graph CNN for learning on point clouds. ACM Trans. on Graphics, 2019, 38(5): 1-12. [doi:10.1145/3326362]
Wang C, Samari B, Siddiqi K. Local spectral graph convolution for point set feature learning. In: Proc. of the European Conf. on Computer Vision. 2018. 52-66. [doi: 10.1007/978-3-030-01225-0_4]
Komarichev A, Zhong Z, Hua J. A-CNN: Annularly convolutional neural networks on point clouds. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2019. 7421-7430. [doi: 10.1109/CVPR.2019.00760]
RosenblattFThe perceptron: A probabilistic model for information storage and organization in the brain195865638610.1037/H0042519
Rosenblatt F. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 1958, 65(6): 386. [doi:10.1037/H0042519]
NingXLiWJLiHGUncorrelated locality preserving discriminant analysis based on bionics201653112623262910.7544/issn1000-1239.2016.20150630
Ning X, Li WJ, Li HG, et al. Uncorrelated locality preserving discriminant analysis based on bionics. Journal of Computer Research and Development, 2016, 53(11): 2623-2629(in Chinese with English abstract). [doi:10.7544/issn1000-1239.2016. 20150630]
NingXLiWJTangBBULDP: Biomimetic uncorrelated locality discriminant projection for feature extraction in face recognition20182752575258610.1109/TIP.2018.2806229
Ning X, Li WJ, Tang B, et al. BULDP: Biomimetic uncorrelated locality discriminant projection for feature extraction in face recognition. IEEE Trans. on Image Processing, 2018, 27(5): 2575-2586. [doi:10.1109/TIP.2018.2806229]
Su H, Maji S, Kalogerakis E, et al. Multi-view convolutional neural networks for 3D shape recognition. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. 945-953. [doi: 10.1109/ICCV.2015.114]
Klokov R, Lempitsky V. Escape from cells: Deep kd-networks for the recognition of 3D point cloud models. In: Proc. of the IEEE Int'l Conf. on Computer Vision. 2017. 863-872. [doi: 10.1109/ICCV.2017.99]
GuoMHCaiJXLiuZNPCT: Point cloud transformer20217218719910.1007/s41095-021-0229-5
Guo MH, Cai JX, Liu ZN, et al. PCT: Point cloud transformer. Computational Visual Media, 2021, 7(2): 187-199. [doi:10.1007/s41095-021-0229-5]
Xiang T, Zhang C, Song Y, et al. Walk in the cloud: Learning curves for point clouds shape analysis. In: Proc. of the IEEE/CVF Int'l Conf. on Computer Vision. 2021. 915-924.
Duan Y, Zheng Y, Lu J, et al. Structural relational reasoning of point clouds. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2019. 949-958. [doi: 10.1109/CVPR.2019.00104]
Hu Q, Yang B, Xie L, et al. Randla-net: Efficient semantic segmentation of large-scale point clouds. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2020. 11108-11117. [doi: 10.1109/CVPR42600.2020.01112]
Zhao H, Jiang L, Fu CW, et al. PointWeb: Enhancing local neighborhood features for point cloud processing. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2019. 5565-5573. [doi: 10.1109/CVPR.2019.00571]
Yan X, Zheng C, Li Z, et al. Pointasnl: Robust point clouds processing using nonlocal neural networks with adaptive sampling. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2020. 5589-5598. [doi: 10.1109/cvpr42600.2020.00563]
LiYBuRSunMPointcnn: Convolution on x-transformed points201831820830
Li Y, Bu R, Sun M, et al. Pointcnn: Convolution on x-transformed points. Advances in Neural Information Processing Systems, 2018, 31: 820-830.
Wu W, Qi Z, Fuxin L. Pointconv: Deep convolutional networks on 3D point clouds. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2019. 9621-9630. [doi: 10.1109/CVPR.2019.00985]
Thomas H, Qi CR, Deschaud JE, et al. KPConv: Flexible and deformable convolution for point clouds. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2019. 6411-6420. [doi: 10.1109/ICCV.2019.00651]
Liu Y, Fan B, Xiang S, et al. Relation-shape convolutional neural network for point cloud analysis. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2019. 8895-8904. [doi: 10.1109/CVPR.2019.00910]
Qi X, Liao R, Jia J, et al. 3D graph neural networks for RGBD semantic segmentation. In Proc. of the IEEE Int'l Conf. on Computer Vision. 2017. 5199-5208. [doi: 10.1109/ICCV.2017.556]
Simonovsky M, Komodakis N. Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2017. 3693-3702. [doi: 10.1109/CVPR.2017.11]
Chen Y, Dai X, Liu M, et al. Dynamic convolution: Attention over convolution kernels. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2020. 11030-11039. [doi: 10.1109/CVPR42600.2020.01104]
YangBBenderGLeQVCondConv: Conditionally parameterized convolutions for efficient inference20193213071318
Yang B, Bender G, Le QV, et al. CondConv: Conditionally parameterized convolutions for efficient inference. Advances in Neural Information Processing Systems, 2019, 32: 1307-1318.
Chen J, Wang X, Guo Z, et al. Dynamic region-aware convolution. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2021. 8064-8073. [doi: 10.1109/CVPR46437.2021.00797]
Xu M, Ding R, Zhao H, et al. PAConv: Position adaptive convolution with dynamic kernel assembling on point clouds. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2021. 3173-3182. [doi: 10.1109/CVPR46437.2021.00319]
Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. In: Proc. of the European Conf. on Computer Vision. Cham: Springer, 2014. 818-833. [doi: 10.1007/978-3-319-10590-1_53]
Kaiser L, Gomez AN, Chollet F. Depthwise separable convolutions for neural machine translation. In: Proc. of the Int'l Conf. on Learning Representations. 2018.
MoodyJDarkenCJFast learning in networks of locally-tuned processing units19891228129410.1162/neco.1989.1.2.281
Moody J, Darken CJ. Fast learning in networks of locally-tuned processing units. Neural Computation, 1989, 1(2): 281-294. [doi:10.1162/neco.1989.1.2.281]
Wu Z, Song S, Khosla A, et al. 3D shapenets: A deep representation for volumetric shapes. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. 1912-1920. [doi: 10.1109/CVPR.2015.7298801]
Hua BS, Tran MK, Yeung SK. Pointwise convolutional neural networks. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2018. 984-993. [doi: 10.1109/CVPR.2018.00109]
Li J, Chen BM, Lee GH. So-net: Self-organizing network for point cloud analysis. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2018. 9397-9406. [doi: 10.1109/CVPR.2018.00979]
AtzmonMMaronHLipmanYPoint convolutional neural networks by extension operators20183711210.1145/3197517.3201301
Atzmon M, Maron H, Lipman Y. Point convolutional neural networks by extension operators. ACM Trans. on Graphics, 2018, 37: 1-12. [doi:10.1145/3197517.3201301]
Lin Y, Yan Z, Huang H, et al. FPconv: Learning local flattening for point convolution. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2020. 4293-4302. [doi: 10.1109/cvpr42600.2020.00435]
Han XF, Jin YF, Cheng HX, et al. Dual transformer for point cloud analysis. arXiv: 2104.13044, 2021.
Mao J, Wang X, Li H. Interpolated convolutional networks for 3D point cloud understanding. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2019. 1578-1587. [doi: 10.1109/ICCV.2019.00166]
Liu Z, Hu H, Cao Y, et al. A closer look at local aggregation operators in point cloud analysis. In: Proc. of the European Conf. on Computer Vision. Cham: Springer, 2020. 326-342. [doi: 10.1007/978-3-030-58592-1_20]
Liu Y, Fan B, Meng G, et al. Densepoint: Learning densely contextual representation for efficient point cloud processing. In: Proc. of the IEEE/CVF Int'l Conf. on Computer Vision. 2019. 5239-5248. [doi: 10.1109/ICCV.2019.00534]
Xu M, Zhou Z, Qiao Y. Geometry sharing network for 3D point cloud classification and segmentation. In: Proc. of the AAAI Conf. on Artificial Intelligence. 2020, 34(7): 12500-12507. [doi: 10.1609/AAAI.V34I07.6938]
Zhang J, Chen L, Ouyang B, et al. PointCutMix: Regularization strategy for point cloud classification. arXiv: 2101.01461, 2021.
Lee D, Lee J, Lee J, et al. Regularization strategy for point cloud via rigidly mixed sample. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition. 2021. 15900-15909. [doi: 10.1109/CVPR46437.2021.01564]
Ben-ShabatYLindenbaumMFischerA3DMFV: Three-dimensional point cloud classification in real-time using convolutional neural networks2018343145315210.1109/LRA.2018.2850061
Ben-Shabat Y, Lindenbaum M, Fischer A. 3DMFV: Three-dimensional point cloud classification in real-time using convolutional neural networks. IEEE Robotics and Automation Letters, 2018, 3(4): 3145-3152. [doi:10.1109/LRA.2018.2850061]
Xu Y, Fan T, Xu M, et al. Spidercnn: Deep learning on point sets with parameterized convolutional filters. In: Proc. of the European Conf. on Computer Vision. 2018. 87-102. [doi: 10.1007/978-3-030-01237-3_6]
Uy MA, Pham QH, Hua BS, et al. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In: Proc. of the IEEE/CVF Int'l Conf. on Computer Vision and Pattern Recognition. 2019. 1588-1597. [doi: 10.1109/ICCV.2019.00167]
Qiu S, Anwar S, Barnes N. Dense-resolution network for point cloud classification and segmentation. In: Proc. of the IEEE/CVF Winter Conf. on Applications of Computer Vision. 2021. 3813-3822. [doi: 10.1109/WACV48630.2021.00386]
QiuSAnwarSBarnesNGeometric back-projection network for point cloud classification202110.1109/TMM.2021.3074240
Qiu S, Anwar S, Barnes N. Geometric back-projection network for point cloud classification. IEEE Trans. on Multimedia, 2021. [doi:10.1109/TMM.2021.3074240]
Goyal A, Law H, Liu B, et al. Revisiting point cloud shape classification with a simple and effective baseline. In: Proc. of the Int'l Conf. on Machine Learning. PMLR, 2021.
ChengSChenXHeXPra-net: Point relation-aware network for 3D point cloud analysis2021304436444810.1109/TIP.2021.3072214
Cheng S, Chen X, He X, et al. Pra-net: Point relation-aware network for 3D point cloud analysis. IEEE Trans. on Image Processing, 2021, 30: 4436-4448. [doi:10.1109/TIP.2021.3072214]