2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

PCTMA-Net: Point Cloud Transformer with Morphing Atlas-based Point Generation Network for Dense Point Cloud Completion

Jianjie Lin, Markus Rickert, Alexander Perzylo, Alois Knoll
Robotics, Artificial Intelligence and Real-time Systems
Department of Informatics, Technical University of Munich

DOI Preprint BibTeX Code

Abstract

Inferring a complete 3D geometry given an incomplete point cloud is essential in many vision and robotics applications. Previous work mainly relies on a global feature extracted by a Multi-layer Perceptron (MLP) for predicting the shape geometry. This suffers from a loss of structural details, as its point generator fails to capture the detailed topology and structure of point clouds using only the global features.The irregular nature of point clouds makes this task more challenging. This paper presents a novel method for shape completion to address this problem. The Transformer structure is currently a standard approach for natural language processing tasks and its inherent nature of permutation invariance makes it well suited for learning point clouds. Furthermore, the Transformer's attention mechanism can effectively capture the local context within a point cloud and efficiently exploit its incomplete local structure details. A morphing-atlas-based point generation network further fully utilizes the extracted point Transformer feature to predict the missing region using charts defined on the shape. Shape completion is achieved via the concatenation of all predicting charts on the surface. Extensive experiments on the Completion3D and KITTI data sets demonstrate that the proposed PCTMA-Net outperforms the state-of-the-art shape completion approaches and has a 10% relative improvement over the next best-performing method.

Qualitative comparison on Completion3D

We compare our proposed shape completion algorithm PCTMA-Net with other state-of-the-art approaches on two large scale data sets: Completion3D and KITTI. The Chamfer distance is employed as a metric in the evaluation.

Completion3D from ShapeNet offers a data set, which consists of 28974 training samples and 800 point cloud evaluation samples with a point resolution of 2048 for training and validation, respectively. The qualitative visualization of completion results shown in the figure below indicates that our approach is able to predict more details. The performance in the quantitative and qualitative evaluations proves the Transformer encoder and the morphing-atlas decoder's effectiveness for predicting and preserving the shape details.

Point completion results on Completion3D with ground truth and input resolution (2048 points) are compared using the Chamfer distance (CD) with L² norm. The results in the table below are multiplied by 10⁴. In our algorithm (PCTMA-Net), we set meshgrid=0.05. The best result is highlighted in bold, and a lower value is better.

Reference

For more detailed information please have a look at our IROS paper. The reference is

Jianjie Lin, Markus Rickert, Alexander Perzylo, Alois Knoll. PCTMA-Net: Point Cloud Transformer with Morphing Atlas-based Point Generation Network for Dense Point Cloud Completion, In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5657-5663, Prague, Czech Republic, September 2021.

@InProceedings{linpctma2021,
  author    = {Jianjie Lin and Markus Rickert and Alexander Perzylo and Alois Knoll},
  booktitle = {Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems},
  title     = {{PCTMA}-{N}et: Point Cloud Transformer with Morphing Atlas-based Point Generation Network for Dense Point Cloud Completion},
  year      = {2021},
  month     = sep,
  pages     = {5657--5663},
  address   = {Prague, Czech Republic},
  doi       = {10.1109/IROS51168.2021.9636483},
}