Article
Hole Repairing Algorithm for 3D Point Cloud Model of
Symmetrical Objects Grasped by the Manipulator
Linyan Cui, Guolong Zhang and Jinshen Wang *
Citation: Cui, L.; Zhang, G.; Wang, J.
Hole Repairing Algorithm for 3D
Point Cloud Model of Symmetrical
Objects Grasped by the Manipulator.
Sensors 2021, 21, 7558. https://
doi.org/10.3390/s21227558
Academic Editors: Yuansong Qiao
and Seamus Gordon
Received: 28 September 2021
Accepted: 12 November 2021
Published: 13 November 2021
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Image Processing Center, School of Astronautics, Beihang University, Beijing 102206, China;
cuily@buaa.edu.cn (L.C.); guolongzhang@buaa.edu.cn (G.Z.)
* Correspondence: wangjinshen@buaa.edu.cn
Abstract:
For the engineering application of manipulator grasping objects, mechanical arm occlusion
and limited imaging angle produce various holes in the reconstructed 3D point clouds of objects.
Acquiring a complete point cloud model of the grasped object plays a very important role in the
subsequent task planning of the manipulator. This paper proposes a method with which to automati-
cally detect and repair the holes in the 3D point cloud model of symmetrical objects grasped by the
manipulator. With the established virtual camera coordinate system and boundary detection, repair
and classification of holes, the closed boundaries for the nested holes were detected and classified
into two kinds, which correspond to the mechanical claw holes caused by mechanical arm occlusion
and the missing surface produced by limited imaging angle. These two kinds of holes were repaired
based on surface reconstruction and object symmetry. Experiments on simulated and real point cloud
models demonstrate that our approach outperforms the other state-of-the-art 3D point cloud hole
repair algorithms.
Keywords: 3D point cloud; hole repair; surface reconstruction; symmetry
1. Introduction
With the development of machine vision, robots can use “eyes” to obtain external
information directly like human beings, which improves the intelligence and autonomy
of robots. For example, the eye-to-hand camera can model the grasped object and the
surrounding environment with 3D reconstruction algorithms. The complete 3D models of
the grasped object are vital in the subsequent tasks, such as planning and controlling the
action of the manipulator.
However, due to the occlusion of the mechanical claw and the limited imaging angle
in 3D reconstruction, a large number of holes will appear in the point cloud model of the
grasped object. Figure 1 provides examples of the real-time reconstructed point cloud mod-
els of symmetrical objects grasped by the manipulator with the ElasticFusion method [
1
].
The occlusion of the mechanical claw comes from the fact that the mechanical claw and
the object are modeled together in the 3D reconstruction. After the mechanical claw is
removed from the 3D point cloud model, several holes with a regular shape and medium
size will be left on the side of the point cloud, whose shape is roughly equivalent to the
contour of the mechanical claw. Due to the limited imaging angle in 3D reconstruction,
the top of the object cannot be imaged when the manipulator grabs the object, resulting
in a missing surface at the top of the point cloud. The missing surface can be regarded
as a large area hole, and it exhibits different characteristics from the holes caused by the
occlusion of the mechanical claw.
In our practical engineering, the 3D reconstruction and the mechanical claw removal
are still in primary research; the high-quality point cloud models cannot be obtained for
complex objects. Therefore, we limit the objects grasped by the manipulator to symmetrical
objects, which can also provide more structural information for the hole repair. Based on
Sensors 2021, 21, 7558. https://doi.org/10.3390/s21227558 https://www.mdpi.com/journal/sensors
