Citation: Deng, X.; Qiu, S.; Jin, W.;
Xue, J. Three-Dimensional
Reconstruction Method for Bionic
Compound-Eye System Based on
MVSNet Network. Electronics 2022,
11, 1790. https://doi.org/10.3390/
electronics11111790
Academic Editors: Pedro Latorre-
Carmona, Filiberto Pla and Samuel
Morillas
Received: 7 May 2022
Accepted: 2 June 2022
Published: 5 June 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Three-Dimensional Reconstruction Method for Bionic
Compound-Eye System Based on MVSNet Network
Xinpeng Deng, Su Qiu *, Weiqi Jin and Jiaan Xue
MOE Key Laboratory of Optoelectronic Imaging Technology and System, School of Optics and Photonics,
Beijing Institute of Technology, Beijing 100081, China; 3120190523@bit.edu.cn (X.D.); jinwq@bit.edu.cn (W.J.);
3120185345@bit.edu.cn (J.X.)
* Correspondence: edmondqiu@bit.edu.cn
Abstract:
In practical scenarios, when shooting conditions are limited, high efficiency of image
shooting and success rate of 3D reconstruction are required. To achieve the application of bionic
compound eyes in small portable devices for 3D reconstruction, auto-navigation, and obstacle
avoidance, a deep learning method of 3D reconstruction using a bionic compound-eye system
with partial-overlap fields was studied. We used the system to capture images of the target scene,
then restored the camera parameter matrix by solving the PnP problem. Considering the unique
characteristics of the system, we designed a neural network based on the MVSNet network structure,
named CES-MVSNet. We fed the captured image and camera parameters to the trained deep neural
network, which can generate 3D reconstruction results with good integrity and precision. We used
the traditional multi-view geometric method and neural networks for 3D reconstruction, and the
difference between the effects of the two methods was analyzed. The efficiency and reliability of
using the bionic compound-eye system for 3D reconstruction are proved.
Keywords: bionic compound-eye system; 3D reconstruction; deep learning
1. Introduction
Three-dimensional reconstruction methods reconstruct dense 3D models from multi-
ple images, and their improvement is a fundamental problem in computer vision. These
methods have been extensively studied in recent decades. The traditional method mainly
comprises image feature extraction and matching, camera parameter estimation, triangula-
tion, and bundle adjustment [
1
]. Structure from motion (SfM) is a common method for 3D
reconstruction and is widely used in autonomous driving, mapping, military reconnais-
sance, and other fields.
SfM methods mainly include incremental, global, and hybrid methods [
2
]. Noah
et al. developed Bundler [
3
], which is a typical incremental system. It reconstructs large
scenes from a large internet image collection and exhibits good reconstruction accuracy and
stability. Schonberger et al. proposed COLMAP [
4
] to improve the incremental method by
introducing a geometric verification strategy and a best-view selection strategy to improve
the robustness of the system initialization and triangulation process; however, it requires
considerable computational time to achieve a complete and accurate 3D model [5].
The global method was designed to improve the computational efficiency and reduce
the accumulated drift error of the incremental method; however, it is less robust to image
mismatch, and the errors are accumulated and difficult to correct, which leads to low
reconstruction accuracy. The hybrid method combines the advantages of the incremental
and global methods. Cui et al. proposed the HSfM system [
6
], which uses the global
method when estimating the camera rotation matrix and the incremental method when
estimating the camera position, and then performs triangulation and bundle adjustment to
optimize the 3D model. Zhu et al. proposed a parallel SfM system [
7
] that can reconstruct
a city-scale scene containing millions of high-resolution images. The system decomposes
Electronics 2022, 11, 1790. https://doi.org/10.3390/electronics11111790 https://www.mdpi.com/journal/electronics
