Simultaneous Material Segmentation and 3D Reconstruction in Industrial Scenarios

Zhao, Cheng and Sun, Li and Stolkin, Rustam (2020) Simultaneous Material Segmentation and 3D Reconstruction in Industrial Scenarios. Frontiers in Robotics and AI, 7. ISSN 2296-9144

[thumbnail of pubmed-zip/versions/1/package-entries/frobt-07-00052/frobt-07-00052.pdf] Text
pubmed-zip/versions/1/package-entries/frobt-07-00052/frobt-07-00052.pdf - Published Version

Download (1MB)

Abstract

Recognizing material categories is one of the core challenges in robotic nuclear waste decommissioning. All nuclear waste should be sorted and segregated according to its materials, and then different disposal post-process can be applied. In this paper, we propose a novel transfer learning approach to learn boundary-aware material segmentation from a meta-dataset and weakly annotated data. The proposed method is data-efficient, leveraging a publically available dataset for general computer vision tasks and coarsely labeled material recognition data, with only a limited number of fine pixel-wise annotations required. Importantly, our approach is integrated with a Simultaneous Localization and Mapping (SLAM) system to fuse the per-frame understanding delicately into a 3D global semantic map to facilitate robot manipulation in self-occluded object heaps or robot navigation in disaster zones. We evaluate the proposed method on the Materials in Context dataset over 23 categories and that our integrated system delivers quasi-real-time 3D semantic mapping with high-resolution images. The trained model is also verified in an industrial environment as part of the EU RoMaNs project, and promising qualitative results are presented. A video demo and the newly generated data can be found at the project website1 (Supplementary Material).

Item Type: Article
Subjects: STM Academic > Mathematical Science
Depositing User: Unnamed user with email support@stmacademic.com
Date Deposited: 29 Jun 2023 05:22
Last Modified: 09 Nov 2023 06:28
URI: http://article.researchpromo.com/id/eprint/1194

Actions (login required)

View Item
View Item