VirtuaLearn3D: New Preprint

We have published a preprint of our recent work in the VirtuaLearn3D project!

Deep learning with simulated laser scanning data for 3D point cloud classification

Esmorís, A.M., Weiser, H., Winiwarter, L., Cabaleiro, J.C. & Höfle, B. (2024)

Laser scanning is an active remote sensing technique to acquire state-of-the-art spatial measurements in the form of 3D point clouds. Semantic labelling is often necessary to extract information from the raw point clouds. The labelling task can be solved in an automatic way using deep learning methods, but this requires large amounts of training data which are costly to acquire and annotate.

In this work, we investigate the use of simulated virtual laser scanning (VLS) data for training deep learning models, which are used to classify real data. We validate the method for two use cases: Multi-class urban classification and binary leaf-wood classification.

For leaf-wood classification, our deep learning model trained purely on VLS data produces results comparable to models trained on real data when evaluated on real data, with just 1 % less overall accuracy (93.7 % with VLS training data, 94.7 % with real training data). For the urban classification with multiple, unbalanced classes, the gap between real and simulated training data seems to be larger and more work is needed in terms of scene modeling to improve informativeness of VLS data.

Overall, the strengths of the VLS-based approach are:

1. Large amounts of diverse laser scanning training data can be generated quickly and at low cost (i.e., without the need for expensive equipment).
2. Simulation configurations can be adapted so that the virtual training data have similar characteristics to the targeted real data.
3. And the whole workflow can be automated, e.g. through procedural scene generation.

Reference

Esmorís, A.M., Weiser, H., Winiwarter, L., Cabaleiro, J.C. & Höfle, B. (2024): Deep learning with simulated laser scanning data for 3D point cloud classification. EarthArXiv, https://doi.org/10.31223/X53Q3Q

Related work

For the virtual laser scanning, we use HELIOS++, the Heidelberg LiDAR Operations Simulator, a flexible and powerful open-source laser scanning simulation software which is hosted on GitHub: 3dgeo-heidelberg/helios

Winiwarter, L., Esmorís, A.M., Weiser, H., Anders, K., Martínez Sanchez, J., Searle, M., Höfle, B. (2022): Virtual laser scanning with HELIOS++: A novel take on ray tracing-based simulation of topographic full-waveform 3D laser scanning. Remote Sensing of Environment, 269, https://doi.org/10.1016/j.rse.2021.112772

Funding

This research was funded by the Deutsche Forschungsgemeinschaft (DFG), German Research Foundation, by the projects SYSSIFOSS (Grant Number: 411263134) and VirtuaLearn3D (Grant Number: 496418931).

This work has also received financial support from the Consellería de Cultura, Educación e Ordenación Universitaria (accreditation ED431C 2022/16 and accreditation ED431G-2019/04) and the European Regional Development Fund (ERDF), which acknowledges the CiTIUS-Research Center in Intelligent Technologies of the University of Santiago de Compostela as a Research Center of the Galician University System, and the Ministry of Economy and Competitiveness, Government of Spain (Grant Number PID2019-104834GB-I00 and PID2022-141623NB-I00).

The many deep learning experiments computed on the FinisTerrae-III supercomputer were possible thanks to the CESGA (Galician supercomputing center). Diverse experiments were also possible thanks to the data curators of the Hessigheim and Wytham Woods datasets and the manually labelled leaf-wood datasets.