With VirtuaLearn3D (Virtual Laser Scanning for Machine Learning Algorithms in Geographic 3D Point Cloud Analysis), a new project of the 3DGeo group has started. The focus of this project is to enable powerful machine learning algorithms for geographic point cloud analysis by advancing the concept of virtual laser scanning to overcome the lack of training data.
This project has two main objectives:
1) Combining Virtual Laser Scanning (VLS) for training data generation and Machine Learning (ML) algorithms to solve point cloud processing problems (e.g., leaf-wood segmentation, multi-class semantic segmentation, biomass estimation, …).
2) Extending VLS to support dynamic scenes, i.e., scenes where some objects move with an acceptable time-synchronization between the VLS ray-tracing algorithm and the rigid motion simulation. This will make it possible to create more realistic simulated training data and will open up new areas of application.
After an exciting few months of research, we look forward to presenting and publishing our first results soon. For objective (1), we trained deep learning (DL) models using only simulated 3D laser scanning data and then we validated them using real-world point clouds. The overall accuracy of this VLS-DL model is only 1% lower than the model trained on real data when applied to the leaf-wood segmentation problem. We also experimented with multi-class semantic segmentation in urban contexts with our model trained on virtual data. Below you can get a visual impression of the performance of the Virtual Laser Scanning meets Deep Learning (VLS-DL) model:
For objective (2) we are working on dynamic simulations with a) vegetation moving in the wind and b) moving vehicles in the context of autonomous driving. Below you can see an animation of a tree with shaking leaves and the resulting multi-scan terrestrial laser scanning point cloud, simulated with HELIOS++. Find out how to configure dynamic scenes for VLS in the HELIOS++ GitHub Wiki!
More details about the VirtuaLearn3D project can be found on the project website.
For the virtual laser scanning, we primarily use HELIOS++, the Heidelberg LiDAR Operations Simulator, a flexible and powerful open-source laser scanning simulation software which is hosted on GitHub: 3dgeo-heidelberg/helios
The VirtuaLearn3D project is funded by the Deutsche Forschungsgemeinschaft (DFG), German Research Foundation (Grant Number: 496418931).
