Background
This research aims to answer two open questions within the space of creating lidar maps using vehicle mounted sensors: how to automatically incorporate expert human knowledge into machine learning models to remove objects in the scene that don’t belong in the map; and how to execute those models in real time on vehicle-relevant hardware rather than the typical high-powered desktop hardware found in the literature. While performing this research, the team also hopes to discover and document best practices for the practical implementation of the results in diverse applications.
Approach
The first step to addressing these challenges is to design a structured representation of the data which can be fed into a neural network in such a way to take advantage of current neural network research. The most common output of a lidar sensor is a cloud of points in 3D space. While this format is easy to transport and visualize, it discards a lot of the spatial information between points. Our approach effectively converts the coordinate frame from a Cartesian coordinate frame into a polar coordinate frame, preserving the relationship between adjacent measurements of depth.
The second step is to add more data to the structure. While lidar sensors are active sensors, the photon detectors pick up not only the return from the emitter, but additional reflections as well. While this information doesn’t contribute to the range calculation, it can be used to generate the equivalent of a 360° array of infrared cameras, extending the range of the lidar sensor beyond what the emitter is capable of.
The third step is network design and implementation. After the network has been bootstrapped, we can work on tuning the dataset by adding relevant data. With a network architecture and training data in hand, we will work on accelerating the network inference performance and implementing the pipeline necessary to conduct the planned experiments.
Accomplishments
The team has designed and tested a neural network architecture to perform the map reduction and creation task. We have also adapted relevant data sources from both publicly available datasets and local data collection, to make them compatible with our neural network design, significantly expanding the available training data without compromising the network’s capability or efficiency. Finally, the team has designed and implemented a preliminary pipeline to run experiments, which will allow us to benchmark our implementation against existing and future mapping implementations.