Background
Autonomous robots are increasingly used in domains that are challenging to computer vision systems that rely on traditional electro-optical (EO) cameras that operate in the visible light spectrum. These robots must identify and localize objects in 3D space, as well as form 3D world models of their environment. Currently, Southwest Research Institute (SwRI) uses technologies such as lidar or stereoscopic cameras for localization, mapping, and object detection. SwRI is experiencing these challenges in areas such as space robotics, where there is limited illumination or harsh lighting, and autonomous ground robots that operate at night. As a result, SwRI is expanding its capabilities in alternate sensing modalities for autonomous systems, including thermal cameras and multi- and hyper-spectral imagery.
Approach
SwRI developed deep learning algorithms for stereoscopic thermal camera pairs and monocular thermal cameras that is based on SwRI’s existing simultaneous localization and mapping (SLAM) algorithms and academic approaches for monocular thermal computer vision. SwRI’s deep learning algorithms can create 3D maps of the environment, even in limited lighting. The 3D maps of the environment are compatible with SwRI’s autonomy and inspection algorithms for a variety of robotic platforms, including aerial and ground systems.
Accomplishments
SwRI tested the deep learning algorithms using commercial, off-the-shelf (COTS), uncooled thermal cameras. SwRI evaluated the system by mapping the off-road testing area of SwRI from a ground vehicle, testing the system in an underground cavern, and creating a 3D representation of computing equipment that is representative of industrial inspection. The deep learning algorithms created accurate 3D point clouds from the thermal cameras and were resilient to visual artifacts from the thermal data if there was enough contrast in the thermal environment. These tests also generated insights into the conditions where these cameras can operate and how much thermal contrast is needed for the stereo matching algorithms.