Background
Robots frequently operate in areas with challenging light conditions, such as caves or in Earth orbit. In these areas, the extreme lighting changes or lack of illumination cause traditional electrooptical (EO) stereoscopic cameras to fail. These failures can be mitigated by additional lighting payloads at the cost of increased power, weight, and complexity, which is not always acceptable. Alternative sensing modalities, such as lidar or radar, can be used in place of stereoscopic cameras for robotic navigation tasks but do not offer the same inspection and environmental sensing capabilities as EO cameras.
Approach
We modified the Recurrent All Pairs Field Transform (RAFT) neural network architecture for both monocular and stereoscopic thermal cameras. The monocular version of RAFT produces optical flow fields, and the stereoscopic version produces disparity maps of the environment. Both architectures are potential inputs to simultaneous localization and mapping (SLAM) algorithms, which produce 3D maps of an unknown environment without any prior knowledge of the operating region. We adopted the RAFT outputs to an internal SwRI SLAM algorithm to produce 3D maps and perform 3D reconstruction of objects in the environment.
Accomplishments
We tested our SLAM algorithm with both stereoscopic and monocular camera inputs in a variety of environments. With our approach, we mapped a local cave system, demonstrating the utility of the system in dark or poorly illuminated regions. We also used our system to create 3D models of computing equipment, highlighting our SLAM algorithm’s usefulness for industrial inspection problems. Finally, we tested our approach using an academic driving dataset, which performed well in poorly illuminated streets, showing the possibilities of this system for autonomous driving systems.