Background
Audio localization provides a key survival benefit for almost every animal with the sense of hearing, including humans (Heffner & Heffner, 2016). This keen ability allows us to sense the presence of predators, prey, and environmental threats, and it allows us to approximate their distance and direction, which can be critically beneficial information for survival. Having two ears, the brain can detect and process minute differences between the received sounds in each ear to provide this benefit. However, in modern times, some of the individuals that could benefit from this ability the most — our military combatants, first responders and security personnel — are unable to utilize this advantage because they communicate primarily by means of mono-channel audio via a radio communication system. This inherently limits the situational awareness of these teams and does not permit passive awareness of team members’ locations. We have sought to return audio localization capability to these teams by incorporating existing spatial audio technologies to their COTS interpersonal communication systems (figure 1).
Spatialized audio or binaural audio, is a digital representation of humans’ natural binaural hearing which applies the minute differences of sound received at each ear to a digital audio source. The SwRI team believes that spatialized audio would be a valuable tool to provide passive (low cognitive demand) situational awareness in strenuous environments. As a result, Division 14 filed a patent for a unique device aimed to target this technology gap. US Patent US11165492B2 filed in January 2020 and granted in 2021 presents an accessorized solution to bring spatial audio to traditional communication systems for use in person-to-person transmissions. In this manner, standard communication over the transmission network is superimposed with location information to provide passive team awareness. The team believes the possibilities for augmented reality audio in tactical environments are vast, but demonstrating the core technology is challenging. For this reason, we have aimed to develop the core technology at SwRI.
Figure 1: Traditional radio systems (right) represent transmissions in a mono audio format, consequently, the audio doesn’t sound like it’s coming from any specific direction. With the ALT-C system in line with a traditional radio system (left), incoming transmissions are altered using directional information and binaural rendering so that the speaker’s direction can be understood by the receiver.
Approach
In order to integrate spatial audio into existing radio systems, data from the transmitter, receiver and user were used to inform a spatial audio algorithm and the resulting audio was sent to the receiver’s headset. A GPS module for real-time location services (RTLS) and an inertial measurement unit (IMU) for head orientation tracking were integrated with a digital signal processor (DSP) for each transmitter and receiver. The GPS data was transmitted between systems using a LoRa (Long Range) radio and the IMU was head mounted to the user to inform the spatial audio algorithm of the listener’s looking direction. Though this effort appended the standard radio with a LoRa system, communication of sensor data could be accomplished using the parent communication device in future efforts.
Using this information and existing techniques for generating spatial audio, incoming transmissions were modified and represented to the listener as directional audio, with the source direction dynamically represented to the user. The ALT-C system has been conceptualized as a drop-in replacement for existing push-to-talk (PTT) modules, which would be plug-and-play with existing radio communication systems.
Accomplishments
Through the course of development and investigation, the Division 14 team has successfully created a system capable of demonstrating the core function of ALT-C. A real-time spatial audio algorithm that operates onboard a Digital Signal Processor (DSP) platform has been developed. The capability of the system to track GPS locations of users on the network, transmit that data between the nodes, track receiver head-orientation and use these inputs to inform the spatial audio system has been demonstrated. For this effort, the packaged system was made backpack carryable and is integrated between the headset and the radio system (figure 2). The opportunity to reduce size and weight to the desired PTT form remains for future work. The system has been tested with blindfolded human listeners who were able to successfully directionalize the transmitter within 20 degrees of accuracy repeatedly. As a final output of this effort, point of view (POV) audio and video recordings have been generated to further demonstrate the system's basic capability to interested parties, and a formal version of these recordings is being produced.
Figure 2: Testing performed using the ALT-C system. A blindfolded user points in the direction of the transmitter (onboard the blue truck) using the binaural audio transmission provided by ALT-C.