The first driverless motor vehicle was probably a 1926 Chandler automobile retrofitted for radio control and operated by a trailing chase car — a journey that ended in a crash. The United States automotive industry’s next foray into self-driving cars — or automated driving systems (ADS) — was in the 1950s and 1960s and largely visionary. Forward-thinking original equipment manufacturers (OEMs) and research labs created imagery suggesting that, someday, people would sit in vehicles and be transported with no human control. While the visions and dreams were thought-provoking, no tangible products were offered for purchase.
The Early Years of Automated Driving
Fast forward a few decades and Congress passed a 1991 transportation bill that included a requirement for the Federal Highway Administration (FHWA) to create and fund an “Automated Highway System” program aimed at developing vehicle prototypes and highway technology that could support self-driving vehicles. In 1997, the program culminated with the demonstration of self-driving passenger vehicles, buses and trucks on an isolated corridor of a San Diego interstate — these vehicles did not interact with traditional human-driven vehicles. Once the funding was exhausted, FHWA interest in the program stopped.
Around the turn of the 21st century, Congress again took interest and issued an unfunded mandate that the U.S. Army should have 25% of its fleet automated within 15 years. Despite the lack of direct funding, more companies and universities began exploring how to meet this ambitious goal. The Defense Advanced Research Projects Agency (DARPA) initiated the Grand Challenge to demonstrate self-driving vehicles in off-road environments. Teams formed between companies and universities, but no one completed the course in the first challenge held in 2004. However, several teams completed the subsequent course in 2005. Realizing that off-road autonomy would not support most of the vehicle miles traveled in the United States, DARPA initiated the Urban Challenge, which was held on an abandoned military base in 2007. Results varied, but at the end of the challenge, DARPA declared their work “done.” It was up to industry to commercialize.
SwRI’s IR&D program invested around $400,000 in hardware loaded in the back of an SUV to create its first automated driving platform, MARTI 1.
While DARPA held the challenges, several engineers at Southwest Research Institute — some who had participated in the DARPA challenges as students — began to talk about how SwRI could become involved. After some discussion, the consensus was that getting in the automated driving game made sense. SwRI had most of the necessary skills across the Institute as well as growing staff excitement. Technical leads organized to develop an automated driving initiative from five of SwRI’s technical divisions. The capabilities they brought to the table included program leadership, perception, safety, testing, intelligence, and command and control.
SwRI had a working automated driving system ready to deploy on New York City streets during a technology demonstration at the 2008 ITS World Congress.
DETAIL
The MARTI acronym is also a nod to SwRI’s visionary second president, Martin Goland, who led the Institute from 1959–97.
Ads Launch
In 2005, the team launched SwRI’s ADS program with $4 million in internal research (IR) funding along with a $500,000 capital work order to create a working prototype. SwRI provided an additional $500,000 in IR funding to get the project across the finish line for a total of $5 million in IR investment. The project was initially called the Southwest Safe Transport Initiative, or SSTI. However, the acronym did not roll off the tongue, so the project was rebranded as MARTI, which stands for Mobile Autonomous Robotic Technology Initiative. Five divisions formed the project’s steering committee to track work orders and monitor technology under development.
The economic environment required SwRI to pivot to military applications, demonstrating MARTI at “robotics rodeos” and operating on unpaved roads and grassy fields to simulate potential military theaters.
The highly multidisciplinary team began technical development, installing about $400,000 in hardware into a stock 2006 Ford Explorer to make this original platform. The hardware filled the back of the SUV and included external sensors attached to its exterior. The team developed and tuned algorithms using a variety of test environments to duplicate DARPA test protocols. The testing team also evaluated vehicles participating in the DARPA Urban Challenge to gain insights into other approaches. The SwRI team focused on having a vehicle ready to publicly display and demonstrate at the Intelligent Transport Systems (ITS) World Congress in New York City in November 2008.
V2V Niche
MARTI’s journey to New York was not a simple one. The team pushed the state of the art in sensor technology and computer power up against the maximum capacity for data collection and processing. The team developed and patented a range of new technologies to develop a viable product. SwRI’s solution was unique to the industry because it included vehicle-to-vehicle, or V2V, communications technology that allows vehicles to share sensor information, effectively expanding what a single vehicle can see by augmenting it with data from other vehicles. SwRI was an early pioneer of the U.S. Department of Transportation’s V2V program and is still considered an expert in the area. In contrast to today’s commercial offerings, SwRI remains the only organization to include V2V technology on every automated driving platform it has developed, which now number in the twenties.
In 2008, the half-million-dollar-plus vehicle was loaded into a trailer for transport to New York City, where the team demonstrated automated driving and connected vehicle capabilities on the streets of Manhattan to approximately 10,000 participants at the ITS World Congress. While access to the streets was restricted during the demonstration, no special infrastructure modifications were needed to showcase the technology.
Unfortunately, the Great Recession hit in the same timeframe so the opportunity to deploy the technology in commercial vehicles evaporated as auto companies fought for their very existence.
SwRI developed gesture recognition capabilities for automated driving systems used in military, highway work convoy and other applications. Image-processing algorithms identify and distinguish different arm gestures from a designated pedestrian that allow more natural, efficient interaction with an automated system.
SwRI engineers successfully demonstrated automated driving in military applications during 2012. These images illustrate the SUMET system’s real-time visualization applications.
Military Momentum
In 2009, the U.S. Army announced a “robotics rodeo,” asking companies to demonstrate their automated driving technology at a military base, particularly under conditions found in military theaters. Operating on unpaved roads or across grassy fields is markedly different from driving on paved roads. SwRI engineers converting the on-road systems for “off-roading” was, in hindsight, one of the best decisions made. This effort kicked off 15 years of projects that catapulted SwRI to the forefront of off-road autonomy. The robotics rodeo required convoys of vehicles to follow the automated vehicle, closely observing its performance. SwRI was given limited time — or no time — to test on the base. Often, our initial test was an actual live demonstration.
From the first days of the program, the team invested tremendous effort in business development. As our capabilities and technology offerings grew, we began visiting clients, including commercial and government organizations in both domestic and foreign markets. The team talked to over 125 companies and agencies to identify potential projects.
SwRI’s autonomy stack has been deployed in more than 20 research platforms from passenger vehicles to buses, shuttles, long-haul trucks, military combat vehicles and more.
After the 2009 Robotics Rodeo at Fort Hood in Killeen, Texas, the Office of Naval Research (ONR) contracted SwRI to start an automated ground vehicle program called the Small Unit Mobility Enhancement Tools (SUMET) project, with the goal of having a vehicle operate in an austere, off-road environment with only passive sensors, such as cameras. Lidar methods, considered standard in many automated driving systems, use a pulsed laser light to measure objects and locations. Combatants can detect the light, so lidar was deemed unacceptable. SwRI developed the entire SUMET code and gave ONR government-purpose rights to both kickstart the program and apply to other autonomy programs. Given the camera-only constraint of the system, SwRI also provided code for another IR&D project using visible-spectrum cameras under the same open license.
While the SUMET program continued to grow, it brought additional collaborators to work with SwRI to develop the same code base. In 2012, the U.S. Army Tank Automotive Research, Development and Engineering Center (TARDEC) began working with SwRI to leverage the SUMET code base to rapidly deploy a robotic system to Afghanistan by 2014. Building on such a base was the best strategy to meet the aggressive deployment timeline. This effort developed into the Dismounted Soldier Autonomy Tools (DSAT)
After the vehicles were deployed to Afghanistan, SwRI began working with TARDEC to leverage the expanded DSAT capabilities to improve and integrate capabilities to other robotic ground vehicle programs. Renamed the Robotics Technology Kernel (RTK) in 2014, it has been expanded and exported to over 15 different Army programs. The SwRI team has been a major player throughout this program, working with Ground Vehicle Systems Center (GVSC) engineers and helping new users learn how to use and contribute to the RTK. Since the 2019 RTK core release, this base has grown to over 1.7 million lines of code. SwRI has been the primary contributor, but over 20% of the code has come from other organizations like GVSC, DCS and newer contributors like Carnegie Mellon University’s National Robotics Engineering Center (NREC).
This graphic illustrates how the initial investment in MARTI has created a multipronged software stack that has DOD branches, commercial extensions, and agriculture and construction applications.
Back on Commercial Track
In addition to its federally funded autonomy work, SwRI is managing the Shell Eco-marathon Autonomous Programming Competition. Through this program, SwRI has developed an integrated platform that uses continuous integration, a robotic simulator, the Robot Operating System and a game engine to automatically build, test and analyze the performance of the simulated vehicles and contestants’ algorithms. This simulation models the performance, behavior and sensors of the real-world vehicle platform used in other competition components. SwRI constructed a virtual representation of one of its automated driving test facilities for testing software performance and behavior in a simulated environment.
Once SwRI established its credibility in the ADS arena, teams performed a number of projects for Japanese OEM and Tier-1 suppliers. These projects ranged from staff training to developing black box functionality to deploying and testing technology. SwRI’s stack is used in a variety of self-driving platforms from passenger vehicles to buses, shuttles, long-haul trucks, military combat vehicles and more. The staff continually upgrades and tests the technology and uses it to help clients in the commercial and military sectors to test and validate the safety and security of automated driving systems.
A key component of an ADS is the visualization technology that allows a vehicle to navigate while recognizing objects, traffic signs, pedestrians and other vehicles. The team continues to enhance and improve the capabilities of these systems using platforms such as an automated driving shuttle that runs regular routes around SwRI’s 1,500-acre San Antonio campus.
On the DOD front, the large defense contractors who initially flooded the space realized that production of automated driving fleets would be limited because the technology simply was not progressing fast enough. This situation enabled SwRI to become a major player in military autonomy because its role is in specific solutions, not in product sales, and it has been developing automated systems for the Army for the last 10 years. In fact, the Army’s vehicle autonomy stack for all their ground vehicles is maintained by SwRI and includes part of the original MARTI code.
Over nearly two decades, SwRI has developed a range of autonomy solutions, including a stack known as Hound that features several layers of technology enabling advanced levels of automated driving. This groundbreaking progress in autonomy started with SwRI’s $5 million internal research investment and has evolved into a diverse automated driving program with a multipronged software stack.
While the automated driving sector has advanced rapidly, it took years to get to this point, and SwRI continues to address many safety and security challenges. Looking back at SwRI’s foray into automated driving, the team can see how much it has contributed to the sector. As autonomous robotic systems become increasingly complex, SwRI recognizes the opportunities to help industry and the public pave a way for safe, secure and reliable autonomy.
Questions about this story or Autonomous Vehicle Research & Testing? Contact Dr. Steve Dellenback at +1 210 522 3914 or Ryan Lamm at +1 210 522 5350.
ABOUT THE AUTHORS
Dr. Steve Dellenback is vice president of the Intelligent Systems Division, directing approximately 270 R&D staff. He served as a member of the ITS America Board for seven years and recently joined the U.S. Department of Transportation’s Transforming Transportation Advisory Committee (TTAC). Executive Director Ryan Lamm has more than 25 years of experience in automated driving systems and enabling technologies. He is appointed to and leads the sensor team of a NATO research task group assessing mobility methods and tools for autonomous mobility ground systems. The authors are pictured next to SwRI’s first automated driving platform, MARTI 1, on display at the San Antonio Museum of Science and Technology’s Area 21™, which exhibits 21st century inventions.