Evaluation of Performance of Positioning Systems for Use as Ground Truth Sensors, 10-R8807

Background

The primary objective of this project was to leverage expensive equipment from the Test Tool Development Project for DENSO and the Crash Avoidance Metrics Partnership (CAMP), while it was still at SwRI, to conduct an in-depth evaluation and comparison of the performance capabilities of several positioning systems that are under consideration for integration into a complete SwRI connected vehicle (CV) test apparatus. SwRI proposed a methodical evaluation of three lower-cost positioning systems, using the Oxford RT3003 as the accepted baseline, to determine the accuracy and practical applicability of each system in varying configurations (with and without a base station or corrections subscription as available for each).

Approach

The SAE J2945/1 standard includes requirements on many items, including security and privacy (certificate management, usage, and roll-over), RF characteristics (channel usage, data rate, and transmit power), vehicle state (light indicators, transmission state indication, brake status, and others), vehicle dynamics (latitude, longitude, heading, speed, three-axis acceleration, and yaw rate), and underlying message encoding. However, for this quick-look project we only evaluated the vehicle dynamics portion of the requirements comparing various “ground truth” positioning systems against the Oxford RT3003. SwRI conducted mobile testing using test cases developed under a related Test Procedure Development Project. We also leveraged software developed to post process the data and ensure conformance to SAE J2945/1 standards when evaluating the different positioning systems.

Each of the following “ground truth” positioning systems were tested against the Oxford RT3003 system:

• Novatel FlexPak 6 (with and without TerraStar corrections)
• Piksi Multi (with and without base station corrections)
• U-blox 8

Accomplishments

The measures of success identified for this project included:

• Successful integration and data collection using each of the three identified positioning systems
• Successful analysis of positioning systems data and comparison of relevant system characteristics including cost, accuracy, and ease of configuration/maintenance
• Identification of at least one lower-cost positioning system that gives equivalent accuracy to the Oxford RT3003

The team successfully gathered data from each of the positioning systems in a variety of driving conditions. The results confirmed the team’s hypothesis that the Novatel and Piksi are both suitable for use as ground truth positioning systems in an SwRI CV test apparatus. There are trade-offs with each, most notably the recurrent TerraStar subscription cost for the Novatel, and the lack of flexibility in the geographic range of tests using the Piksi (with its base station). The cost of the Novatel with TerraStar subscription (approximately $8,000) and the Piksi (approximately $2,500) achieve and exceed the target of a 10:1 reduction in hardware costs for the positioning system, when compared to the Oxford RT3003.