Background
Visual information is a critical component of surface studies in planetary science. A key component of any imaging dataset is knowing the location of an image on the body. As an extreme example, taking an image of Earth’s north polar ice cap and not knowing it is at the north pole would make any interpretation more difficult. In the modern era, placing images correctly on a surface relative to each other, and other datasets, is important not only for Earth, where extremely precise and accurate tracking of spacecraft and ground surveying is possible, but also for other solar system bodies where such information is much less well known. Inaccuracies in cartography manifest as offsets between data in a single dataset (a feature will be placed at different locations from one image to another) to offsets between data in different datasets (e.g., spectrometer data will not align with visible light camera data); this can make interpretation very difficult.
Creating a control net solves this problem by requiring the same feature (a control point) in multiple images (control measure) to have the same 3D location. Software can solve for how the spacecraft and data-taking instrument must have been positioned in order to force those features to project to the same location on the body. During this process, the software is able to solve for the latitude, longitude, and elevation (distance from the planet’s center, also referred to as “radius”) of each control point. Once this has been done, the data are now controlled, and a product such as a controlled mosaic can be produced.
The US Geological Survey’s (USGS) Integrated Software for Imagers and Spectrometers (ISIS) software is used for non-terrestrial cartography. However, it has a steep learning curve, is inefficient, and has no inherent automation. For these reasons, very few researchers in the planetary science community know how to use it well, and most believe that creating a high-quality product requires a significant amount of manual effort. We have developed an automated approach to using that software which greatly decreases the manual effort required without sacrificing quality and operates on a modern computer in a relatively short period of time (hours to days as opposed to months or years).
The control net process itself can be divided into four primary stages:
Data acquisition and processing.
Candidate control point selection.
Candidate control measure matching or registration.
Solving the control network for updated spacecraft and camera data.
Step 4 requires, as input, the control net file, which lists those points, their measures, and where the measures should be based on feature matching. An existing USGS ISIS program then attempts to solve the control net so that when the images are projected onto a planet’s surface, those measures project properly to where they should, as opposed to where they did originally. The mathematics behind this program is effectively an iterative matrix solution, where the dimensions are the number of images versus the number of control points. A problem arises when the number of images and/or control points becomes extremely large, and modern computer hardware simply cannot solve it because the required computer resources do not exist. This Regular IR&D is meant to address this.
Specifically, we want to control the Context Camera (CTX) image dataset from NASA’s Mars Reconnaissance Orbiter. CTX is the second-largest existing planetary image dataset for any solar system body other than Earth, and many other smaller ones remain uncontrolled. This dataset currently comprises over 115,000 images that are up to 1GB each. This line scan camera takes images approximately 5,000 pixels across and has enough onboard RAM to scan up to approximately 50,000 pixels long. These massive images produce a ground scale of 6 meters per pixel. Through previous, smaller IR&Ds, we developed automated software to produce control nets of small regions of Mars, comprising up to a few hundred images. Merging them, however, has met with skepticism among funding agencies and peer-review panels.
Our goals are to:
Automated-control ≈50% of Mars (but ≈60% of CTX images due to how they are distributed across the planet).
Manually check the regions that constitute that area of Mars.
Develop ways to optimize the control networks that are produced, so they can be merged and still solved with existing hardware.
Put another way, our main underlying goal is to develop optimization tools that allow us to solve massive control networks, prove that it can be done, and then propose to outside funding agencies to do it for smaller datasets.
Approach
Goals 1 and 2 were straightforward and simply required time. The tools to do them were created in previous IR&Ds.
For the third goal, we developed two approaches to optimize the control net. The first was that we would optimize the creation of the control network itself, producing many fewer, but better, control points which would produce a smaller control network. By optimizing the process as the control net is being made, it takes less time (fewer points to operate on) and reduces the need to optimize the network later (i.e., if we can produce control networks that are only 10% of the previous size, then we can merge 10 times as many together before running into matrix-solving problems on computer hardware). The second approach was that after the control networks were created and potentially merged, we would remove points that did not contribute meaningful information to the network. For example, imagine two points within 1 meter of each other. One of those points connects three images together, while the other connects two, and those two are also connected by the point that connects three. In this situation, we should remove the point that only connects two images together because it is redundant. By removing a point, we have now made the matrix-solving problem easier because we have removed an entire row that needs to be solved.
Accomplishments
We have completed our period of performance and successfully completed all three goals.
Goal 1: We were fully successful in controlling >50% of Mars, and >60% of the data catalog. In fact, through our improved code (discussed below), we were able to control ≈75% of Mars’ surface area during the lifetime of this IR&D. The only deficit is that we were unable to complete Mars’ north pole. The poles represent unique challenges with changing illumination geometries and changing surface features which make any automated feature matching very difficult. For example, if one takes an image of a mountain with a glacier and then takes another image six months later when the glacier is gone, automated software will have a hard time recognizing similar features on that mountain because there would not be any. While we have made progress with the north pole, and it drove much of our code development during this work, only approximately half of it has been cartographically controlled to date. However, the control of much larger areas of the rest of the planet more than make up for this deficit.
Goal 2: For the above-described automated control work, we accomplished this manual work.
Goal 3: We completely rewrote most of our cartographic control network code to be much more efficient with how it creates a control point. Previously, our code would work by creating a grid of points and then testing the points to see how well they matched features between overlapping images. This can be very inefficient because it could create many hundreds of good points on two overlapping images, but only about 10 points are needed. Our new code determines where unique sets of images overlap, and it then creates just a few points in those areas at random and tries to match features between the images. This means that in those large areas of overlap, only a few points would be created, tested, and survive to the end, saving significant amounts of both disk space and time while reducing the matrix math problem that must be done to solve the network and update spacecraft pointing data. We were extremely successful with this, reducing control networks by a factor of approximately 10 times in size, and because so many fewer points needed to be tested, it also sped the automated code up by a factor of almost 10 times, as well. To put real-world numbers to it, with the old code, we could control 0.2% of Mars in about 1 day, and the control network would be on the order of 100MB. With our new code, we can control 3.4% of Mars (non-polar regions) in about 5 days, and the control networks are an average of 150MB. We also wrote a new “thinning” algorithm for retroactive control network analysis. The code uses the techniques from commercial photographic mosaic software to optimize control point placement so that important points are kept while excess points are removed. The code has several parameters that can be tuned, depending on exactly how much one wishes to reduce network size. It has succeeded in reducing our networks from our old code by about 80%, and from our new code by about 30%, while still retaining good point coverage across images.
Overall, this was a successful IR&D.