Bionic Eyes Are No Match for the Curiosity of Pentagon Cartographers

Around the time the computer game "Where in the World Is Carmen Sandiego?" was teaching Generation Y about geography, grown-up versions of geography software had analysts at spy map agencies a little concerned about job security. 

Back then, Defense Intelligence Agency imagery analyst Robert Cardillo and his colleagues thought visualization technology would replace their tradecraft.

Today, Cardillo, now director the National Geospatial-Intelligence Agency, and his roughly 14,500 employees only wish more powerful image recognition software existed.

"Going back to 1983 when I was welcomed into the business, somebody said to me they are going to automate our job," Cardillo recounted in a recent interview. There will be "these ones and zeroes" that will pinpoint the changing positions of tanks, missiles, ships "and I thought well, that's a little scary."

The rumormonger told Cardillo it would be another six months or six years before the technology arrived, so he didn't have to worry about a pink slip just yet. 

"Fast forward 32 years: We’ve come a long way, but that’s still a very hard thing for computers to do, not impossible, very hard," he said. 

Right now, technology is up against a problem that didn't exist in the Atari age: big data. The ever-increasing petabytes of pictures and other information generated by sensors require quick and constant observation. 

"If you’re my analyst on Sub-Saharan Africa, I don’t want you spending time scanning all that imagery searching for something that wasn’t there last year, last month," Cardillo said. "I do want you thinking hard about the Boko Haram [terrorist group] challenge or thinking hard about Central African Republic sectarian divide, Muslim-Christian, etc. and building mental models yourself."

So, he's got computer programs handling "broad-based change detection," a method for flagging items that didn't appear in the previous day's data dump and items that have disappeared from view. 

Dumb Robots

Such analysis, however, is no substitute for the experience, education and curiosity of the human mind, lifelong geospatial intelligence professionals say.   

The computer is "looking for those triggers -- whether it’s a text or an email -- and then cuing the brain to go engage," Cardillo said. 

On certain problem sets, such as Chinese aggression in the South China Sea, the software is doing a good job at prompting analysts. 

"That’s a very tough place to monitor," he said. "Very big, very broad, very noisy. We have had some success with modeling to help us cue when and where something has changed."

For instance, the Chinese slowly are "are dredging up sand and they are building these islands" to mark their territory and, "over time, our ability to human cognitively process all of that coverage is just going to be overwhelmed," Cardillo said. 

Paul Weise, who spent over three decades analyzing imagery for the Pentagon, says he heard the same gossip as Cardillo about software taking over his position. The opposite dynamic occurred. 

"I can’t tell you how many times we brought on a new system, a new software set, that showed promise in either doing that job entirely or greatly assisting the analyst or cartographer," he said. "We ended up turning the tool off because it took more time for the cartographer to fix the mistakes that were falsely interpreted by such algorithms." 

Weise retired three years ago from his role as director of the Office of Geospatial Intelligence Management. Now, as Lockheed Martin’s GEOINT mission officer, he helps the agency expand its signature "Map of the World" project, among other things. The portal consumes and synthesizes all sorts of hush-hush intelligence so analysts can submerse themselves in the situation at hand. 

Weise says supersophisticated geographic information systems would be nice to have today.

"I think that would be of great value not because I’m looking to put cartographers and regional analysts out of business, but because they are drowning in data," he said. "GIS tools are doing a tremendous job at being able to organize that data as a decision-support tool, but the volume of data that GIS now has to manage or incorporate, archive, make sense out of, is orders of magnitude above what it was when I was facing the same challenges."

Big data requires more brains than the federal government could ever afford to hire. "At the end of the day, it’s the humans that have to recommend courses of action," Weise said.

‘Looking Through a Soda Straw’

The quest to automate image recognition began in the 1980s and 1990s when it became apparent technological progress was creating too many pictures to examine.

Back then, the feeling was that "if you had images you weren't looking at that they were wasted," said Keith Masback, a former NGA source operations group director.

Automation has shifted the tools of the trade from wet plate photography and hand-drawn maps to GIS. 

Increases in computer processing power, software agility, and Internet connectivity, along with decreases in the cost of storage now allow geospatial products to be spit out faster than a hand can cut and paste. And advances in sensor technology from minisatellite providers, like DigitalGlobe, Planet Labs and Google's SkyBox, have expanded the field of view.

But ultimately, the new equipment only reveals a base layer of the full picture.

"The reality is the most agile processor in this process remains the human brain," said Masback, now chief executive officer of the U.S. Geospatial Intelligence Foundation, a professional association.

"Human engagement is required to do the quality control and the fact-checking and the ground truthing, especially in the world of safety of navigation,” he said. “Because of the very ongoing life and death nature, the navigation products -- whether subsurface, surface or air -- have got to be right."

The advent of GIS can be compared to the rise of the drone, says Masback, who once worked as an Army intelligence master plan director.

Scout missions traditionally were the domain of helicopters. Now, unmanned aerial systems can fly longer hours and don't require an immediate search and rescue operation if an aircraft crashes.

But reconnoitering through a drone is like "looking through a soda straw," Masback said. The unmanned aircraft does not have inquisitiveness or peripheral vision.

Humans also are more interactive than GIS user interfaces, for the most part.

"I want to have an analyst standing up before me as a decision-maker presenting her work and to be able to challenge it. What is your source for this? Where did you find this? Why do you trust this source over another? Did you check this?  Did you reach out to a counterpart in another agency before you did this?" Masback explained. 

So, it seems the agency's 14,500 employees will still have a job next Monday.

"Until someone really cracks artificial intelligence and can put this into some software package, we’re fine," Cardillo said.

(Image via debra hughes/ Shutterstock.com)