The U.S. Air Force’s Next AI Project is About to Kick Into High Gear

The U.S. Air Force is doubling down on efforts to fuse text, video, and virtually every potential source of data or information together through artificial intelligence. The goal is to change the way every commander, airman, and even thinking machine makes decisions. It’s a program called Data to Decision, or D2D. Launched last year, the program will expand significantly in 2018, service officials say.

What is D2D aiming for? “A complete cycle of understanding, from predicting what we expect to encounter, prescribe what can be done to help understand the environment, then find, fix, track, target, engage, assess, anything, anytime, anywhere in any domain” Mark Tapper, the Air Force’s Champion for Data 2 Decision, an experimentation campaign underpinning the Air Superiority 2030 flight plan, said via email. “Our mantra is improve every decision – those made by airmen or by machines.”

That includes decisions about where a piece of information or data needs to go next. “How you aggregate all the data, take the pieces that matter for a mission, move it where you need to move it (perhaps for different purposes) then think about where it needs to move and why” will be key, Tapper wrote.

The program is grand in its ambitions to use a wide variety of data, extending well beyond traditional aerial surveillance footage to potentially include, well, everything: social media posts, live-streaming diagnostic data off of jets, drones, and other aircraft, attainable whether data, pilot biophysical data from soldier-worn sensors, and more. “We will use all data available” including “unstructured, open and national sources,” Tapper wrote. “We are shifting thinking to focus on the object in question and looking for any data that may be relevant. Machines will help us determine relevance, especially as we aggregate in ways never before considered. So all the potential data cited could be part of the relevance discussion.”

How do you take all of that data and use it to output a dynamic and credible picture of the future? Here, too, the strategy is: diversity in approach. The potential artificial intelligence tool kit the Air Force is using ranges from neural nets and deep learning approaches to less exotic machine learning methods, useful for tasks where the data is structured or the variables are fewer, like chess.

It’s not the first time the military has experimented with fusing a wide variety of data for better decision making. A 2014 Defense Advanced Research Projects Agency, or DARPA, program called Insight also sought to create a fuller picture of the battlespace by combining and crunching data. It used multiple neural nets to populate a larger engine that then used Bayesian statistical methods to output predictions and probability.

D2D uses some lessons from Insight. “AI and Machine Learning can help by creating neural networks through disparate data sources that would then help us better understand the potential to use the nets for Air Force mission execution,” said Tapper.

That’s key, because Bayesian statistical methods don’t provide the most accurate answer the first time you use them. You run the formula over and over again, stirring in new data or information that you receive, coming closer and closer to a prediction in which you can have confidence.

Modeling and simulation will play a big role in the experimentation. The reason has everything to do with artificial intelligence in 2018, and the challenge of accepting outputs from artificial intelligence tools, especially those outputs based on opaque processes. It’s one reason why public-policy makers were historically slow to embrace artificial intelligence, and particularly neural networks, for decision making even as social media companies and high-frequency traders have raced ahead.

Neural nets and deep learning approaches have proven effective in chaotic, unstructured environments and problems, such as helping self-driving cars navigate city streets, helping algorithmsidentify objects in YouTube videos, or helping computer games win the high-variable game of Go. But the underlying processes behind them defy easy explanation, and that’s a big drawback.

“The biggest part of the problem of artificial intelligence is: they build these incredibly long algorithms with all of these gates to go through. They push all of this machine learning and data through it. Frankly, we are not entirely sure how all of that works, all the time,” said Dale Ormond, the principal director for research in the Office of the Assistant Secretary of Defense (research and engineering,) speaking at a recent AUVSI conference in Maryland.

“If someone sabotages your data that your feeding your algorithm for learning, we’re not entirely sure what’s going to come out of the other end. It’s going to take a while to figure out how to organize these things to be successful,” Ormond said at a recent AUVSI conference in Maryland. 

That’s not a big issue when the objective is to show the best possible ad to the right user at the right time a particular social network. Even when the goal is to predict the future behavior of a stock, a hedging strategy can mitigate losses. But when war, peace, and human lives are on the line, the stakes are a bit higher. Commanders and policymakers need some way to better explain why they arrived at the decision they did, besides “the machine made me do it.”

Gen. Paul Selva, Vice Chairman of the Joint Chiefs of Staff, expounded on that point during a recent sitdown with reporters. “We haven’t cracked the nut on man-machine teaming yet. I don’t believe anybody has. The closest we’ve gotten is the extremely high level of information we’ve pushed to aviators in cockpits,” Selva said.

Nevertheless, the military has reached a new level of comfort experimenting with artificial intelligence for decision-making.

“The components of the Data to Decision (D2D) Program have transitioned into operational prototypes in a variety of mission areas and include activities in video, image, and text analytics. The D2D technical products and lessons learned are critical elements of the DoD roadmap for future capability in machine learning and artificial intelligence,” Ormond said in an email to Defense One.

On that roadmap, Selva has high expectations. “You’ll see some modest investments in that space. If you look inside what we are pushing to aviators, it’s a hell of a lot of information to absorb. So it’s about, can you actually push the information to the human. Can you make it make sense to the human, to help the human make decisions at greater speed? That last part is the hard part. The human has to decide faster. So parsing the data and how you decide actually matters. That’s a technology piece that’s really going to require some experimentation,” he said.

Selva’s office later clarified that he was not speaking about D2Dspecifically but more broadly about Defense Department investment in artificial intelligence.

“Some would argue that is man-machine teaming. I’ll describe it differently: you take that same person that’s working in that cockpit — with all that information streaming to them — and you augment it by a factor of ten by teaming that airplane — that can push that amount of information to an aviator who is making target engagement decisions (with offboard sensors that push ten times more data to that same person) and give them the weapons that will allow them to prosecute those targets. Now you are talking about more than what fighter pilots call beyond-visual-range shots with high confidence. You’re talking about being able to prosecute the enemy at hundreds of kilometers away from your airplane with very high confidence that you’re actually engaging the targets you intend to engage, because we don’t engage targets unless we have high confidence,” he said.

“Wouldn’t it be great if we could shoot someone in the face at 200 kilometers? They don’t even know you are there. That’s the kind of man-machine teaming we really want to get after.”