DOD’s generative AI task force looks for ‘blind spots’

Understanding the limitations of artificial intelligence technology is key to the Department of Defense’s broader adoption of emerging tools, according to the head of the Pentagon’s task force for exploring how generative AI technologies can be safely used for military operations.

During a presentation at the Chief Digital and Artificial Intelligence Office’s defense data and AI symposium on Thursday, U.S. Navy Captain Manuel Xavier Lugo said the DOD initiative, known as Task Force Lima, is focused on understanding “the areas where people want to utilize these technologies.”

DOD launched the task force in August with the goal of “harnessing the power of artificial intelligence in a responsible and strategic manner.” Since then, the initiative has collected more than 180 use cases across the Pentagon to determine the capabilities of these tools.

“The only way that users are going to get comfortable with the technology is by using it, but also it’s the only way that we're going to learn the limits,” said Lugo, the mission commander of Task Force Lima.

Many of these use cases focus on automating routine tasks or more mundane activities for DOD personnel. Lugo said a large part of this process entails looking at “how this technology can actually be in the background without the humans even knowing that it’s there.”

As such, Lugo said a top priority for the task force has been balancing the potential of generative AI use cases to streamline daily activities with “the blind spots” that could undermine their effective implementation.

Some of the use cases Task Force Lima has been exploring, for instance, focus on email summarization, where DOD personnel would receive “a summary of your email traffic [so] you don't have to read every single one of those emails.” But the initiative is also simultaneously exploring the downsides of using such a tool, including the possibility that it ignores or excludes key information. 

“That sounds great, until the summary misses that one email that says, ‘you must be at this meeting at this time with your boss,’ and it didn’t put it in the summarization because, for some reason, it didn’t pick it,” Lugo said. “So, again, that’s the dilemma of where we are with the current technology.”

For roughly 60% of the task force’s use cases, however, Lugo said “all [DOD personnel] are asking for is a chatbot” to help them have “that generative capability” to get more intensive projects underway.

Although the task force is only scheduled to operate for an 18 month period, Lugo said one of its long-term goals is to “have a transition plan to where those pieces need to go” when it comes to policy recommendations and architectural guidance for the department moving forward.

“I cannot call ‘mission complete’ on Lima until those pieces are understood,” he said.