OpenAI national security lead endorses ‘appropriate human judgment’ in AI

Workforce transformation and the application of “appropriate human judgment” will be necessary when incorporating advanced artificial intelligence capabilities into defense operations, according to OpenAI’s head of national security policy.

“I do think there's a workforce transformation that we're facing where we're going to have to educate a generation of analysts, of service members, of Foreign Service officers, to ensure that they're still applying — in the Pentagon, they call it ‘appropriate human judgment,’” Sasha Baker, head of National Security Policy at OpenAI, said during a Thursday conference hosted by the Special Competitive Studies Project.

In defense operations, the consequences of an incorrect decision fueled by AI systems are “much greater” depending on the use case, Baker continued. Her comments come amid the ongoing fallout between Anthropic and the Pentagon over how AI products contracted by the federal government should be used in warfighter operations. 

Just after the Trump administration canceled the contract between the Pentagon and Anthropic and further blacklisted the company in federal operations, OpenAI announced a deal with the Pentagon and said the agency agreed to Anthropic’s original stipulations: prohibitions on U.S. citizen surveillance and human presence in autonomous weapons. 

“When you think about the potential for AI transformation, it really impacts the entire spectrum of the work that we do as a national security community, from making your paperwork a little bit easier and more efficient all the way to potentially revolutionizing the targeting cycle,” Baker said. 

Baker reiterated OpenAI’s prioritization of safety in advanced AI deployment during the conference, saying that, as of now, no large language model is foolproof, including OpenAI’s flagship model ChatGPT. Baker mentioned OpenAI’s Trusted Access program as a filter for AI models with a level of safety applied, such as user protocols.

Baker’s comments were in response to the recent announcement of Project Glasswing, a new Anthropic program that is distributing the company’s high-powered beta Claude Mythos Preview model to select tech companies as a means to test its cyber capabilities.

She also briefly discussed the latest OpenAI model, Spud, which is still in development, saying cybersecurity is top of mind ahead of its release, similar to the process Anthropic is taking with its Mythos Preview model.

“As Spud leaves the laboratory environment and becomes widely accessible, we do want to make sure that cyber defenders get a chance to both understand the model capabilities and potentially use it to patch vulnerabilities that might otherwise become significant from a national security perspective,” Baker said.

In addition to expanding on OpenAI’s security posture when crafting and deploying its AI models, Baker discussed her effort to bring OpenAI engineers to Congress, the White House and the Pentagon to evangelize the technical facts about AI with policymakers and to foster smart, strategic AI adoption in government workflows.

“I do worry a little bit that we'll try a bunch of wrong things before we get to the right thing,” she said. “I hope that we get there before there is a crisis that really forces us to galvanize and come together. I think we can do it.”