Russian Invasion Highlights Growing Importance of Open Source Intelligence

During a morning political opinion show on March 2, Marc Polymeropoulos, a retired CIA field agent and supervisor, said he is sure his former employer is scouring its intelligence sources for signs of cracks in the walls of oligarchs and military supporters of Russian President Vladimir Putin as the invasion of Ukraine grinds on.

One hopes that the analysts are checking Twitter, TikTok, Telegram, and the broader social media environment.

“Ukraine Forces Reportedly Kill Russia General Andrei Sukhovetsky in Blow to Invading Army,” Newsweek reported March 3, citing a message posted to VKontakte, a Russian social media and social networking service.

Open source intelligence, or OSINT, is defined by the Center for Strategic and International Studies as “intelligence collected from publicly available or available-for-purchase information, obtained for addressing a specific intelligence requirement and processed to derive new insights.”

In January—weeks before Putin launched the invasion—CSIS released a report on the potential of OSINT, when combined with artificial intelligence and machine learning, to provide “fast, attributable, relevant reporting from anywhere, including austere field environments and [analysts’] homes.”

“High-level [intelligence community] policymakers have publicly stated the necessity of embracing open-source intelligence as a core analytic discipline,” the report stated. “[T]he intelligence community should stop ‘recreating the internet in a classified environment’ … and instead accept a small amount of risk to run applications on an unclassified cloud, taking advantage of increasingly sophisticated and automated cloud security and obfuscation capabilities.”

Anyone paying attention to the Russian invasion of Ukraine is getting a crash course in the value of OSINT, said Kari Bingen, chief strategy officer of HawkEye 360 and former principal deputy under secretary of defense for intelligence, the second-highest ranking civilian intelligence job in the Pentagon.

“We are seeing the unprecedented release of intelligence, sharing of information and transparency that is shining a light on Russian disinformation efforts and creating no doubt about Russia's aggressive actions,” she said. “The urgency of the moment demands it. The question going forward is, how do we better institutionalize this greater sharing of information and greater use of open-source intelligence to benefit our national security and that of our allies and partners?”

But the IC has been slow to incorporate OSINT into its intelligence products, for a number of reasons, and is lagging behind other countries—particularly China—in tapping into the vast oceans of publicly available information, the report noted. “While China has an estimated 100,000 science and technology intelligence officials, who do most of their work on open sources, the United States has an estimated 100.”

The author of the report, Emily Harding, deputy director and senior fellow of CSIS’ International Security Program, said the IC’s risk equation is part of the obstacle to committing to using OSINT.

“The IC lives in risk all the time, so they minimize it wherever they can,” Harding said. “That turns into restrictions and regulations.” She said the belief within the IC has been that if you don’t need to ask questions in the open, don’t, because it lets hostile parties know what they are interested in.

"Part of [the challenge] is culture, risk taking, the process of acquisition, but a lot of it just comes down to trust—building trust with OSINT providers, understanding the heritage of the information and having confidence in the data and analysis from those providers. And that takes stronger user-provider relationships, working together to better understand the problems and iterate on potential solutions," said Bingen.

There is indeed a big cultural component to the IC’s resistance to embracing OSINT, a combination of the “not invented here” syndrome, an ingrained belief in secrecy because field agents are out risking their lives, and budget fears. “Often, adding funding [to OSINT projects] would require removing funding from existing, proven systems—anathema to a government bureaucracy,” the report observed. “For contract officers, in particular, flexibility and risk-taking are not part of the job description … Contracts that finish under budget and on time, with deliverables that match exactly the specifications in the initial contract, count as successes for contracting officers.”

Harding suggested that the IC should keep the constellation of OSINT capabilities “on the low side”—that is, outside of classified networks—and accept the small amount of risk, because the upside is gaining real-time intelligence and insights.

“What OSINT traditionally meant was translating foreign documents,” said Bingen. “Open source was [considered] secondary” to the information gathered by clandestine sources and methods. “When you look at what’s become open source [today], the different types, it’s a far cry from just translating foreign articles. It’s not just your news articles or social media, it’s financial, it’s satellite information, the ability to bring it all together [and] apply advanced compute, AI and ML.”

CSIS’ Harding said the IC risks losing credibility with elected officials if it doesn’t move quickly to embed OSINT into its intelligence products. “In a best-case scenario, the IC will lose policymaker attention and trust as they compete with private intelligence,” she wrote in the report. 

This is particularly true as news media make greater efforts to capitalize on open source information; elected officials expect their classified briefings to be at least as well informed as the latest articles out of Ukraine, for instance.

“So much of the intelligence world is being commercialized,” Harding said when asked to elaborate on that observation. “You could commercialize analysis … You could definitely see Stratfor or other firms out there [get into this.] So many of these companies [started by] former U.S. government officials or military are still committed to the mission.”

Nor is the generation of OSINT limited to commercial activity. For instance, Bellingcat is an “international collective of researchers, investigators and citizen journalists” that uses social media and open source information to investigate a wide range of activities, from climate change to the spread of COVID-19 to the Ukraine invasion, to name just a few. “Using publicly available information, they have discovered illegal shipping of chemical weapons precursors, identified a high-ranking Russian intelligence officer as a key suspect in the shooting down of Malaysian Airlines Flight 17 and identified Russian intelligence officers as suspects in the poisoning of Sergei and Yulia Skripal,” according to the CSIS report.

“I come from a world where you buy billion-dollar satellites that launch every five to 10 years,” Bingen said. “These innovators are now building satellites that are launching every three to six months. It’s just such a different paradigm when you’re constantly evolving capabilities.”

There is a convergence of three technology areas happening right now that impels the OSINT discussion, she said. “First is the data, the variety of data that's out there now, open and unclassified ... Second is the advanced analytic and machine learning tools that allow us to bring those datasets together and draw out insights on activities, patterns of behavior, etc. Third is the cloud, which allows you to bring all the data together, and apply the analytics, and deliver a product to users across the globe."

There are ethical concerns to be addressed. Harding said the IC should not “vacuum up” all the available open source information, which is China’s approach. Included in her report’s recommendations is the need for “a clear framework of ethics and governance principles to guide how technology is applied to U.S. intelligence.”

"Open source intelligence is valuable to the Intelligence Community; it's one of the IC's core collection disciplines,” an ODNI spokesperson said.

An NSA spokesperson referred Nextgov to the Office of the Director of National Intelligence and the Defense Intelligence Agency on questions of OSINT as an intelligence discipline.

“On open source more generally, NSA will use open source information to support authorized mission activities. For intelligence activities, in addition to other applicable law and policy, NSA follows the procedures contained in Department of Defense Manual 5240.01 which, among other things, governs the collection, use, retention, and dissemination of open source information for Defense intelligence activities. These procedures are designed to protect the civil liberties and privacy of U.S. persons,” the spokesperson said.

In the meantime, every person following the Russian invasion of Ukraine is seeing the value of OSINT, as well as the risk—Google Maps has turned off its live traffic data in Ukraine for the safety of the people there.