recommended reading

What IARPA Knows About Your Canceled Dinner Reservation

Andrey Bayda/

At the intelligence community's research and development agency, the wisdom of the crowd is often more important than that of experts.

The Intelligence Advanced Research Projects Activity is running several simultaneous research expeditions into how crowdsourced data could be used to predict specific events, including disease outbreaks or riots. Some analyses have surfaced unexpected connections between seemingly unrelated data and the probability of an event occurring, including that, in aggregate, canceled dinner reservations are pretty good predictors of disease outbreaks.

» Get the best federal technology news and ideas delivered right to your inbox. Sign up here.

In an Open Source Indicators program, IARPA attempted to predict significant events including disease outbreaks and global instability using publicly available data, IARPA Director Jason Matheny said during a Predictive Analytics World conference in Washington on Tuesday. A Virginia Tech team developed a system that incorporated tens of thousands of data streams—including social media data, router traffic and online reservations—and created a forecast for whether a particular event was likely to occur, "like whether or not there was going to be political instability on the streets of Caracas on Tuesday," Matheny explained.

By the end of a forecasting tournament analyzing data from Latin America, China and parts of Africa, OSI was able to generate a forecast seven days out with about 70 to 85 percent accuracy, he said.

IARPA also has a Cyber Attack Automated Unconventional Sensor Environment project intended to predict when cyberattacks are being planned, pulling data from chatter on hackers forums, web search queries, and changes in the prices of malware on black markets, he said.

 "When people start buying malware ...there's a price response," Matheny said.

So far, forecasting economic instability has been a distinct challenge, Matheny said.

"This is a really, really hard problem because if you could do this accurately, you would then able to outperform Wall Street," he said. "Despite our best efforts, we have not been able to beat Wall Street."

Threatwatch Alert

Thousands of cyber attacks occur each day

See the latest threats


Close [ x ] More from Nextgov

Thank you for subscribing to newsletters from
We think these reports might interest you:

  • Modernizing IT for Mission Success

    Surveying Federal and Defense Leaders on Priorities and Challenges at the Tactical Edge

  • Communicating Innovation in Federal Government

    Federal Government spending on ‘obsolete technology’ continues to increase. Supporting the twin pillars of improved digital service delivery for citizens on the one hand, and the increasingly optimized and flexible working practices for federal employees on the other, are neither easy nor inexpensive tasks. This whitepaper explores how federal agencies can leverage the value of existing agency technology assets while offering IT leaders the ability to implement the kind of employee productivity, citizen service improvements and security demanded by federal oversight.

  • Effective Ransomware Response

    This whitepaper provides an overview and understanding of ransomware and how to successfully combat it.

  • Forecasting Cloud's Future

    Conversations with Federal, State, and Local Technology Leaders on Cloud-Driven Digital Transformation

  • IT Transformation Trends: Flash Storage as a Strategic IT Asset

    MIT Technology Review: Flash Storage As a Strategic IT Asset For the first time in decades, IT leaders now consider all-flash storage as a strategic IT asset. IT has become a new operating model that enables self-service with high performance, density and resiliency. It also offers the self-service agility of the public cloud combined with the security, performance, and cost-effectiveness of a private cloud. Download this MIT Technology Review paper to learn more about how all-flash storage is transforming the data center.


When you download a report, your information may be shared with the underwriters of that document.