recommended reading

What IARPA Knows About Your Canceled Dinner Reservation

Andrey Bayda/

At the intelligence community's research and development agency, the wisdom of the crowd is often more important than that of experts.

The Intelligence Advanced Research Projects Activity is running several simultaneous research expeditions into how crowdsourced data could be used to predict specific events, including disease outbreaks or riots. Some analyses have surfaced unexpected connections between seemingly unrelated data and the probability of an event occurring, including that, in aggregate, canceled dinner reservations are pretty good predictors of disease outbreaks.

» Get the best federal technology news and ideas delivered right to your inbox. Sign up here.

In an Open Source Indicators program, IARPA attempted to predict significant events including disease outbreaks and global instability using publicly available data, IARPA Director Jason Matheny said during a Predictive Analytics World conference in Washington on Tuesday. A Virginia Tech team developed a system that incorporated tens of thousands of data streams—including social media data, router traffic and online reservations—and created a forecast for whether a particular event was likely to occur, "like whether or not there was going to be political instability on the streets of Caracas on Tuesday," Matheny explained.

By the end of a forecasting tournament analyzing data from Latin America, China and parts of Africa, OSI was able to generate a forecast seven days out with about 70 to 85 percent accuracy, he said.

IARPA also has a Cyber Attack Automated Unconventional Sensor Environment project intended to predict when cyberattacks are being planned, pulling data from chatter on hackers forums, web search queries, and changes in the prices of malware on black markets, he said.

 "When people start buying malware ...there's a price response," Matheny said.

So far, forecasting economic instability has been a distinct challenge, Matheny said.

"This is a really, really hard problem because if you could do this accurately, you would then able to outperform Wall Street," he said. "Despite our best efforts, we have not been able to beat Wall Street."

Threatwatch Alert

Network intrusion

FBI Warns Doctors, Dentists Their FTP Servers Are Targets

See threatwatch report


Close [ x ] More from Nextgov

Thank you for subscribing to newsletters from
We think these reports might interest you:

  • It’s Time for the Federal Government to Embrace Wireless and Mobility

    The United States has turned a corner on the adoption of mobile phones, tablets and other smart devices, outpacing traditional desktop and laptop sales by a wide margin. This issue brief discusses the state of wireless and mobility in federal government and outlines why now is the time to embrace these technologies in government.

  • Featured Content from RSA Conference: Dissed by NIST

    Learn more about the latest draft of the U.S. National Institute of Standards and Technology guidance document on authentication and lifecycle management.

  • A New Security Architecture for Federal Networks

    Federal government networks are under constant attack, and the number of those attacks is increasing. This issue brief discusses today's threats and a new model for the future.

  • Going Agile:Revolutionizing Federal Digital Services Delivery

    Here’s one indication that times have changed: Harriet Tubman is going to be the next face of the twenty dollar bill. Another sign of change? The way in which the federal government arrived at that decision.

  • Software-Defined Networking

    So many demands are being placed on federal information technology networks, which must handle vast amounts of data, accommodate voice and video, and cope with a multitude of highly connected devices while keeping government information secure from cyber threats. This issue brief discusses the state of SDN in the federal government and the path forward.

  • The New IP: Moving Government Agencies Toward the Network of The Future

    Federal IT managers are looking to modernize legacy network infrastructures that are taxed by growing demands from mobile devices, video, vast amounts of data, and more. This issue brief discusses the federal government network landscape, as well as market, financial force drivers for network modernization.


When you download a report, your information may be shared with the underwriters of that document.