recommended reading

Cybersecurity index aims to penetrate the fog of marketing hype

It's impossible to fix something if you can't even gauge that it's broken. It's a classic problem that systems engineers and defense contractors face: they are staring into a fog of elusive threats made worse by marketers trying to make a sale on security hype.

Frustrated by this lack of clarity, Dan Geer, chief information security officer of CIA venture capital arm In-Q-Tel, launched a personal project that aims to measure threats in a meaningful way. Teaming up with a financial industry professional, he built a monthly sentiment index to capture the security community's impressions on whether risks to IT systems and networks were rising or falling.

The project, called The Index of Cyber Security, highlights a young, growing movement within the security community to craft metrics that can give professionals direction if they are groping in the dark. The experiment was driven in part by "the despair of the security metrics guy thinking, 'Where am I going to get the kind of aggregate data that allows us to get the big picture?' " Geer said.

"What perpetuates the fog is when different people who try to quantify technology risks may have an ax to grind," added his partner Mukul Pareek, a risk professional working in New York. "So they want to present a numeric representation of risk to sell a product or create marketing gimmicks."

Here's how the year-old index is constructed. Every month, roughly 200 people are polled on how they feel about myriad security threats from industrial espionage to insider threats. The duo doesn't rate actual risks, but evaluates if perceptions of risks are growing or decreasing. "In this way, we do not have to calibrate one respondent to the next such as to ensure that each of them has identical definitions and tastes," Geer said.

To protect the anonymity and privacy of those polled, the survey is electronically configured to not allow anybody -- including Geer and Pareek -- visibility into any respondent's answers. Many are CISOs and risk officers from banks, government agencies and institutions that house sensitive data. The two cast their net by getting referrals from people on the front lines of security. Their goal is to reach 300 respondents. Those who fill in the survey get a monthly aggregate snapshot and analysis of what the pool has submitted, as well as excerpts of comments from respondents, who sometimes reveal how they are tweaking their security budgets.

When the index was first conceived, the pair envisioned that cybersecurity insurance providers could use it to bring transparent pricing models to a market that is notoriously opaque. Another possibility was to propose the index as the basis of a tradable financial product that companies could buy and sell as a hedging tool in an investment portfolio. But they will not fully develop those applications until the index is more mature.

The duo behind The Index of Cyber Security plans to do a formal review of what they have learned in the past year. "We designed the index in a way that adjustments could be made to its components without damaging its continuity," said Geer, "There is a lot of maturity that can happen in the meantime."

Geer is no stranger to the difficulty of creating security metrics. In a separate 2007 collaboration, he created a price index for stolen passwords to routers and credit card information auctioned on websites. His goal was to measure how much hacked data was worth and see how financial incentives for hacking were changing over time. That became tricky when law enforcement officials started shutting down the websites that were put in the limelight, he recalls. It is also difficult to put a price tag on data sought by politically motivated actors.

While the appetite for sounder metrics is growing in the security industry, Geer admits, "it is not the roar of the crowd, but more like the hum of a cocktail party, frankly as evidenced by the reception our index has received."

The index is up 26.4 percent since it was launched, and has risen every month during the past year.

Dawn Lim, a financial reporter in New York, was formerly an intern at NextGov.

Threatwatch Alert

Thousands of cyber attacks occur each day

See the latest threats


Close [ x ] More from Nextgov

Thank you for subscribing to newsletters from
We think these reports might interest you:

  • It’s Time for the Federal Government to Embrace Wireless and Mobility

    The United States has turned a corner on the adoption of mobile phones, tablets and other smart devices, outpacing traditional desktop and laptop sales by a wide margin. This issue brief discusses the state of wireless and mobility in federal government and outlines why now is the time to embrace these technologies in government.

  • Featured Content from RSA Conference: Dissed by NIST

    Learn more about the latest draft of the U.S. National Institute of Standards and Technology guidance document on authentication and lifecycle management.

  • A New Security Architecture for Federal Networks

    Federal government networks are under constant attack, and the number of those attacks is increasing. This issue brief discusses today's threats and a new model for the future.

  • Going Agile:Revolutionizing Federal Digital Services Delivery

    Here’s one indication that times have changed: Harriet Tubman is going to be the next face of the twenty dollar bill. Another sign of change? The way in which the federal government arrived at that decision.

  • Software-Defined Networking

    So many demands are being placed on federal information technology networks, which must handle vast amounts of data, accommodate voice and video, and cope with a multitude of highly connected devices while keeping government information secure from cyber threats. This issue brief discusses the state of SDN in the federal government and the path forward.

  • The New IP: Moving Government Agencies Toward the Network of The Future

    Federal IT managers are looking to modernize legacy network infrastructures that are taxed by growing demands from mobile devices, video, vast amounts of data, and more. This issue brief discusses the federal government network landscape, as well as market, financial force drivers for network modernization.


When you download a report, your information may be shared with the underwriters of that document.