Business Intelligence

What is it?

Business intelligence uses software programs to analyze huge amounts of electronic data collected by a federal agency or private company. It is increasingly being used to dig through seemingly unrelated data to find relationships that explain events, and predict future ones, so an organization can plan its strategy accordingly.

It's impossible for humans to keep track of the numerous terabytes and petabytes of information in many government data warehouses or to easily identify trends that are lurking behind the numbers. BI, as it's known, uses software programs to sift through the information to find relationships among the data that otherwise may go unnoticed. The results can help managers determine better ways to operate an organization, improve products or services, identify which strategies are working or not working, or find other opportunities.

For example, according to the Congressional Research Service report "Data Mining and Homeland Security: An Overview," traditional analysis requires someone to develop a hypothesis and then test it based on the data. For example, a hardware store owner may believe that a customer who buys a hammer would also buy a box of nails, therefore leading the store owner to pace the nails near the hammers. The owner then can measure any increase in sales of nails.

But BI allows users to discover multidimensional relationships that never would have occurred to them before. "For example, a hardware store may compare their customers' tool purchases with home ownership, type of automobile driven, age, occupation, income, and/or distance between residence and the store," the report noted. This in turn could lead the store owner to rethink how the business operates and which products to carry.

For the federal government, BI can help an agency find relationships in its data that could lead to better delivery of services to the public, sounder policy decisions, or possibly information that could effect legislation.

BI gives an agency a unified view, or big picture, of what data may be signaling. Until recently, few applications, short of a spreadsheet, could analyze large amounts of information in a timely manner and provide insights. But now, business intelligence gives agencies a far more sophisticated way to look at data using complex modeling that provides deep interpretations. BI builds on tools developed for data mining, the process of finding information stored in a system that may not be easily identifiable.

Who's using it?

The Office of Management and Budget has not mandated the use of business intelligence, but agencies are judged by their performance in providing services and meeting OMB milestones. BI can help them get there, but it is not without cost and detailed planning. The federal government is one of the biggest collectors of electronic data in the world, and BI can help agencies make sense of that enormous cache of information and help use the data in new and unique ways. "It is a higher level of analysis and fundamentals of how the business runs," said Robert Simmers, director of corporate business at Mitre Corp., a nonprofit organization that operates federally funded research and development centers.

Many federal agencies use BI to mine information in databases. Among them are the intelligence agencies and the military for warfare modeling, or predicting the possibility of future terrorist events. The Census Bureau uses business intelligence to analyze and predict demographic trends. Health care agencies, such as the Health and Human Services Department and the Centers for Disease Control and Prevention, use it to analyze and predict health-related outcomes and to detect patterns of fraud in benefit claims.

The Bureau of Labor Statistics uses BI to analyze economic statistics such as prices and unemployment rates. The Securities and Exchange Commission turns to business intelligence to help spot insider trading, while the Internal Revenue Service detects filing anomalies with it. And BI can be applied as a financial management tool to track how an agency spends its budget and whether there are more efficient ways to do so.

Why should I care?

Simply put, BI will make your job easier. With the government collecting ever larger amounts of data, it provides a good way to answer the question: "What will happen next?" It may enable agencies to better plan for bad weather, a poor crop, a pandemic or more roads to handle the growing population in one region of the country.

"Through the years of using information and understanding predictions, you are able to anticipate the catastrophic event," said Bill Hostmann, vice president and analyst for GartnerInc., an information technology and research firm. "You have leading indicators saying there may be an event occurring. The analysis forks you into a new direction."

Business intelligence steps in to detect patterns that are too big to see with the naked eye. It can help you be proactive instead of reactive. It allows you to monitor a data system on a daily basis, divide it into relevant pieces and ignore parts that are not relevant.

With Congress on the lookout for mismanagement as well as fraud, waste and abuse in federal programs, BI can provide ways to reduce the risk of errors and therefore reduce the chance that you may be called to defend poor performance.

Are there any downsides?

One of the biggest obstacles for BI is privacy. Who gets to see the data, and who decides who gets to see the data? What form should it take?

It is important to have standard policies about access to information and how it can be used. Are government personnel allowed to drill down to personal information and identity? And how is data protected from unauthorized disclosure or a phishing expedition? Agencies must determine who gets to see protected information and how the rules are enforced. An information security framework should be in place to identify the sensitivity levels of information and what must be protected.

While it can provide eureka-type of insights into an agency's business and how it can accomplish its mission, business intelligence isn't a silver bullet. Business intelligence does reveal patterns and relationships between characteristics within the data, but it frequently doesn't give the user information on how significant those relationships are. It also doesn't necessarily reveal the causal relationship between different data sets.

How to get started

You will need help. Whether you tap outside expertise or work within your agency, you will need a roadmap to get the right outcome. BI isn't easy.

First, you must determine what information you have, where it is stored and what you want to get out of it. It's important to turn to experts who can help you analyze what you have, where to find it and know how to use the technology to get what you need. Large vendors have recognized the importance of business intelligence competency and many have begun to buy smaller BI companies. Oracle purchased Hyperion, a provider of performance management software, and Siebel Analytics, which provides prebuilt BI solutions. IBM announced in late 2007 it would buy BI company Cognos.

How much does BI cost?

It is difficult to identify exactly how much it will cost you to implement a BI program. What you want to do and the size of the job are only two variables in a complex equation. For sure, the job will involve a software tool or suite that can capture information and put it into easily understood language. The cost will depend on how much work you want BI to perform and how deep it has to dive into your data, and whether it scans all your data or specific pieces of it. Deploying a system can be done in modules and stages; everything doesn't have to be done at once. Ultimately,it will make your operation more efficient and save money down the road.

"Every system needs to have some sort of management view of the data; I think it ought to be part of everything," said David Kreigman, president of Command Information's federal division, a Vienna, Va., solutions provider. "Unlike other technologies that require agencies to change what they are doing, this makes use of data without making any change.