Treatment program agency shows how a dashboard can work

Director H. Westley Clark says the dashboard will provide accountability to SAMHSA. James Kegley

The Substance Abuse and Mental Health Services Administration has been relying on dashboards for more than half a decade to obtain funding from Congress for its mission to provide treatment programs, its director said on Monday at a conference on using metrics for decision-making.

The White House is pushing federal agencies to establish and track performance measures on spreadsheets, charts and other information technology applications, tools collectively referred to as dashboards. President Obama's fiscal 2011 budget request proposes departments follow the progress of everything from IT acquisitions to staffing using online dashboards for the public to view.

SAMHSA was one of the first agencies to launch a dashboard. Since 2003, the agency has used an internal dashboard called the Services Accountability Improvement System since 2003 to monitor the results of grants it has given communities to address substance abuse and mental illness.

With SAIS, "we're using the data to explain to people who write the checks and ask whether this is a program that should be continued or not," said SAMHSA Director H. Westley Clark at an annual conference on government performance management. "If we can't account for it, they give the money to someone else." The Performance Institute, a research group focused on results-oriented government, hosted the event.

SAMHSA's mission is to reduce the adverse effects that substance abuse and mental illness inflict on communities by funding behavioral health services in the neediest states and communities. The agency must provide the public with high-quality, up-to-date information about behavioral health issues and treatments.

The agency launched the grant-making accountability system after a White House program assessment rated its Substance Abuse Prevention and Treatment block grant program as ineffective.

The system collects data from grant recipients to determine, for example, the percentage decline in substance abuse as reported in a specific program or the percentage increase in employment as a result of an intervention such as peer support. The tool also tracks population-specific data so the agency can assess how well ethnic groups such as Caucasians, Hispanics and others are responding to treatment. For example, for Hispanics and Latinos, grantees reported that 37.1 percent of patients were employed when they were admitted into substance abuse treatment programs. After six months in the programs, 49.5 percent had a job, according to information submitted to SAMHSA.

Currently, grantees can access their own data in the system and SAMHSA can view everyone's information. But for now, the public cannot see the data. Clark said the agency has plans to benchmark and release the information, but officials first must develop a way to compare the outcomes of patients or communities in different situations.

Grantees' reporting requirements are similar to the data collection and analysis that soon will be required from health care providers and communities to identify best practices, under the recently passed health care reform law, according to Clark. "It's not that we were being overly burdensome. It's that we were being prescient," he noted.

An audience member pointed out during a question-and-answer session that the performance assessments are self-reported, and she asked what controls SAMHSA has instituted to ensure recipients aren't skewing data in their favor.

The agency conducts random, independent audits to verify the accuracy of the reporting and relies on whistleblowers, Clark said. "I'm not saying that our system is foolproof. . . . People do game the system," but spending federal dollars to track federal spending would become expensive, he added.