The truth behind transparency

In a world where open data doesn’t always mean open government, progress is tough to gauge.

Put a handful of media reports from the days before Sunshine Week in March next to a series of White House blog entries on keeping citizens informed and you might think they’re talking about two different governments. 

The journalists tell a story of un-answered Freedom of Information Act requests, evasive public information officers and an intransigent, classify-first culture that has shown little improvement since the Bush years. The White House, by contrast, calls its commitment to transparency “unprecedented.” 

Politico reporter Josh Gerstein opened a March 5 article quoting longtime Washington attorney Katherine Meyer, who has filed numerous FOIA requests during her career. She called the Obama administration “the worst on FOIA issues” since the law was passed during the Johnson administration. 

The next week, during a March 13 Senate Judiciary hearing, Sens. Charles Grassley, R-Iowa, and Patrick Leahy, D-Vt., pummeled the Office of Management and Budget for stalling proposed FOIA reforms. “There’s a complete disconnect between the president’s grand pronouncements about transparency and the actions of his political appointees,” Grassley said. 

A September 2011 survey by the Columbia Journalism Review and the nonprofit public interest news organization ProPublica found that 30 percent of science, health and environmental reporters gave Obama a “poor” or “very poor” transparency grade on these issues, not much better than the Bush administration’s 44 percent.

The White House, on the other hand, opened Sunshine Week on March 12 by rattling off eight initiatives it said are making government more transparent than ever. By some counts that’s actually a low number. The U.S. National Action Plan for the international Open Government Partnership lists 18 major transparency initiatives and 26 smaller initiatives, plus eight goals that largely have been achieved. 

A handful of disputed facts contribute to this disparity in perception.

The transparency-focused National Security Archive, for example, has accused the Justice Department of grossly exaggerating an advertised 95 percent FOIA response rate. The actual rate, they say, is closer to 60 percent.

The real conflict, though, is about a broader question: Just what does government transparency mean in the Digital Age?

Most of the criticisms made by Gerstein, Grassley and others center on the sort of basic accountability questions that haven’t changed much in the past 40 years: How quickly and fully do agencies respond to Freedom of Information Act requests? Are public officials available for press interviews? And how much will they answer once you’re in the room with them? The chief audience for this information consists of journalists, transparency groups, and through them, the American public. 

Many of the advances touted by government officials have been about open data: the Internet-age concept of making raw information from federal surveys, studies and satellite feeds automatically available to the public in clean, machine-readable formats. The primary consumers are private developers who can use that data to build Web and mobile applicationssometimes to improve government service delivery and help benefit the public, sometimes to turn a profit, or sometimes a combination of the two.   

The hallmark of the Obama administration’s open data movement is Data.gov, a unified repository for federally produced data sets on everything from flight delays to soil contamination rates to the nation’s broccoli consumption. 

Another component is Challenge.gov, where federal agencies can hold competitions, sometimes with cash prizes, for the best use of government data to achieve civic, environmental or even security goals. 

The Environmental Protection Agency, for instance, has sponsored challenges to build mobile apps supporting more efficient electricity use. NASA is sponsoring numerous competitions, including one to build computer code that instructs a satellite to grab and hold on to an object “tumbling through space.” NASA will test the winning code aboard the International Space Station.

Fuzzy Usage

The increased focus on open data, especially since the start of the Obama administration, has caused an identity crisis for some transparency purists who point out that governments can have strong open data records while being otherwise opaque or even repressive. 

Saudi Arabia, for example, launched an open data portal in 2011. The site is full of spreadsheets detailing the oil-rich desert kingdom’s retail trade and agricultural grants. There’s nothing there, though, about the government’s murky internal operations, women’s access to education or the working conditions of foreign laborers. 

Data.gov is much more expansive than Saudi Arabia’s site—nearly 400,000 data sets so far—but only a handful are directly related to government accountability, mostly pooled under the “data community” Ethics.gov. 

The go-to example of federal open data is about as politically neutral as it gets—the National Oceanic and Atmospheric Association, which pumps out real-time, machine-readable information from weather satellites that shows up everywhere from the Weather Channel to iPhone applications, with most people never realizing the government has a hand in it. 

Commonly cited examples at the local level are public transportation mobile apps such as Washington’s NextBus and NextTrain, which rely on data from Global Positioning System trackers to pinpoint the location of buses, subways and trolley cars.

The conflict between traditional open government and open data flared briefly in May when Tom Slee, a Canadian author and software product manager who blogs about technology and economics, filed a post titled “Why the ‘Open Data Movement’ Is a Joke.” The piece set off a barrage of rebuttals from open data proponents and transparency groups, including the Sunlight Foundation. 

Slee complained the Canadian government was winning plaudits for open data advances at the same time it was cutting back on real transparency, including limiting media access to government scientists. He accused open data activists of being, at best, politically neutral about government accountability and, at worst, cloaking “a private sector initiative in the mantle of progressive politics,” essentially repackaging taxpayer-funded data so that they could sell it back to those same taxpayers at a premium.

Critics of Slee’s position largely carried the day but the battle produced a lot of online soul searching in the open government and open data worlds.  

Researchers Harlan Yu and David Robinson say the real problem is fuzzy usage. In a February paper titled “The New Ambiguity of Open Government,” they describe accountability-focused open government and technology-driven open data as largely separate ideas that have been mushed together by the Obama administration, muddying the meaning of transparency and clouding the government’s actual record.

“In the last three years there’s been an increased blurring of the two ideas,” Yu says. “I don’t think there’s anything nefarious behind the merging, but you have two communities that in many ways are talking past each other.”

Yu is a doctoral candidate in computer science at Princeton University and Robinson is a Yale Law School student. 

Beth Noveck, a former deputy federal chief technology officer who helped launch the administration’s flagship Open Government Initiative in 2009, struck a similar note in an April 2011 Huffington Post essay. She said she regretted using the term “open government” because it caused confusion about the campaign’s major goals.

“Many people, even in the White House, still assume that open government means transparency about government,” Noveck wrote. Instead, she said, it was meant to be “shorthand for open innovation or the idea that working in a transparent, participatory and collaborative fashion helps improve performance, inform decision-making, encourage entrepreneurship and solve problems more effectively.”

Behind the Data

The distinction between open government and open data isn’t really so simple, some transparency advocates say. Seemingly innocuous spreadsheets about water contamination rates could reveal a rotten city waste disposal contract. Or Agriculture Department information about federal farm subsidies could be the starting point for an investigation into lobbyists’ influence on the Farm Bill.  

Even the sort of data that goes into public transportation apps can serve as a back door to questions about local government accountability. 

Urban planning student David Daddio, for example, crunched together the usage numbers for Washington’s Capital Bikeshare program with information about demographics, retail density and public transportation for his master’s thesis at the University
of North Carolina at Chapel Hill.

Daddio’s findings not only showed how the system’s bike share racks could be distributed more effectively, they also suggested how political considerations might have influenced initial rack placements, leading to more racks than necessary outside the city’s center.

The main goals of Data.gov and Challenge.gov are to enlist the public to help improve government services and to use government data to launch private sector products. But the more outsiders look at agency data the more accountable agencies might become. 

Data.gov staffers spend much of their time urging federal agencies to put more of their work online, whether it’s raw data, analyses or commentary, according to Jeanne Holm, director of the site. “In my opinion open government encompasses a lot of different activities, only some of which have to do with technology,” she says. “I’m a strong advocate that we should be sharing as much data as practical from the government, and Data.gov is just a drop in the bucket. We all realize that.”

Accounts and Accountability

As the concept of transparency expands, White House officials have attempted to be more specific about how they use
the term.

When agencies released 2.0 versions of their open government plans in April, a White House blog post by Office of Information and Regulatory Affairs Administrator Cass Sunstein and Deputy Chief Technology Officer Chris Vein described three open government goals. The first, they said, is to “increase accountability.” The second is to “provide people with information that they can readily find and use in their daily lives,” and the third is to “use the dispersed knowledge of the American people.”

Sometimes it seems the transparency tent has become so broad, however, that it’s difficult to have a clear conversation. 

When Politico reporter Gerstein sought official comment for his Sunshine Week article, which focused almost entirely on FOIA issues, White House spokesman Eric Schultz responded in the context of open data, noting administration efforts “to provide people with information that they can use in their daily lives and to solicit public participation in government decision-making and thus tap the expertise that resides outside of government.”

Similarly, many critics have complained that White House visitor logs, which the Obama administration began publishing in 2009, don’t include meetings outside the White House and often list the names of staffers arranging clearances rather than the officials meeting with visitors. In an earlier era that would have been a simple question of accountability. 

In a September 2011 status report on the Open Government Initiative, White House officials said it was a matter of open data. The criticism was “misplaced, they said, because “the visitor entry system—used by previous administrations as well—was created
solely for the purposes of ensuring secure access to the White House complex,” not to record White House visitors. They did not address the issue of gathering more data to meet the level of accountability the critics sought. 

One reason White House initiatives have tended more toward open data and technology-enabled service delivery than accountability is simply because those goals are easier to tackle.

The Obama team, after all, is a tech savvy bunch, heralded for running the most digitally dynamic presidential campaign in history. Obama’s predecessors underutilized new technology in the White House, so there was a lot of room to catch up.

“The Obama administration, I’d argue, is the first truly modern administration when it comes to capitalizing on digital technology,” says Columbia Journalism Review Science Editor Curtis Brainard. “For whatever reason, the Clinton and Bush administrations just didn’t get it, and we saw that change from the earliest days of the Obama campaign.” 

Boosting accountability, though, is tougher than building new websites. It means changing the way thousands of federal workers do their jobs and how much trust they put in public oversight. 

On Obama’s first day in office he issued a memorandum instructing agencies to release more information under FOIA, to improve how they administer FOIA requests and to do their work with a presumption of openness. Memos followed in quick succession from the White House counsel, the White House chief of staff, the attorney general, and other department and agency heads. The result, by the end of the administration’s first full fiscal year, was a 6 percent increase in fully responsive FOIA replies—to 56 percent—and a 10 percent reduction in the backlog of requests.

That’s a respectable accomplishment considering the slow pace of many government processes. It might even be impressive considering the uptick in FOIA responsiveness was the first in a decade. But it’s sour medicine for fervent Obama supporters who expected a watershed change. And it certainly doesn’t carry the same aura of success as Data.gov, which developed from a nebulous idea to hosting 250,000 data sets in just one year. 

Turning Up Transparency

Advocates acknowledge that the merging of open government and open data as a method of improving service delivery has made the concept of transparency much more difficult to pin down. The broadening definition is worth it, many say, because it transitions transparency from a niche industry for intelligence-gathering journalists and eggheads into information American citizens can see in their daily lives. 

“We think it’s important that campaign finance data and government spending data and visitors logs are open and online,” says Gavin Baker, a federal information policy analyst at the nonprofit public interest group OMB Watch. “And that is very important. But the thing is, 90-plus percent of the American public is never going to look at that.” He notes that many people do use government-provided information such as weather forecasts, nutrition guidelines, and health and safety warnings.

“By making open government into a term that includes those other purposes, it becomes something more people can actually use and identify and relate with,” Baker says. “The downside is you lose a bit of clarity.”

Daniel Schuman, policy counsel for the Sunlight Foundation, agrees. “Open data is the best tool to disseminate the most information to the most people and that’s what the Internet is about,” he says. “But it’s just a tool; it’s not an end goal. The end goal is a more democratic society with all the values we care about.”

Schuman is among the strongest proponents of open data as a transparency tool that can ultimately raise accountability. He envisions something like a data-first, and (with minor exceptions) wholly transparent, government where everything down to officials’ email chatter and meeting minutes would be published online and metatagged so that journalists and citizens could crawl through it, turning up government lapses where they never would have thought to look before. 

If everything becomes open data, of course, traditional open government concerns such as FOIA responsiveness would be an anachronism.

While that level of automatic government transparency may never be realized, it wouldn’t even have been conceivable before the Internet Age. And a transparency movement that begins with politically neutral data could raise agencies’ comfort level, paving the way for the release of more politically sensitive information. 

“We’re trying to create a cultural change across government,” says Holm of Data.gov. “We’re trying to get people to say at the moment they collect things, ‘Let’s make this public so the public gets information as quickly as possible.’ I grew up at NASA, which biases my perspective. Our mission there is to share as much as possible. As soon as the 1s and 0s come down from the spacecraft we have to have them up on the Web so the public can see it as quickly as the scientists do.”

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.