recommended reading

NSA Hid Explosions, Delays at $1.2 Billion Spy Compound

NSA's Utah Data Center, shown in June, occupies 200 acres in Utah.

NSA's Utah Data Center, shown in June, occupies 200 acres in Utah. // Rick Bowmer/AP file photo

While the government was nearing a shutdown and bracing for a default, a $1.2 billion National Security Agency supercomputer homestead in Utah, built to ingest surveillance data, reportedly was spending hundreds of thousands of dollars to replace fried equipment.

NSA officials masked the woes out west, at a time when the agency already was facing accusations of lying about domestic spying practices.

As recently as June 19, agency officials told Nextgov the spy center -- 15 times the size of MetLife stadium -- would be "operational" by Oct. 1. That month, NSA spokeswoman Vanee Vines also said in an email the "exterior is done.  . .  It's now time for IT guts to be installed,” referring to information technology systems.

But, behind the security perimeter, persistent electrical surges apparently sparked explosions and a year-long delay that NSA officials did not disclose. On Monday night, the Wall Street Journal reported there have been 10 meltdowns during the past 13 months that have prevented NSA from running computers.

The revelations come after ex-NSA contractor Edward Snowden this spring leaked files to the media detailing the agency’s mammoth collections of Internet and phone data, revelations later acknowledged by U.S. officials. The 200-acre Utah computer fortress was designed to warehouse this data.

In June, Harvey Davis, a top agency official, who handles military installations, as well as power, space and cooling for all NSA data centers, said in an interview that the construction – undertaken by the Corps of Engineers – has been an open project. 

"The military construction process by design is a very, very transparent process,” he said. “It's a public discourse. When we give out our request for proposal, that's through FedBizOpps.gov," a website that advertises jobs for prospective vendors, so "by its nature, military construction is a very, very open process."

Project contract papers called for a 65-megawatt center with a chiller plant, fire suppression systems, electrical generators and an uninterruptible power supply backup capacity.

On Tuesday, NSA and Army Corps of Engineers officials declined to answer questions about the failure to disclose that there would be some sort of delay.

Norbert Suter, the Corps chief of construction operations, said in an email that power connectivity remains incomplete. "The final commissioning of the entire electrical system is still ongoing," he said, declining to address cost overruns. "The Corps is ensuring it functions as required and that it will be completely reliable prior to releasing the project to the customer,” meaning NSA. Suter did not say when the site would turn on. 

He acknowledged there have been a series of glitches. "During the testing and commissioning of the Utah data center, problems were discovered with certain parts of the electrical system. Issues such as these can arise in any project, and are the reason the Corps tests and reviews every aspect of any project prior to releasing it to the customer," he said.

More Corps professionals were summoned and the source of the problems was discovered, Suter said, without elaborating further. 

"Because of the scope of this project, a specialized team of experts from the Corps' headquarters was sent to review and assist with addressing these issues. The cause of the electrical issues was identified by the team and is currently being corrected by the contractor," he said.

The team also uncovered ventilation troubles. In order to function, data centers need an air management system able to cool IT equipment and remove waste heat. Suter said, "An airflow issue was discovered with the generators and the contractor is correcting the problem."

On Monday night, Vines said, "On the whole, this is why acceptance testing is done - so that issues can be found and resolved before the government takes possession."

While final assessments continue, Suter said the Corps is "making preparations to release it to the customer.”

The final tab for the ultimate customer, the taxpayer, is a question mark. Davis this summer said, "I came in under cost,” referring to $100 million in savings gained partly by refusing to let contractors adjust the plan. 

This story has been updated with minor edits.

Threatwatch Alert

Thousands of cyber attacks occur each day

See the latest threats

JOIN THE DISCUSSION

Close [ x ] More from Nextgov
 
 

Thank you for subscribing to newsletters from Nextgov.com.
We think these reports might interest you:

  • Modernizing IT for Mission Success

    Surveying Federal and Defense Leaders on Priorities and Challenges at the Tactical Edge

    Download
  • Communicating Innovation in Federal Government

    Federal Government spending on ‘obsolete technology’ continues to increase. Supporting the twin pillars of improved digital service delivery for citizens on the one hand, and the increasingly optimized and flexible working practices for federal employees on the other, are neither easy nor inexpensive tasks. This whitepaper explores how federal agencies can leverage the value of existing agency technology assets while offering IT leaders the ability to implement the kind of employee productivity, citizen service improvements and security demanded by federal oversight.

    Download
  • Effective Ransomware Response

    This whitepaper provides an overview and understanding of ransomware and how to successfully combat it.

    Download
  • Forecasting Cloud's Future

    Conversations with Federal, State, and Local Technology Leaders on Cloud-Driven Digital Transformation

    Download
  • IT Transformation Trends: Flash Storage as a Strategic IT Asset

    MIT Technology Review: Flash Storage As a Strategic IT Asset For the first time in decades, IT leaders now consider all-flash storage as a strategic IT asset. IT has become a new operating model that enables self-service with high performance, density and resiliency. It also offers the self-service agility of the public cloud combined with the security, performance, and cost-effectiveness of a private cloud. Download this MIT Technology Review paper to learn more about how all-flash storage is transforming the data center.

    Download

When you download a report, your information may be shared with the underwriters of that document.