In the United States, it is usually a safe bet to attribute massive government fuck-ups to the bloated contractors we’ve outsourced our projects to.
And the electrical problems plaguing NSA’s new UT data center — described as lightening in a box that has caused $100,000 of damage each of the 10 times it has happened — do seem to stem from poorly supervised contractors.
The Army Corps of Engineers is overseeing the data center’s construction. Chief of Construction Operations, Norbert Suter said, “the cause of the electrical issues was identified by the team, and is currently being corrected by the contractor.” He said the Corps would ensure the center is “completely reliable” before handing it over to the NSA.
But another government assessment concluded the contractor’s proposed solutions fall short and the causes of eight of the failures haven’t been conclusively determined. “We did not find any indication that the proposed equipment modification measures will be effective in preventing future incidents,” said a report last week by special investigators from the Army Corps of Engineers known as a Tiger Team.
It took six months for investigators to determine the causes of two of the failures. In the months that followed, the contractors employed more than 30 independent experts that conducted 160 tests over 50,000 man-hours, according to project documents.
Contractors have started installing devices that insulate the power system from a failure and would reduce damage to the electrical machinery. But the fix wouldn’t prevent the failures, according to project documents and current and former officials.
Now, don’t pee your pants laughing.
But I did have two thoughts as I read this.
First, this extended confusion sounds similar to that which Iranian nuclear scientists experienced as they tried to figure out why their centrifuges kept blowing up, thanks to StuxNet. While I think the chances some kind of hack caused this are small (but not zero), I do find it ironic that we cause ourselves the same kind of havoc we cause our worst enemies.
And consider the mission!
Back in February, Keith Alexander warned of the possibility of cyberattacks on our grid (which, anonymous sources made clear, could probably only be launched by China or Russia, but that didn’t stop Alexander from suggesting Anonymous might launch such attacks). The NSA needs more authority to protect against attacks that might bring down our power sources, the head of the NSA suggested.
But the entity that proposes to wield that authority, it seems, can’t even build a brand spanking new electrical system immune from some kind of failure.