One Year After Collateral Murder Release, DOD’s Networks Are Still Glaring Security Problem

As I have posted several times, the response to WikiLeaks has ignored one entity that bears some responsibility for the leaks: DOD’s IT.

Back in 2008, someone introduced malware to DOD’s computer systems. In response, DOD announced it would no longer allow the use of removable media in DOD networks. Yet that is precisely how Bradley Manning is reported to have gotten the databases allegedly leaked. In other words, had DOD had very basic security measures in place they had already been warned they needed, it would have been a lot harder for anyone to access and leak these documents.

Often, when I have raised this issue, people are simply incredulous that DOD’s classified network would be accessible to removable media (and would have remained so two years after malware was introduced via such means). But it’s even worse than that.

A little-noticed Senate Homeland Security hearing last month (Steven Aftergood is one of the few people who noticed) provided more details about the status of DOD’s networks when the leaks took place and what DOD and the rest of government have done since. The short version is this: for over two months after DOD arrested Bradley Manning for allegedly leaking a bunch of material by downloading information onto a Lady Gaga CD, DOD and the State Department did nothing. In August, only after WikiLeaks published the Afghan War Logs, they started to assess what had gone wrong. And their description of what went wrong reveals not only how exposed DOD was, but how exposed it remains.

Two months to respond

Bradley Manning was arrested on or before May 29. Yet in spite of claims he is alleged to have made in chat logs about downloading three major databases, neither DOD or State started responding to the leak until after the Afghan War Logs were published on July 25, 2010.

The joint testimony of DOD’s Chief Information Officer Teresa Takai and Principal Deputy Under Secretary for Intelligence Thomas Ferguson explains,

On August 12, 2010, immediately following the first release of documents, the Secretary of Defense commissioned two internal DoD studies. The first study, led by the Under Secretary of Defense for Intelligence (USD(I)), directed a review of DoD information security policy. The second study, led by the Joint Staff, focused on procedures for handling classified information in forward deployed areas.

In other words, “immediately” (as in, more than two weeks) after the publication of material that chat logs (published two months earlier) had clearly explained that Manning had allegedly downloaded via Lady Gaga CD months earlier, DOD commissioned two studies.

As State Department Under Secretary of Management Patrick Kennedy explained, their response was no quicker.

When DoD material was leaked in July 2010, we worked with DoD to identify any alleged State Department material that was in WikiLeaks’ possession.

It wasn’t until November–at around the time when NYT was telling State precisely what they were going to publish–that State started responding in earnest. At that time–over four months after chat logs showed Manning claiming to have downloaded 250,000 State cables–State moved its Net Centric Diplomacy database from SIPRNet (that is, the classified network) to JWICS (the Top Secret network).

DOD’s exposed IT networks

Now, frankly, State deserves almost none of the blame here. Kennedy’s testimony made it clear that, while the WikiLeaks leak has led State to enhance their limits on the use of removable media access, they have systems in place to track precisely who is accessing data where.

DOD won’t have that across their system for another year, at least.

There are three big problems with DOD’s information security. First, as the Takai/Ferguson testimony summarized,

Forward deployed units maintained an over-reliance on removable electronic storage media.

It explains further that to make sure people in the field can share information with coalition partners, they have to keep a certain number of computers accessible to removable media.

The most expedient remedy for the vulnerability that led to the WikiLeaks disclosure was to prevent the ability to remove large amounts of data from the classified network. This recommendation, forwarded in both the USD(I) and Joint Staff assessments, considered the operational impact of severely limiting users’ ability to move data from SIPRNet to other networks (such as coalition networks) or to weapons platforms. The impact was determined to be acceptable if a small number of computers retained the ability to write to removable media for operational reasons and under strict controls.

As they did in 2008 after malware was introduced via thumb drive, DOD has promised to shut off access to removable media (note, Ferguson testified thumb drives, but not CDs, have been shut down for “some time”). But 12% of the computers on SIPRNet will still be accessed by removable media, though they are in the process of implementing real-time Host Based Security System tracking of authorized and unauthorized attempts to save information on removable media for those computers.

In response to a very frustrated question from Senator Collins, Ferguson explained that DOD started implementing a Host Based Security System in 2008 (the year DOD got infected with malware). But at the time of the leak, just 40% of the systems in the continental US had that system in place; it was not implemented outside of the US, though. They weren’t implemented overseas, he explained, because a lot of the systems in the field “are cobbled together.”

In any case, HBSS software will be in place by June. (Tech folks: Does this means those computers are still vulnerable to malware introduced by removable media? What about unauthorized software uploads?)

Then there’s data access control. DOD says it can’t (won’t) password protect access to information because managing passwords to control the access of 500,000 people is too onerous for an agency with a budget larger than Australia’s gross national product. Frankly, that may well be a fair approach given the importance of sharing information.

But what is astounding is that DOD is only now implementing public key infrastructure that will, first of all, make it possible to track what people access and–some time after DOD collects that data–to start fine tuning what they can access.

DoD has begun to issue a Public Key Infrastructure (PKI)-based identity credential on a hardened smart card. This is very similar to the Common Access Card (CAC) we use on our unclassified network. We will complete issuing 500,000 cards to our SIPRNet users, along with card readers and software, by the end of 2012. This will provide very strong identification of the person accessing the network and requesting data. It will both deter bad behavior and require absolute identification of who is accessing data and managing that access.

In conjunction with this, all DoD organizations will configure their SIPRNet-based systems to use the PKI credentials to strongly authenticate end-users who are accessing information in the system. This provides the link between end users and the specific data they can access – not just network access. This should, based on our experience on the unclassified networks, be straightforward.

DoD’s goal is that by 2013, following completion of credential issuance, all SIPRNet users will log into their local computers with their SIPRNet PKI/smart card credential. This will mirror what we already do on the unclassified networks with CACs.

[Takai defines what they’re doing somewhat just before 88:00]

Note what this says: DOD is only now beginning to issue the kind of user-based access keys to protect its classified network that medium-sized private companies use. And unless I’m misunderstanding this, it means DOD is only now upgrading the security on its classified system to match what already exists on its unclassified system.

Let’s hope nothing happens between now and that day in 2013 when all this is done.

And this particular problem appears to exist beyond DOD. While the two DIA witnesses mostly blew smoke rather than provide a real sense of where security is at (both blamed WikiLeaks on a “bad apple” rather than shockingly bad information security), the testimony of DNI’s Intelligence Community Intelligence Sharing Executive Corin Stone seems to suggest other parts of the IC area also still implementing the kind of authentication most medium sized corporations employ.

To enable strong network authentication and ensure that networks and systems can authoritatively identify who is accessing classified information, the IC CIO is implementing user authentication technologies and is working with the IC elements to achieve certificate issuance to eligible IC personnel in the first quarter of fiscal year 2012.

So that’s the issue of removable media and individualized access tracking.

Which leaves one more big security hole. According to Takai/Ferguson, DOD didn’t–still didn’t, as of mid-March–have the resources in place to detect anomalous behavior on its networks.

Limited capability currently exists to detect and monitor anomalous behavior on classified computer networks.

This confirms something Manning said in chat logs: no one is following the activity occurring on our networks in Iraq (or anywhere else on SIPRNet, from the sounds of things), and flagging activities that might be an intrusion.

The part of the Takai/Ferguson testimony that details very hazy plans to think about maybe implementing such a system (pages 6-7) is worth a gander just for the number of acronyms of titles of people who are considering maybe what to implement some time in the future. It’s all a bunch of bureaucratic camouflage, IMO, to avoid saying clearly, “we haven’t got it and we haven’t yet figured out how we’re going to get it.” But here are the two most concrete descriptions of what the Department of Defense plans to do to make sure no one is fiddling in their classified networks. First, once they get HBSS completely installed, then they will install an NSA audit program on top of that.

One very promising capability is the Audit Extraction Module (AEM) developed by the National Security Agency (NSA). This software leverages already existing audit capabilities and reports to the network operators on selected audit events that indicate questionable behavior. A great advantage is that it can be integrated into the HBSS we have already installed on the network, and so deployment should be relatively inexpensive and timely. AEM is being integrated into HBSS now and will be operationally piloted this summer.

But in the very next paragraph, Takai/Ferguson admit there are better solutions out there. But DOD (again, with its budget larger than the GNP of most medium sized countries) can’t implement those options.

Commercial counterintelligence and law enforcement tools – mostly used by the intelligence community – are also being examined and will be a part of the overall DoD insider threat program. These tools provide much more capability than the AEM. However, while currently in use in some agencies, they are expensive to deploy and sustain even when used in small, homogeneous networks. Widespread deployment in DoD will be a challenge.

In other words, DOD wants to be the biggest part of the intelligence community. But it and its budget bigger than Brazil’s GNP won’t implement the kind of solutions the rest of the intelligence community use.

Department. Of. Defense.

Now, let me be clear: DOD’s embarrassingly bad information security does not, in any way, excuse Bradley Manning or the other “bad apples” we don’t know about from their oath to protect this information. (Note, there was also testimony that showed DOD’s policies on information sharing were not uniformly accessible, but that’s minor compared to these big vulnerabilities.)

But in a world with even minimal accountability, we’d be talking about fixing this yesterday, not in 2013 (five years, after all, after the malware intrusion). We’d have fired the people who let this vulnerability remain after the malware intrusion. We’d aspire to the best kind of security, rather than declaring helplessness because our very expensive DOD systems were kluged together. And we’d be grateful, to a degree, that this was exposed with as little reported damage as it has caused.

If this information is really classified for good reason, as all the hand-wringers claim, then we ought to be using at least the kind of information security implemented by the private sector a decade ago. But we’re not. And we don’t plan on doing so anytime in the near future.

Marcy has been blogging full time since 2007. She’s known for her live-blogging of the Scooter Libby trial, her discovery of the number of times Khalid Sheikh Mohammed was waterboarded, and generally for her weedy analysis of document dumps.

Marcy Wheeler is an independent journalist writing about national security and civil liberties. She writes as emptywheel at her eponymous blog, publishes at outlets including the Guardian, Salon, and the Progressive, and appears frequently on television and radio. She is the author of Anatomy of Deceit, a primer on the CIA leak investigation, and liveblogged the Scooter Libby trial.

Marcy has a PhD from the University of Michigan, where she researched the “feuilleton,” a short conversational newspaper form that has proven important in times of heightened censorship. Before and after her time in academics, Marcy provided documentation consulting for corporations in the auto, tech, and energy industries. She lives with her spouse and dog in Grand Rapids, MI.

  1. earlofhuntingdon says:

    The DoD must have hired computer systems experts from the FBI. Or maybe they hired the GOP hacks Karl Rove brought in to misdesign the White House’s dysfunctional Bush-era e-mail system. (Fixing it, assuming Obama looked back on at least that transgression, probably cost $100 million).

    • JMLagain says:

      Don’t underestimate Rove and his computer guys. There was/is method to the madness. And speaking of Rovian uploads and downloads, why is there a concerted effort to play deaf, blind and mute with the circumstances of our electronic voting machines? Someday it will be recognized that this tinfoil hat is an avant garde fashion statement.

      • earlofhuntingdon says:

        Oh, I know there was deceptive method in Rove’s madness; that “system” somehow lost the White House and Dick Cheney’s most sensitive records. The other ones were sent to the Bush library. It wasn’t perfect though. A few must have slipped onto the servers or printers, because there was that infamous “fire” in the West Wing.

        • Gitcheegumee says:

          And the untimely death of GOP IT wunderkind Mike Connell, who crashed his plane and died just prior to testifying regarding the irregularities in 2004 Ohio elections.

          He was involved with setting up the WH systems,in addition to desigining the man in the middle servers that allegedly rerouted vots to a server in Tennessee.

          And while we are on the subject,does anyone recall Abramoff repping for some IT firms involving the WH,in addition to his casino lobbying activity?

          • Gitcheegumee says:

            To further amplify, Abramoff receved a $640,000 fee in 2003 and 2004 to lobby for Unisys.(The charges involving Abramoff in subsequent years do NOT relate to Unysis,btw.)

            According to Wiki and Congressional reports, security breaches involving DHS- and major billing irregularities-involving Unisys back in ’06 and ’07. This breach ,allegedly involving Chinese hackers ,resulted in Unisys not being awarded a new contract for DHS security systems.

            Unisys was at first going to file GAO a complaint about being left out of a new bid contract ,but eventually demurred.

            Unisys has a most interesting WIKI,and according to the entry, IRS,SWIFT,Transportation Safety Board,among others, are listed as clients. (SWIFT was the subject of a recent thread here.)

            It is well worth a read.

  2. earlofhuntingdon says:

    Item in the president’s daily national security briefing:

    “Whistleblowers determined to copy digital records of government malfeasance.”

    “OK, you’ve covered your ass. Now watch this jump shot.”

  3. earlofhuntingdon says:

    The DoD’s intelligence apparatus spends billions monitoring US and global e-mails and telecoms. It even has time to engage would be “experts” to study how to hack critical bloggers and to generate false digital persona that can be used to insert pro-government propaganda onto a myriad of supposedly all foreign websites (many of which are still read by Americans, including others in the government). But it hasn’t the resources to improve its own security against a handful of routine intrusions. How funny. No wonder Rube Goldberg’s cartoons was so popular.

  4. behindthefall says:

    You’d almost think that DOD _wants_ to let someone look inside their system without their knowing about it.

  5. stryx says:

    At least the DOD isn’t alone:

    The inspector general cited a May 2009 incident in which cyber criminals infected a computer system that supports one of NASA’s mission networks.

    Due to the inadequate security configurations on the system, the infection caused the computer system to make over 3,000 unauthorized connections to domestic and international Internet Protocol (IP) addresses including addresses in China, the Netherlands, Saudi Arabia, and Estonia,” the report said.

    It said that in January 2009, cybercriminals stole 22 gigabytes of export-restricted data from a Jet Propulsion Laboratory computer system.

    And it’s far from the first time this has happened to the DOD. The WSJ reported in April 2009 that a cyber attack heisted 1.5 Tb of plans for the F-35.

    And that smart card that they are pinning all their hopes on? PWND:

    Security consultants from Mandiant have identified techniques that have been employed to breach security on government systems that require the use of smart cards and passwords for authentication.

    I want to make a joke here but instead I think I’m gonna go punch the wall.

      • earlofhuntingdon says:

        You beat me to the punch. Then there is the sometimes porous private sector, such as yesterday’s announcement by Epsilon, a unit of Allied Data Systems, that may be the biggest domestic breach to date. Companies whose customer data may have been breached included Citigroup, JPMorgan Chase, Capital One, Kroger’s, and Walgreen’s.

          • earlofhuntingdon says:

            If the US had EU-style data privacy rules, I suspect the annual cost to companies who fail to comply with them would be billions, and that’s after spending billions to get their systems, use and commercialization policies up to standard.

            Ayn Rand never entered government; her acolytes have. Their participation in and bending of government rules to service corporations make a mockery of her stand-alone “philosophy”.

            • qweryous says:

              Aw… schitt! No problemo until I saw that.

              As far as kludge or kluge and its origin(s)
              this pretty much agrees with what I know of the word. I think its pretty much alternate spellings meaning more or less the same thing.

  6. JTMinIA says:

    From what I know about the DoD’s HBSS (which is just a jazzed-up version of the package McAfee sells to anyone), the focus is still on two things: blocking “rogue” computers from connecting and stopping people from downloading to USB devices (e.g., flash-drives). There doesn’t seem to be much on stopping people from uploading malware from an already-connected and authorized machine.

    But wait for a more expert opinion than mine.

    ps. “cobbled together” is what you want in the next-to-last paragraph; I’ve never heard of “kluged together” and a “kluge” is quite different from merely using something for a purpose it wasn’t originally intended for; “kluge” is really just short-hand for “the second of two wrongs that makes a right,” as when you do something wrong on purpose to correct for some other, previous wrong

  7. Deep Harm says:

    At a national conference several years ago, I bumped into a systems security consultant who said he produced a detailed set of suggestions for fixing the holes in a federal agency’s system. The agency, he said, “acted like I had done something wrong,” and snubbed his recommendations. A few years later, the agency experienced a major data theft, and then another, and similar breaches were reported at other agencies. Investigation showed that the agencies had ignored security recommendations. Most of the data losses reported publicly involved personnel data protected by the Privacy Act. But, could some thefts involve money? It seems possible, for the agency I was discussing also was known for ‘losing’ money and had not balanced its books in years. An accountant friend says his agency has similar problems.

      • JohnLopresti says:

        Thanx for the patience. It took a while to return to the thread here. My spelling was only a humorous mention. A classic illustration from the DOS world was a NEC motherboard that managed to expand RAM to 5 MB, but the circuiting modifications by which it accomplished that *feat* in the 80xxx era rendered impossible any third party vendor attempt to piggyback yet more memory. Essentially, as the folks said up thread; a nonstandard, clumsy way to agglomerate components and traces for a short term goal; and a guaranteed pathway to quick obsolescence and undocumentable conflicts with standard gear which might be attached, if only the OEM had not shipped a quick kludge in the first place. I think my spelling universe was populated by articles from the notorious publications like Byte, through which one would have to leaf in many 50-page leaps to reach the end of its 400+ pages of shiny analysis. Having tried a job search a few times in an employment office at a military base, I thought the term kludge a delightfully blissful description of the experience. Still, I agree with friar Ockham*s friendly characterizations. It kind of adds to the appreciation due to DARPA longtime.

  8. Softail says:

    Umm, we don’t want the empire to be competent.

    Yep, kludge is a technical term ;-)

    Even with serious security, people are always going to be the weak link.

  9. JTMinIA says:

    The old definition of kluge (from German, it seems) is the third one in the second set of definitions on the page you (EW) linked to: something that works for the wrong reason. This term is used in a serious way in many areas (but maybe not so much in the computer world, it seems). Classic examples of a kluge are the various “fixes” for violations of statistical assumptions in data analysis. Rather than actually fix the original violation, you make another mistake on purpose to get the correct conclusion. The second, purposeful mistake is the kluge.

    Software that works for the wrong reason (or flat-out doesn’t work at all) is not called a kluge. It’s called Windows, instead.

    • PJEvans says:

      My understanding of kludging something together is the same as EW’s, and I’ve been playing with computers for a long, long time. (Never ran into your version before, actually.)

  10. JTMinIA says:

    I had never heard the term “kludge” (or “kludging together”) until today and have no idea what it’s used for other than what I read on the page cited above and what you all have told me. I was only talking about “kluge” (which was in EW’s post), which I’ve been using for many years and always in the technical way of “the second of two wrongs makes a right.”

    If this is a case of a typo (missing “d”) and not a misunderstanding of what a word means, then I’m rather embarrassed and should really just lurk.

  11. JTMinIA says:

    We are really ruining this thread, so I’ll drop it … after saying that computer folks often borrow terms and use them differently, but enough definitions pop up (when I search) that include the idea of “fixing” a problem by not actually fixing the problem that I’m satisfied that it’s the same core concept.

    What’s more important is that the patches and additions to the McAfee system used by the DoD are not like this. There isn’t a basic problem that is being “fixed” in a way that doesn’t actually fix the problem. They don’t seem to be trying to fix anything related to uploading from “trusted” machines.

  12. Citizen92 says:

    So SIPRet is just one big shared drive with all content available to all users?

    No wonder Yoo’s emails and memos went missing.

    • emptywheel says:

      Wouldn’t his emails have gone over JWICS?

      And it does sound like there are some access controls in SIPRNet–just none of them all that meaningful.

  13. tjallen says:

    Kludge – and Kluge

    You can look up all these funny technical terms at the Hackers Jargon File:
    Hackers Jargon File

    (The Hackers Jargon File is a hilarious read, btw. It’s been around since the earliest days of news groups.)

    • earlofhuntingdon says:

      Good resource. It notes that “kludge” is a common incorrect spelling of “kluge”: a Rube Goldberg device that, however ungainly, somehow works.

  14. WilliamOckham says:

    DoD’s network security is a huge challenge, by far the biggest network security challenge in the world. 100,000’s of points of entry, the most valuable secrets, the most determined adversaries, and the necessity of throwing up nodes in war zones are just the most obvious obstacles to network security for DoD. On the other side of the equation, they have an effectively unlimited money budget and more enforcement ability than any other organization in the world (if you are an outsider threat, they can shoot you and if you are an insider threat they can put you in the brig and torture you).

    The more we hear about their response to the challenges they face, the more the response seems to be the organizational equivalent of curling up in a ball and crying for your mommy. It’s really pathetic. In an effective organization, the equivalent of the WikiLeaks disaster or the 2008 incident would have led to a top to bottom overhaul of security procedures and new leadership at the top of the IT security unit. The DoD bureaucracy is too sclerotic to change quickly and the board of directors (i.e. Congress) is clueless (with a couple of notable exceptions, Rush Holt, I’m looking at you).

    I wouldn’t work for the DoD IT bozos (and I’m referring to the leadership, not the real tech folks who I’m sure are doing their best in a dysfunctional situation) for any amount of money in the world.

    • emptywheel says:

      Yeah, Congress is clueless.

      I didn’t emphasize this, but of course the SHSC is a Lieberman-Collins gig. In this particular hearing, Lieberman showed up, blathered about data sharing, and then left. Collins took over and, frankly, she had a few decent questions (such as when she asked why this happened). But she seemed to have no idea what she was being told: that DOD didn’t have the most basic forms of security, and they didn’t know how they were going to get it. (Note, Congress passed a bill requiring person specific access controls some years ago, which means DOD is out of compliance with that law, but no one was cracking heads about DOD failing to fulfill a legal mandate).

      Then Scott Brown came in and complained that he, as a National Guard JAG, couldn’t get this data, so why could a Private (actually, Specialist when he is alleged to have taken this)?

      Uh, maybe bc Manning had an intelligence function and you were defending servicemembers over DUI charges?

      Then Lieberman came in and whined about the WL leak (which of course made Lieberman look like even MORE of an ass than he already does).

      Which leaves the following Senators who apparently don’t give a damn that DOD has gaping security holes: SASC Chair and Ranking Members Levin and McCain, Akaka, Carper, Pryor, McCaskill, Tester, Begich, Coburn, Johnson-WI, Ensign, Portman, and Paul.

      • PJEvans says:

        I’d be willing to bet that none of those senators has the faintest idea what network security (and individual computer security) is or should be. They probably have aides handling all the computer stuff for them, because their knowledge is limited to AOL (at best).

  15. emptywheel says:

    Incidentally, just watched the end of a SJC Committee hearing on ECPA.

    Basically Senator Whitehouse, DOJ’s James Baker, and Commerce’s Cameron Kerry were all bitching about how Admin has been AWOL on cybersecurity.