If Amazon Web Services Goes Down, Do the Cloud Services AWS Provides the Intelligence Community Too?

As you may have heard, Amazon has had a bad outage today, taking down many entities that rely on its cloud service.

Most of the coverage has focused on the private businesses that have been affected, from small businesses to larger ones (I suspect Office Max was broadly affected, because they were down today too), to media outlets.

I want to know if, when Amazon’s Northern Virginia cloud services go down, whether the cloud services Amazon provides to the Intelligence Community goes down too. The IC cloud is supposed to be completely separate from AWS’ commercial services. But if things are going haywire generally in Northern Virginia, those problems may extend to Amazon’s (understood to be NoVA located) IC servers.

I raise that, in part, because of a point I made in these two posts about the new EO 12333 sharing rules Obama implemented in January. The data sharing envisioned can happen in one of three places: on NSA’s own servers, on the recipient agency’s own servers, or on the cloud.

NSA may choose to make raw SIGINT available (i) through NSA’s systems; (ii) through a shared IC or other Government capability, such as a cloud-based environment; or (iii) by transferring some or all of the information to the recipient IC element’s information systems. Only information that can be afforded appropriate handling, storage, retention, and access protections by the recipient IC element will be made available.

Indeed, rolling out the IC cloud was a necessary technical precondition for this sharing process.

As I subsequently pointed out, one application for this expanded sharing was to make counterintelligence information — of the kind that would be central to the investigation into Russia’s hack of the DNC and/or other influence peddling with Trump allies — more widely available (for example, to CIA and FBI).

In the procedures, the conditions on page 7 and 8 under which an American can be spied on under EO 12333 are partially redacted. But the language on page 11 (and in some other parallel regulations) make it clear one purpose under which such surveillance would be acceptable, as in this passage.

Communications solely between U.S. persons inadvertently retrieved during the selection of foreign communications will be destroyed upon recognition, except:

When the communication contains significant foreign intelligence or counterintelligence, the head of the recipient IC element may waive the destruction requirement and subsequently notify the DIRNSA and NSA’s OGC;

Under these procedures generally, communications between an American and a foreigner can be read. But communications between Americans must be destroyed except if there is significant foreign intelligence or counterintelligence focus. This EO 12333 sharing will be used not just to spy on foreigners, but also to identify counterintelligence threats (which would presumably include leaks but especially would focus on Americans serving as spies for foreign governments) within the US.

Understand: On January 3, 2017, amid heated discussions of the Russian hack of the DNC and public reporting that at least four of Trump’s close associates may have had inappropriate conversations with Russia, conversations that may be inaccessible under FISA’s probable cause standard, Loretta Lynch signed an order permitting the bulk sharing of data to (in part) find counterintelligence threats in the US.

This makes at least five years of information collected on Russian targets available, with few limits, to both the CIA and FBI. So long as the CIA or FBI were to tell DIRNSA or NSA’s OGC they were doing so, they could even keep conversations between Americans identified “incidentally” in this data.

Certain state adversaries would have big incentives to destabilize AWS, just for shits, giggles, and the chaos it would cause. If they could get into Amazon private clients’ servers, there would be plenty of data to make such an attack worthwhile.

But if such an attack also affected the IC cloud, that might be a different thing entirely.

Marcy has been blogging full time since 2007. She’s known for her live-blogging of the Scooter Libby trial, her discovery of the number of times Khalid Sheikh Mohammed was waterboarded, and generally for her weedy analysis of document dumps.

Marcy Wheeler is an independent journalist writing about national security and civil liberties. She writes as emptywheel at her eponymous blog, publishes at outlets including the Guardian, Salon, and the Progressive, and appears frequently on television and radio. She is the author of Anatomy of Deceit, a primer on the CIA leak investigation, and liveblogged the Scooter Libby trial.

Marcy has a PhD from the University of Michigan, where she researched the “feuilleton,” a short conversational newspaper form that has proven important in times of heightened censorship. Before and after her time in academics, Marcy provided documentation consulting for corporations in the auto, tech, and energy industries. She lives with her spouse and dog in Grand Rapids, MI.

10 replies
  1. klynn says:

    This, Don Jr. meeting in Paris – Russia contacts, FBI & MI6 contact…busy day just before the address tonight. Interesting signals.

    • klynn says:

      As for drinks for the Orange Jubilee…I say every “believe me” is a shot.  And have you read the international papers in the past week that have reported it was requested from high up in RU to stop praising Orange One in the RU media?

  2. jon says:

    Amazon’s IC Cloud is (supposed to be) hosted machines on that aren’t co-located with or connected to their commercial offerings.  So if it’s implemented correctly, an attack on AWS shouldn’t give a path to the IC cloud.  This is the kind of thing that government security people take pretty seriously … that said, boundaries can in practice be fuzzier than they’re supposed to be.

  3. lefty665 says:

    From the linked article “The most common causes of this type of outage are software related… ‘Either a bug in the code or human error’.”  Even if the IC cloud is appropriately isolated from AWS, it has its own software and liveware generated opportunities to crash, among other causes. We probably will not know when it happens, but it does, even at NSA itself as Bamford documented in “The Shadow Factory”. That is when people like Snowden really earn their pay.

  4. badtux says:

    Okay, so I work in the security industry, know people involved in various decisions, and have information on what’s necessary to run our product inside GovCloud. So here’s the scoop:

    1) No, GovCloud shares no — zero — infrastructure with the public AWS infrastructure. This was an absolute requirement imposed by the NSA. It’s in its own data centers and has its own fibers connecting the data centers. All services accessed from GovCloud virtual servers and software running within GovCloud virtual servers must be GovCloud services — no services provided by any other regions or zones.

    2) Thus any outage on us-east-1 is unlikely to have affected GovCloud. In particular, if no other AWS regions went offline, GovCloud didn’t go offline. The outage likely is related to an S3 upgrade that Amazon deployed in us-east-1 last week (I noticed the S3 UI suddenly change at that time), Amazon never deploys such upgrades to other regions until they’ve been in production in us-east-1 for weeks. My guess is that GovCloud is last on the list of regions to upgrade, they’ll upgrade all the other regions before they do GovCloud for obvious reasons.

    3) GovCloud generally contains information that is either public or may be sensitive, but it is not authorized for classified secret information. The intelligence agencies and DoD have their own networks for that kind of information, networks that are so secure that they’re air-gapped. In short, GovCloud might hold the NSA’s public web pages, but will not hold transcripts of intercepted telephone conversations between the Prime Minister of Germany and the Prime Minister of England — that is on a classified network in one of the NSA’s own data centers.

    So chances of GovCloud having been taken out by this outage? So close to zero as to be indistinguishable from zero. Chances of any NSA / CIA / DIA / etc. services being affected? Zero. They just aren’t connected to any network that Amazon controls.

    In short, a foreign attacker could have attacked us-east-1 in an effort to harm the U.S. economy. But far more likely is that the S3 upgrade they did last week had a bug that ended up causing it to collapse.

    Finally, about the effects on cloud products: All of Amazon’s API’s quit operating properly, affecting the ability to launch new servers or call any other Amazon-specific API’s such as their DynamoDB apis. However, already-running cloud servers continued operating through the S3 outage, they simply weren’t capable of spinning up new servers and any images stored in S3 weren’t displayed by their UI. Some people were affected worse than others. I wasn’t affected much, a friend’s startup lost his entire service because he uses an Amazon API to run snippets of code on demand. So results were mixed. It was definitely not armageddon though. Amazon may have managed to DoS its own product, but they didn’t take down the Internet.

  5. lefty665 says:

    Here’s their story and they’re sticking to it. It was the liveware. An employee doing maintenance took too many servers down at one time and it cascaded from there. It could happen on the Govt side just as easily, and probably has. We on the outside just don’t hear about it.

    https://www.washingtonpost.com/news/the-switch/wp/2017/03/02/the-embarrassing-reason-behind-amazons-huge-cloud-computing-outage-this-week/?hpid=hp_regional-hp-cards_rhp-card-technology%3Ahomepage%2Fcard&utm_term=.2d18c52c3100

Comments are closed.