NSA, Destroying the Evidence
In my obsessions with the poor oversight over the phone dragnet techs, I have pointed to this description several times.
As of 16 February 2012, NSA determined that approximately 3,032 files containing call detail records potentially collected pursuant to prior BR Orders were retained on a server and been collected more than five years ago in violation of the 5-year retention period established for BR collection. Specifically, these files were retained on a server used by technical personnel working with the Business Records metadata to maintain documentation of provider feed data formats and performed background analysis to document why certain contact chaining rules were created. In addition to the BR work, this server also contains information related to the STELLARWIND program and files which do not appear to be related to either of these programs. NSA bases its determination that these files may be in violation of BR 11-191 because of the type of information contained in the files (i.e., call detail records), the access to the server by technical personnel who worked with the BR metadata, and the listed “creation date” for the files. It is possible that these files contain STELLARWIND data, despite the creation date. The STELLARWIND data could have been copied to this server, and that process could have changed the creation date to a timeframe that appears to indicate that they may contain BR metadata.
The NSA just finds raw data mingling with data from the President’s illegal program. And that’s all the explanation we get for why!
Well, PCLOB provides more explanation for why we don’t know what happened with that data.
In one incident, NSA technical personnel discovered a technical server with nearly 3,000 files containing call detail records that were more than five years old, but that had not been destroyed in accordance with the applicable retention rules. These files were among those used in connection with a migration of call detail records to a new system. Because a single file may contain more than one call detail record, and because the files were promptly destroyed by agency technical personnel, the NSA could not provide an estimate regarding the volume of calling records that were retained beyond the five-year limit. The technical server in question was not available to intelligence analysts.
This is actually PCLOB being more solicitous in other parts of the report. After all, it’s not just that there was a 5 year data retention limit on this data, there was also a mandate that techs destroy data once they’re done fiddling with it. So this is a double violation.
And yet NSA’s response to finding raw data sitting around places is to destroy it, making it all the more difficult to understand what went on with it?
I will chime in with limited sympathy for the NSA in this case. I’ve been there, in the sense of stumbling across a server that time forgot and finding things on there which should have been destroyed long ago. It happens. You’d hope that in a high security environment with all the money they want, it wouldn’t happen, but it’s also pretty clear that the NSA technically environment was a rapidly changing one and one that rewarded operational expediency. You can lose track of a server or chunks of data. And, btw, if it didn’t have problems keeping track of things, Edward Snowden wouldn’t have had the opportunity to obtain and leak documents.
@Saul Tannenbaum: Yeah, It may well be nothing to worry about.
But when you’re fiddling with the raw data of AMerican phone based relationships in an already insufficiently overseen area, I’d really prefer if you didn’t break two rules while doing so.
It’s true, boys will be boys and all that. I mean, it’s not like Scott Bloch calling in professionals to scrub hard drives at a government office after getting a subpoena — that would be criminal, criminal, criminal and if I recall, he even pleaded guilty and was thereupon sentenced to a whole day in jail after the DOJ tried to get him out of any jail time at all. So we got that going for us, America.
But still, what part of a message of “transparency” does this NSA conduct fit into? “Accountability”, perhaps? No? Well, then, certainly they’ll have to concede that the stuff they destroyed bore some “relevancy” to some kind of legal obligation, right? After all, they’re the ones who rely on “relevancy” as being all-encompassing, right?
@emptywheel: It’s possible it was real data being inappropriately used as test data. The techs bring up a new server, want to test a new bit of code to do data processing and upload some real data to test the code against. When testing is complete they then forget to delete the data. Now, generally when you want to test a program against “real” data you pass the data through a scrubber. All names become Jane Doe, #s are 555 etc. But that’s an easy thing to put off implementing.
That or they wanted to save data for a while longer.
I wonder if this resided on a remote corporate server. Eagle Alliance was providing a lot of the infrastructure by this time.
Not seeing anything here about destroying backups. Everything is backed up — drives fail all the time. RAID back then but servers fail too. Might have been around on tape backup. Might be multiple backups … hard to be sure you’ve found and erased them all.
I’m skeptical that the files are all in one server or in one physical location. They evacuated Ft. Meade as 9/11 was ongoing — not going to put all the eggs in one basket afterwards, not that they ever did.
Not seeing mention of where these sample files came from — were they just copied over from a much larger set, leaving the originals or just the ones not in the set that weren’t copied or moved over?
Did it take some ETL (http://www.webopedia.com/TERM/E/ETL.html) to get them over to where StellarWind data resided? It would be very unusual for data from different sources to sit in identical columns (fields). Creation date … ouch, sounds like that was really mishandled if instead it were only date of inter-server transfer.
This could be an understatement: “a single file may contain more than one call detail record”. Maybew a single file contain thousands of CDRs. They were running relational mysql databases at this time so a “file” might be all the CDR off a given index field value (eg, all the calls originating from a given fixed phone number over a certain date range, say five years). If so, 3,032 files might contain tens of millions of records (individual phone calls).
Rulebreaking aside, I’d guess the missing files were mere representatives, so there’s no special interest in retrieving them, should all copies not have been deleted.
Shame that Snowden moved on before he could straighten out their storage. That was his job, moving huge sets of files around for better compartmentalization.
In case anyone missed them, there are also reports from the 2 dissenting PCLOB members as well (both served as Assistant Attorney General for legal policy at the DOJ during the Bush administration):
PCLOB Cook Statement – http://www.pclob.gov/All%20Documents/Report%20on%20the%20Telephone%20Records%20Program/PCLOB%20Cook%20Statement.pdf
PCLOB Brand Statement – http://www.pclob.gov/All%20Documents/Report%20on%20the%20Telephone%20Records%20Program/PCLOB%20Brand%20Statement.pdf
Where does the NSA begin and its contractors end? Whose servers were this data on, and were all copies of it destroyed, governmental and privately held, destroyed?
@Snoopdido: Here’s one of Elisebeth Collins Cook’s final dissents:
“In many ways, the evaluation of this long-running program was the most difficult first test this Board could have faced. Unfortunately, rather than focusing on whether the program strikes the appropriate balance between the necessity for the program and its potential impacts on privacy and civil liberties, and moving immediately to recommend corrections to any imbalance, the Board has taken an extended period of time to analyze (a) statutory questions that are currently being litigated, and (b) somewhat academic questions of how the Fourth Amendment might be applied in the future and the First Amendment implications of programs that do not presently exist.”
Let’s parse exactly what Ms. Cook is saying here.
She is basically saying that after over 12 years of running the Potemkin Dragnet (see Potemkin Village here – http://en.wikipedia.org/wiki/Potemkin_village) in which not a single terrorist has been found to be hiding in all of the 300 million plus American phone records held in her dragnet, Ms. Cook would like us to ignore the massive violation of the Constitution’s 4th Amendment and instead keep on fishing in hopes that someday a terrorist will jump into her boat.
She’s got a hell of sense of humor, doesn’t she?
@Snoopdido: Just a note that I didn’t realize that the 2 dissenters’ views were already incorporated into the body of the PCLOB report as Annex A and B until I re-read the table of contents. No harm, no foul?
@Snoopdido: I suspect some of the arguments PCLOB made, precisely on necessity, challenge the justifications for programs that are far more cherished by TPTB. That includes Section 702.
So in addition to 1) objecting bc her DOJ approved these wackyass theories and 2) realizing that PCLOB just provided a map for other lawyers to challenge this, 3) she’s probably worried about implications of this legal argument for other spying programs.