The assessments of the phone dragnet suck.
I don’t mean the assessments of the phone dragnet show the program sucks, though that may well be the case. I mean the assessments of the phone dragnet I’ve seen do a very poor job of assessing the value of it. Which serves to show how much of the larger dragnet remains, if not secret, still largely undiscussed.
To see what I mean, consider this post, from Just Security’s Ryan Goodman.
Insiders disagree about the phone dragnet value with outsiders
The strongest part of his post compares the seemingly contradictory assessments of the phone dragnet by two different members of the NSA Review Group. University of Chicago Professor Geoffrey Stone and Deputy Director of CIA Mike Morell.
Stone, based on what he learned from public sources and from the briefings the Group received, believes the program did not prevent any terrorist attacks. Morell, whose former agency receives Tippers from the program and even had direct access to query results until 2009 just like the FBI does and did (though no one talks about that) insists it has helped prevent terrorist attacks.
Goodman also notes that the Gang of Four immediately defended the phone dragnet after the Review Group released its results (actually, they object to more than the phone dragnet recommendation but don’t say what other recommendations they object to), but doesn’t note the terms they use to do so:
However, a number of recommendations in the report should not be adopted by Congress, starting with those based on the misleading conclusion that the NSA’s metadata program is ‘not essential to preventing attacks.’ Intelligence programs do not operate in isolation and terrorist attacks are not disrupted by the work of any one person or program. The NSA’s metadata program is a valuable analytical tool that assists intelligence personnel in their efforts to efficiently ‘connect the dots’ on emerging or current terrorist threats directed against Americans in the United States. The necessity of this program cannot be measured merely by the number of terrorist attacks disrupted, but must also take into account the extent to which it contributes to the overall efforts of intelligence professionals to quickly respond to, and prevent, rapidly emerging terrorist threats. [my emphasis]
In other words, Goodman presents evidence that the Gang of Four and a former top CIA official believe there are other reasons the phone dragnet is valuable, while someone relying on limited briefings evaluates the program based on its failure to stop any attack.
That ought to make Goodman ask what Morell and Dianne Feinstein know (or think they know) that Stone does not. It ought to make him engage seriously with their claim that the phone dragnet is doing something else beyond providing the single clues to prevent terrorist attacks.
One they’re not willing to talk about explicitly.
Assessments and the terrorist attack thwarted metric
Instead, Goodman assesses the phone dragnet solely on the basis of the public excuse offered over and over and over since the Guardian first published the Verizon order in June: to see which Americans are in contact with (alleged) terrorist associates so as to prevent an attack.
Goodman lectures program critics that identifying funders or members of terrorist groups might help find terrorists, too, and “peace of mind” might help dedicate resources most productively.
The key objective of course is to stop terrorist attacks against the US homeland and vital US interests abroad. An important distinction, however, is whether the intelligence generated by the program is:
(a) “direct”: timely information to foil a specific attack; or
(b) “indirect”: information that enables the government to degrade a terrorist group or decrease the general likelihood of attacks
Examples of the latter might include information on individuals who have joined or are funding a terrorist organization. Intelligence could help to identify and successfully prosecute such individuals, and hence disable them and deter others. The important point is that both types of information aid the overall goal of stopping terrorist attacks. That point appears to have been lost on some critics of the program. When the government cites the latter information yields, critics often consider such situations irrelevant or little to do with stopping attacks.
But Goodman imagines only those affirmatively supporting terrorism would help the government prevent terrorism, which is not necessarily the case.
Does the NSA’s network analysis even pick the right calls?
One thing missing from such assessments are the failures. Why didn’t, for example, Faisal Shahzad’s planning with the Pakistani Taliban identify him and his hawala before the attack? There are plausible explanations: he used good enough operational security such that he had no communications that could have included in the dragnets, his TTP phone and Internet contacts were not among the services sucked up, the turmoil in the phone and (especially) Internet dragnet in 2009 and 2010 led to gaps in the collection. Then there’s a far more serious one: that the methods NSA use to identify numbers of interest may not work, and may instead only be identifying those whose doings with terror affiliates are relatively innocent, meaning they don’t use operational security (though note the US-based phone dragnets would use more sophisticated analysis only after data gets put in the corporate store, whereas data collected overseas might be immediately subject to it).
And for those who, like Goodman, place great stock in the dragnet’s “peace of mind” metric, they need to assess not just the privacy invasion that might result, but the resources required to investigate all possible leads — which could have been upwards of 36,000 people in the Boston Marathon case.
That is, unless we have evidence that NSA’s means of picking the interesting phone contacts from the uninteresting ones works (and given the numbers involved, we probably don’t have that), then the dragnet may be as much a time suck as it is a key tool.
What about the other purposes the Intelligence Community has (quietly) admitted?
The other problem with assessments of the phone dragnet is they don’t even take the IC at its word in its other, quieter admissions of how it uses the dragnet (notably, in none of Stone’s five posts on the dragnet does he mention any of these — one, two, three, four, five — raising questions whether he ever learned or considered them). These uses include:
- Corporate store
- “Data integrity” analysis
Corporate store: As the minimization procedures and a few FISC documents make clear, once the NSA has run a query, the results of that query are placed in a “corporate store,” a database of all previous query results. Continue reading