Yesterday evening, I took the ill-advised step of jumping into an already ongoing Twitter discussion with Professor Christine Fair on the topic of drones in Pakistan. My jumping in was ill-advised on two fronts: I had not seen the comments to which Fair was responding, but, more importantly, I can’t come close to the experience, language skills and overall knowledge Fair brings to the issues of South Asia.

My first entry into the discussion was to respond to a statement from Fair in which she said that she supports drones and does not believe their use to be collective punishment. I asked whether the use of drones to attack first responders and mourners in Pakistan qualified as collective punishment and in a follow-up provided a link to the work by Chris Woods and Christina Lamb at The Bureau of Investigative Journalism where they document such attacks. Fair’s response was to point out that Woods and Lamb have not been to FATA and that the Pakistani press is heavily manipulated. She referred me to a piece she wrote for Monkey Cage for elaboration on the points she was making.

It appears that this is the post Fair was asking that I read. Before diving into it, I should point out that it is about a year old and was written primarily in response to earlier work by Woods and Lamb. For fairness, I should also point out that from the context of other tweets later in the evening, Fair was a passenger in a car during our conversation and so would have been working with fewer resources at hand than if at home and using a computer.

With that as prologue, here is Fair’s dissection of the reliance on press reports for analysis of drone attacks in Pakistan’s tribal areas (BIJ is The Bureau of Investigative Journalism and NAF is the New America Foundation, where Peter Bergen and others have produced another drone strike database):

Their methodologies and data are fundamental weaknesses, although neither seem aware of this. Both NAF and BIJ claim that they have assembled a database which covers each individual strike in Pakistan in detail.  Unfortunately, both efforts fundamentally rely upon Pakistani press reports of drone attacks. Both claim that they use non-Pakistani media reports as well. For example the BIJ explains in their methodology discussion that the “…the most comprehensive information on casualties lies in the thousands of press reports of drone strikes filed by reputable national and international media since 2004. Most reports are filed within a day or two of an attack. Sometimes relevant reports can be filed weeks – even years – after the initial strike. We identify our sources at all times, and provide a direct link to the material where possible.”

/snip/

While these methodologies at first blush appear robust, they don’t account for a simple fact that non-Pakistani reports are all drawing from the same sources: Pakistani media accunts [sic]. How can they not when journalists, especially foreign journalists, cannot enter Pakistan’s tribal areas?  Unfortunately, Pakistani media reports are not likely to be accurate in any measure and subject to manipulation and outright planting of accounts by the ISI (Pakistan’s intelligence agency) and the Pakistani Taliban and affiliated militant outfits.

The more recent report from Woods and Lamb (in which the first responder accusations are made), however, appears to have taken steps to address at least a portion of the shortcomings Fair has pointed out. Since it is not safe for foreign journalists to enter the tribal areas, Woods and Lamb engaged a group of local researchers to carry out interviews on their behalf: Continue reading →