mars-mro-orbiter-gale-crater-hirise-esp_043961_1740-pia20167-full

Why Is DOD Paying Dataminr $13M for Data It Claims to Believe Twitter Won’t Deliver?

Last week I did a post on John McCain’s promise, given in a Senate Armed Services Committee hearing, to “expose” Twitter for refusing to share you Tweets in bulk with intelligence agencies. Later in the hearing, Jeanne Shaheen returned to the issue of Twitter’s refusal to let Dataminr share data in bulk with the Intelligence Community. She asked Under Secretary for Intelligence Marcell Lettre what the committee needs to get more cooperation. Lettre responded by suggesting one-on-one conversations between Executive Branch officials and the private sector tends to work. Shaheen interrupted to ask whether such an approach had worked with Twitter. Lettre responded by stating, “the the best of my knowledge, Twitter’s position hasn’t changed on its level of cooperation with the US intelligence community.”

That’s interesting, because on August 26, 2016, DOD announced its intent to sole-source a $13.1 million one-year contract with Dataminr to provide alerting capability based off Twitter’s Firehose.

The Department of Defense (DoD), Washington Headquarters Services, Acquisition Directorate (WHS/AD), on behalf of the Office of the Under Secretary of Defense for Intelligence (OUSDI) intends to award a sole source contract pursuant to the requirements of 41 U.S.C. 3304(a)(1) Competition in Contracting Act of 1984 (CICA) as implemented by FAR Subpart 6.3, and IAW the requirements of FAR Subpart 6.303-1, Only One Responsible Source and No Other Supplies or Services will Satisfy Agency Requirements.

WHS/AD intends to issue this sole source contract to Dataminr, Inc located at 99 Madison Ave Floor 3, New York, NY 10016 (CAGE 6Q6Z6). The anticipated Period of Performance for 1500 license subscriptions are 12 months from the date of contract award. The estimated value of this procurement is approximately $13.1M.

This contract will address the requirements of OUSDI Technical Collection and Special Programs division. The award will be made for licenses, support, and maintenance which allows DoD to receive indication and warnings, situational awareness, and contextual analysis of social media data in order to provide actionable decision support in response to real-time information.

Salient Characteristics of the Data Analytics Software: The contractor shall deliver an alerting capability that, at a minimum, includes:

  • Alerting: Based on the algorithmic analysis of the complete Twitter Firehose, the Contractor shall deliver near-real time alerts on breaking developments relevant to military security.
  • Content: The Contractor’s platform shall generate data from the Twitter firehose. Alerts shall include from the original data source at least the text, embedded links, and associated metadata, to include the Tweet ID.

Perhaps the Under Secretary of Defense for Intelligence’s Technical Collection and Special Programs division doesn’t count as “intelligence community,” but it sure seems to qualify.

Or perhaps there are a number of loopholes in the policy that purports to keep Twitter customers’ data out of the hands of intelligence agencies.

Marcy has been blogging full time since 2007. She’s known for her live-blogging of the Scooter Libby trial, her discovery of the number of times Khalid Sheikh Mohammed was waterboarded, and generally for her weedy analysis of document dumps.

Marcy Wheeler is an independent journalist writing about national security and civil liberties. She writes as emptywheel at her eponymous blog, publishes at outlets including the Guardian, Salon, and the Progressive, and appears frequently on television and radio. She is the author of Anatomy of Deceit, a primer on the CIA leak investigation, and liveblogged the Scooter Libby trial.

Marcy has a PhD from the University of Michigan, where she researched the “feuilleton,” a short conversational newspaper form that has proven important in times of heightened censorship. Before and after her time in academics, Marcy provided documentation consulting for corporations in the auto, tech, and energy industries. She lives with her spouse and dog in Grand Rapids, MI.

4 replies
  1. Peterr says:

    I would be willing to not provide the DOD with all the information in Twitter’s Firehose for 12 months at the cost of a mere $10M.

     

  2. rugger9 says:

    Was this a cost-plus type of contract? If it was and past DOD oversight form holds, 13 M$ isn’t surprising in the least for a DOD CF.

  3. Khepry Quixote says:

    Perchance the NSA et al is already collecting Twitter’s posts surreptitiously via the spritzer, garden hose, or fire hose APIs and needs a preconfigured search engine to absorb and search the collected data.

Comments are closed.