The White House Response on Backup Tapes

Hey, what do you know? The White House still sufficiently recognizes the third branch of government to respond to a judge’s request regarding all its lost emails. And as I suspected, the answer to whether or not the back-up tapes for White House emails include the emails not properly archived between March 2003 and October 2005 is, partly, "no." As the CIO of the Office of Administration, Theresa Payton, explains, the White House recycled its backup tapes up until October 2003, so it would not have any missing emails from March 2003 (the beginning of the period when the emails started going missing) and October 2003 (the period when the OA stopped recycling its backup tapes).

Prior to October 2003 and continuing through 2005 and to the present, this office has regularly created back-up tapes for the EOP Network, which includes the system’s email servers. Consistent with industry best practices relating to tape media management for disaster recovery back-up systems, these tapes were recycled prior to October 2003. In October 2003, this office began preserving and storing all back-up tapes and continues to do so.

But watch how Payton pretends that this doesn’t mean the White House might be missing a chunk of emails.

For that reason [the post-October 2003 preservation of backup tapes], emails sent or received in the 2003-2005 time period should be contained on existing back-up tapes.

[snip]

…in view of this office’s practice in the 2003-2005 time period of regularly creating back-up tapes for the EOP Network, which includes servers containing emails, and in view of this office’s practice of preserving all such back-up tapes from October 2003 to the present, the back-up tapes should contain substantially all the emails sent or received in the 2003-2005 time period.

As everyone who has read this can understand clearly, Payton’s statement doesn’t mean what it says. Rather, it is an admission that the White House may well be missing emails written or received between March 2003 and October 2003.

Such a misleading response is only one of the ways in which this response is disingenuous.

Payton explains in her statement that she has been CIO since May 2006–more than six months after the disappearing emails stopped disappearing, and two months after Fitzgerald received the missing OVP emails. Since Payton wasn’t present for this documented example of email recovery, she can speak in hypotheticals about the whole process, even while she appears to admit that the tapes have been used to restore data in the past.

The back-up tapes have been used to restore information that is not otherwise available on the EOP Network. When applied to email recovery, the process is complex, labor intensive and costly. By way of hypothetical example, a request to recover a specific file(s) from a particular date or date range within the period of 2003-2005 would be forwarded to the OCIO [that is, to Payton]. The OCIO would then ordinarily consult a media database, similar to a catalogue or back-up tape index, to identify a range of tapes that correspond to that request. In this process, the OCIO may pull tape sets backed up prior to and/or subsequent to the target date period to ensure they have the full population of potential tape sets that may contain the requested file. Additionally, the OCIO would then restore the data-type environment, applicable software, and/or information, and then conduct a search for the information requested. [my emphasis]

Mind you, Payton isn’t admitting this has been done (I’m not sure it has, but it seems likely it was in the Plame case), she’s just speaking hypothetically.

But the real disingenuous stance she takes has to do with her treatment of CREW’s claim that a bunch of email has disappeared. Payton writes,

I am aware of a chart created by a former employee within the OCIO that purports to identify certain dates and EOP components for which the chart’s creator appears to have concluded that certain EOP components were missing emails on certain dates in the 2003-2005 time period. Specifically, the chart appears to have concluded that some components on some dates had either (i) a lower-than-expected number of emails preserved in the normal electronic archiving process, or (ii) no emails preserved in the normal electronic archiving process. I believe this is what Plaintiffs refer to as the "detailed analysis."

The OCIO has reviewed the chart and has so far been unable to replicate its results or to affirm the correctness of the assumptions underlying it. Accordingly, this office has serious reservations about the reliability of the chart.

But here’s how CREW describes this "chart."

… when the problem was uncovered the White House Office of Administration created abundant documentation that included multiple estimates of the volume of missing email, not a single chart that the White House now suggests is the only documentation. Could it be that having now destroyed the evidence documenting the missing email problem, the White House feels free to retreat from its acknowledgment to Mr. Fitzgerald that White House emails are missing? [my emphasis]

Lucky for us (ha!), Payton assures us she will shortly complete her own review.

…this office has undertaken an independent effort to determine whether there may be anomalies in Exchange email counts for any particular days resulting from the potential failure to properly archive emails for the 2003-2005 time period. That process is underway and we expect the independent assessment to be completed in the near term.

At the very least, the judge should ask for this review as soon as it is completed (heck, a deadline would be nice–after all, the White House responded the last time it got a deadline from this judge), as well as the name of the former employee who put together the "chart" in the first place.

I’d also love to have CREW ask whether Payton’s assertions about the Executive Office of the President (EOP)’s emails hold true, as well, for OVP’s emails, because Payton engages in some squirminess about Dick’s emails. In her general description of the OCIO’s duties, Payton explains that it serves both EOP and OVP.

The OCIO, which is an operating unit of OA, provides around-the-clock customer service for all EOP components and the Office of the Vice President, consisting of more than 3000 users and customers, in excess of 200 servers, and over 100 applications.

This seems to suggest that she considers (as the White House has, at times, to protect Dick) OVP a separate entity from EOP. That’s curious because the rest of Payton’s assertions about the treatment of email backups refer to the "EOP network." Has OCIO been backing up Dick’s emails in the same way it does Bush’s?

For all its disingenuousness, Payton’s statement is useful for one reason. It pinpoints the date when, if these emails and backups were deliberately deleted, that deletion was done: no later than October 2003. Which, of course, happens to be when DOJ started investigating the Plame leak.

image_print
166 replies
  1. BlueStateRedHead says:

    Shit. word I don’t usually type here or anywhere. Will it hit the fan? Can the progblogosphere do anything to support CREW? to the AG called on this when next before the SJC?

  2. FrankProbst says:

    EPUed, so reposting:

    The original e-mails AND the backups were erased. And yet Luskin says of Rove’s e-mails, “There’s never been any suggestion that Fitzgerald had anything less than a complete record.”

    Let me make it official, then: I suggest that the erasure of both the original e-mails AND the backups could mean that Fitzgerald has something less than a complete record. I further suggest that you would have to have some sort of neurological problem if you thought otherwise.

    I also suggest that erasing information that you have a legal obligation to preserve is not “consistent with industry best practices”, unless you are in the business of habitually breaking the law and then getting caught red-handed doing so.

    • emptywheel says:

      Luskin said that after Fitz did a scan of Rove’s machine, presumably using checking the hard drive to check the emails thta had been sent. I don’t know if that guarantees that Fitz had a complete set, but remember that Fitz looked for data in several more places.

      I’m waiting for the day when Fitz finds Libby’s emails to Judy. She claims he never used email with her. But we know (from my own emails with her, and the emails of the person she outed) that SHE uses emails pretty religiously. There are emails to Judy, somewhere, I almost guarantee it.

  3. BlueStateRedHead says:

    There a front page story on Dkos. Outrage busting out all over with people there are asking what the law is here. Talkleft’s JM is in courts. Can you help us out here, EW, by way of an update.

    BTW, are any MSM reporting this?

      • BlueStateRedHead says:

        Just found it. http://ap.google.com/article/A…..AD8U73I302

        Wasn’t this a court-ordered deadline?

        Timing relative to what or should we say which? Like the Pres. being out of town? IIRC he made the Scooter announcement in DC after leaving the White House Press stenographers Corps back in Maine. Or the AG’s coming testimony.

        CAN SOMEBODY DO SOMETHING? SAY SOMETHING? (I am yelling).Can Waxman blanket warn all administrators of backup dates in all exec. depts that they will be halled before congress in batches (like the emails) and questioned? I am literally too riled up to work.

  4. bobschacht says:

    I am in favor of legislation banning the use of passive grammatical constructions in these communications. Responses like “Mistakes were made” are not adequate!

    Wishfully thinking,
    Bob in HI

  5. merkwurdiglieber says:

    The passive voice enables long, rambling, seemingly explanatory versions
    of shit happens, so sue me.

    • BayStateLibrul says:

      Theresa Payton ought to enroll in a basic writing course, with emphasis on clear declarative sentences.
      I think Theresa Payton is a former exec with Bank of America (not sure)
      In any case, I don’t know what she is fucking talking about…

      • merkwurdiglieber says:

        That is the exact purpose of the syntax she has been trained in, leaves
        you with the WTF look in the mirror.

  6. looseheadprop says:

    The law is : A doctrine called “spoliation” which says that if you destroy evidence that you had an obligation to preserve, if you knew or reasonably should have known that the evidence in question related to a possible claim against you, then your opponent is entitle to a “negative inference” against you.

    More simply, with less jargon. The court will instruct the jury that your opponenet is entitled to an inference that the destroyed evidence would either support their case or damage the theory of your case.

    This was the problem when Arthur ANderson had it’s shredding party after the Enron collapse. It didn’t matter whether or not they had a regular practice of shredding stuff once it got old. Once htey Knew or should have known that any of that paper might have been relevant to the Enron investigation, they had an obligation to preserve it.

    Further, WH has additional obligation to preserve under the Presidential Records Act.

    • BlueStateRedHead says:

      Under normal circumstances, what does a judge do in this case? And in terms of violation of the pres. records act, who refers the complaint to whom, who has standing?
      And who is the judge,is a Bush Co appointee?

  7. JimWhite says:

    Consistent with industry best practices relating to tape media management for disaster recovery back-up systems, these tapes were recycled prior to October 2003.

    I know it’s not news to anybody here, but someone in MSM needs to point out that the 1978 Presidential Records Act requires record retention. The White House was completely aware of that law and chose to disregard it. This, alone, is sufficient ground for criminal prosecution of all those participating in it and impeachment if any evidence of participation by Cheney or Bush is demonstrated.

  8. merkwurdiglieber says:

    There will turn out to be other copies of the tapes, kept for their
    own purposes, on some other servers, maybe NSA. They need them as badly
    as anyone else, liars have to keep track of their lies to stay in the
    game.

  9. AZ Matt says:

    DOJ starts the Plame investigation and Libby, Cheney, Judy and others all pucker up their butts and swallow the e-mails whole. Just dissect the bunch of them.

  10. MadDog says:

    Just slipping back from the link to report that Theresa Payton, CIO of the Office of Administration did not personally write that declaration. Twas written by Helen H. Hong, who is…wait for it…

    Helen H. Hong
    Trial Attorney
    U.S. Department of Justice, Civil Division

    Who just happens to be legal counsel for the defendants residing at, you guessed it, the White House.

    The same Helen H. Hong mentioned in this filing and this filing. These are both related to these same legal proceedings about the WH non-Backup tapes.

      • MadDog says:

        I suspect that good lawyers never, ever allow techies to write stuff by themselves. We would definitely ruin their day if allowed to pen our own responses. *g*

        • jdmckay says:

          I suspect that good lawyers never, ever allow techies to write stuff by themselves

          There’s a joke heard frequently amongst programmers re: their qualifications for corp. accounting systems, goes something like…

          “Must be skilled in obfuscation techniques required to satisify upper management.”

  11. BlueStateRedHead says:

    A suggestion to chew on from mspicarta at dkos (don’t know how to link a comment, so I will quote and h/t the author

    that there are two server systems in the White House –one secure, one not — I’m not buying this revelation. Why? Because there were obviously email communications both within and emanating from the White House that required encryption. That means that they would have had to go through the Central Security Service. That further means a second opportunity to back them up.

    An excellent start in investigating this would be to have General Randal D Fullhart testify about the White House communications systems. It’s his job to secure them, not some low-grade White House clerk.

    • MadDog says:

      I’ve made this point before, but yes, the WH must have 2 different server environments. The secure server environment is probably used for National Security stuff and staff like stuff Stephen Hadley and crew do. They probably also have a secondary, secure email account for such purposes.

      I would hazard a guess that backups from the secure server environment do not generally duplicate information that is contained on the other WH server environment.

      There may be some crossover as people and topics intertwine, but I wouldn’t bet much that the “missing WH emails” regarding topics like the firing of the US Attorneys is contained on the secure server backups.

    • watercarrier4diogenes says:

      FWIW, the date/time stamp in the lower right of a dKos comment is a permanent link to that comment. You can pull up individual comments at dKos dating all the way back to their adoption of Scoop in 2003 (mostly completed by roughly Aug. 2003). You can also search for comments made by individual Users, by their UserName or UserID in the ‘Search’ link on the right side of the main page.

  12. AZ Matt says:

    OT but we don’t need no environmental laws! Damn seeky whales!

    Bush exempts Navy from environmental law

    Bush Exempts Navy From Environmental Law to Continue Military Exercises Off California Coast

    PAULINE JELINEK
    AP News

    Jan 16, 2008 09:32 EST

    President Bush exempted the Navy from an environmental law so it can continue using sonar in its anti-submarine warfare training off the California coast — a practice critics say is harmful to whales and other marine mammals.

    The Raw Story

  13. JohnLopresti says:

    The technical people hereabouts were discussing Exchange2000; MS usually actually is a few years behind the product date, though, so it would be interesting to hear more about the taxonomy of the MSExchange rollout in EOP, whether it began in select server banks in 2002, e.g. Also, post9-11 best practices, especially for corporate parent offices and key government were for geographic dispersal of archive, meaning some of Paxton’s parsed language could be obscuring offsite archives; the recommendation was at a minimum several hundred miles from the backedup site for mirror archive. I also note Paxton’s bailiwick is nonClassified, which indicates there could be a simple extra-OCIO net way to keep comms confidential by stamping all one’s emails TreatAsIf, and that other classified net would observe offsite redundancy archival. I hear only normal IT jargon about the likelihood OCIO has tapes of everything 24+7 in the October2003ff period, however; and doubt what Paxton was saying would have been someone could have emailed but erased all trace of it by completing the write-erase cycle between backups. Maybe writing an intranet bot to watch for such chicanery would be covered under the rubric of what Paxton describes as responsibility for writing inhouse apps. Reading the new remarks, especially MDog’s Hong advisory, the legalese in the ITese becomes more definite, but the meaning seems straightforward, outside the parsing to avoid divulging much topologic information or actual practices or utils the OCIO writes and maintains. Most of the lexis is standard. Some IT folks might smile to know the acronym ocio in one language means spare time pursuits.

  14. FrankProbst says:

    EW @ 5

    What about the e-mail that Rove “found” right before the Libby indictment? They had a hard copy of that one, right? Did they ever find the electronic copy?

    • emptywheel says:

      Presumably yes–My assumption has always been thta Fitz received the email in October 2004, planned to indict Rove, but when Luskin gave his stupid ass excuse for why the email hadn’t turned up before, Fitz started looking for the other emails. That was in October 2005, just before indictment week (and note, Fitz also got additional testimony from Adam Levine regarding an email sent at the same time as Rove’s Hadley email), which may bethe explanation for the change in archive policy. And then Fitz got his emails by around February of the following year. Rove got his all clear in June of that year.

  15. bmaz says:

    A few scattered thoughts:
    1) LHP @11 is right about “spoilation” of evidence and the inference against interest to the spoiling party. The PRA makes the matter pretty cut and dried.

    2) Best Practices? You have got to be kidding me; the twit should be fired on the spot for putting such a spurious and laughably false statement in writing anywhere, much less a formal court pleading/response. This is insane.

    3) This quote by Payton cracks me up:

    When applied to email recovery, the process is complex, labor intensive and costly. By way of hypothetical example, a request to recover a specific file(s) from a particular date or date range within the period of 2003-2005 would be forwarded to the OCIO [that is, to Payton]. The OCIO would then ordinarily consult a media database, similar to a catalogue or back-up tape index, to identify a range of tapes that correspond to that request. In this process, the OCIO may pull tape sets backed up prior to and/or subsequent to the target date period to ensure they have the full population of potential tape sets that may contain the requested file. Additionally, the OCIO would then restore the data-type environment, applicable software, and/or information, and then conduct a search for the information requested.

    So what she is saying is that such a request would call on her to actually DO HER FUCKING JOB. Oh my, the horror of it all.

    4) Payton says “That process is underway and we expect the independent assessment to be completed in the near term.” Um, I thought it HAD been underway. Do you have the freaking emails or not? Yes or no? This is stupid, they aren’t even trying.

    5) Has anybody asked the OCIO from the Clinton white House what they did? Why not?

    6) This Payton twit can’t replicate CREW’s pie chart or whatever, she sure as hell doesn’t contradict it; that is all that really matters.

    7) They had to take until one minute before midnight to file this piece of shit? This piece of dissembling, disingenuous, non-responsive crap could have been ginned up in an hour after the court entered it’s order. The Administration is fucked and they know it. They are like kids at the table running out dinner hour by scrambling the food on their plate, hoping the parents don’t notice and they can just move along.

    Back to the non-responsive part. Read this filing against the court’s order. I would suggest it is non-responsive to the point of being contemptible.

    9) Any member of Congress refusing to consider impeachment proceedings, against somebody/anybody, is so derelict in their duty and oath to office that they are unfit to serve. It is that simple.

  16. BlueStateRedHead says:

    More dataidea mining fromDkos. not sure if he means the wingnuts gotten a hold of the stuff postfacto but get a hold they did.

    Anything they can do we can do better?

    I worked in the EOP during the Clinton administration. I recall the administration having been accused of the same type of e-mail chicanery toward the end of our days in office (fairly in some ways, unfairly in other ways given that we were the first WH to really use email on a day-to-day basis and that the technology was so limited at the EOP I’m not surprised there weren’t reliable e-records being systematically archived). The right wingers went nuts with claims that the administration did every evil thing under the sun via email then erased every trace of such. Some Free Republic types even went so far as to post some of the email that was able to be extracted as examples of all our transgressions. In fact, my wife (who also worked there) had some of her email posted on the internet by those freaks.

    So now it seems we’re in the same situation with the bush administration, with the same excuses being made about the taping over of archives and such. Wonder if the wingnutospehere will go equally ballistic that records were destroyed?*

    *That was a rhetorical question, of course.

    Permanent good can never be the outcome of untruth and violence.

    by bakerkm45

  17. maryo2 says:

    Payton says “this office has undertaken an independent effort …to see if … anomalies in Exchange email counts resulting from the potential failure to properly archive emails”
    Two things: 1. It is not “independent” if a judge has ordered you to do it. 2. The question is not whether the emails are missing -because- of failure to preserve backups. (The emails are missing because people deleted them.) The question is what do the backup tapes that you do have tell you empirically about the number of emails to expect in the time period in question. And do the expected number of emails exist in all Exchange accounts (Rove’s, etc.) on the backups — or can you infer that the backups are fewer in number than is reasonable to expect?

    • MadDog says:

      2. The question is not whether the emails are missing -because- of failure to preserve backups. (The emails are missing because people deleted them.)

      This is a common misconception of non-techies. I don’t meany you maryo2, but for the folks in the WH and its attorneys who think that an individual email user, someone like Karl Rove for example, has the ability to delete his emails.

      Yes, Karl thinks he can just open his mailbox and delete emails, but that ain’t the same thing as really deleting them on the server.

      And just so we’re even clearer about this, ain’t no feckin’ way that Karl Rove was given Administrator’s access to the WH’s EOP Microsoft Exchange email servers.

      Even if Karl was given the Exchange server’s Admin logon ID and password, he would have absolutely no clue on how to go about finding these emails, and then somehow deleting them from the server itself.

      Deleting one’s emails from one’s own mailbox ain’t deleting them from the server.

      And even further, if the Exchange server has massive hard drives (which is typical of email servers), even using the Exchanger server’s Admin privileges to delete emails doesn’t really delete the data from the actual hard drives. Low level utilities can still recover stuff from there.

      So in the end, the WH drivel about Karl “deleting” his own emails is just that, feckin’ drivel!

      As bmaz so eloquently put it at # 30:

      7) They had to take until one minute before midnight to file this piece of shit? This piece of dissembling, disingenuous, non-responsive crap could have been ginned up in an hour after the court entered it’s order. The Administration is fucked and they know it. They are like kids at the table running out dinner hour by scrambling the food on their plate, hoping the parents don’t notice and they can just move along.

      These folks are dissembling like crazy and only a naive 4-year old could possibly buy their BS.

      • AZ Matt says:

        These folks are dissembling like crazy and only a naive 4-year old could possibly buy their BS.

        That would be George.

  18. BlueStateRedHead says:

    5) Has anybody asked the OCIO from the Clinton white House what they did? Why not?

    see my @33.

    Got to sign off to do the day job.

  19. bmaz says:

    Why didn’t Helen “Kong” Hong simply submit a one sentence declaration to the effect “The Filipino Monkey did it!”? Really, at this point, it would be just as credible….

  20. Rayne says:

    We learned a bunch of stuff today, didn’t we?

    1) Reliance on “industry best practices” — they will trot this out, a nice little piece of spin that will work on the corporatists in the crowd who are easily fobbed off with buzzwords and corporate-speak. But this is GOVERNMENT, not industry, and the law dictated the practice, best or otherwise.

    Come to think of it, if a corporation did such a pissy job of adhering to reporting requirements (like SarbOx compliance), the guilty parties could find themselves in a serious world of hurt. They’d have been fired at the least, prosecuted more likely if a pattern of evasion that exposed the company to legal jeopardy.

    “Industry best practices” my left arse cheek. Not unless your industry is racketeering.

    2) “EOP network” — yeah, that’s what I thought; the executive offices including Fourth Branch are all part of the same data center. 200 servers? I am NOT impressed, and I would put money on it right now that in the time frame in question, we’re talking about DATA servers, not EMAIL servers for 3000 people. I know for a fact that a 6000 person enterprise running Exchange could run on a mere 10 email servers, depending on the size of the RAID sets. The 200 figure is a red herring, don’t let it throw you off. Even running a parallel secure system wouldn’t require 200 servers. (In 2002, I managed more than that, globally located.) Ditto the 100 applications; how many do you have on your machine right now, the one you’re reading this with? How many on the servers you’re attached to as well? She’s indulging in puffery (or her ghost writer is).

    3) Where’s the manuals? Yeah, the documentation they used to manage their systems; this particular agency would not only have a crapload of documentation, but they’d also have added a bunch of new documentation since 2001 reflecting a hardened system. Payton’s still avoiding the fact that those “industry best practices” were theoretically derived from corporations’ IT documentation, and that there’s all kinds of firm guidance in black and white that should have prevented this inexcusable loss of data.

    4) “process is complex, labor intensive and costly”, in regards to email recovery. Bullsh*t. Bullsh*t. Bullsh*t. Maybe it’s expensive if you keep your email backups in a black site in Thailand, but that’s not to “industry best practices”, is it? Email recovery should have been a snap; I know this from experience.

    I’m sure I’ll see more, but I’ve got a splitting headache from being up all night with this joke — excuse me, clusterfuck — of a primary.

  21. WilliamOckham says:

    I’m still snowed under at the day job, but here’s a few quick hits from my initial glance throught the WH document:

    They archive email using .pst files?!?!? (I’ll be back to explain. This is very revealing and a little weird)

    Recycling the backups that contain your email archives is definitely not industry-standard practice.

    I’ll have to talk to some folks I know to find out a little more, but I think there are some good clues to what’s been going on.

    • MadDog says:

      They archive email using .pst files?!?!? (I’ll be back to explain. This is very revealing and a little weird)

      My thoughts exactly! Talk about freakin’ amateur-hour! I was just about to post this:

      Since the phased deployment of Microsoft Exchange as the email system for the EOP network, that process has entailed saving journaled email files to .pst files that are then maintained on the EOP network…

      …For example, the snapshot captures all email information present on the EOP network in the journals, the .pst archives, and the customer mailboxes at the time the back-up is created. In the event of a disaster or catastrophic incident that results in the partial or total loss of operating systems and data, these tapes could be used to reconstruct the data on the EOP network as it existed at the time of the last back-up…

      …Similarly, any emails sent or received after the latest back-up but prior to the disaster or catastropic incident may not be backed up on tapes and, thus, may not be reflected in the reconstructed system. This limitation in disaster recovery back-up process is consistent with industry standards for these types of disaster recovery systems…

      And say just like Rayne, Bullsh*t, Bullsh*t, Bullsh*t!!!

      Unless the WH deliberately hired the most incompetent IT folks who submitted resumes, then this is an IT organization run by the 3 Stooges.

      And given the fact that EW has told me personally of Microsofties who are under contract to provide services to the WH, and I know for a feckin’ fact that Microsoft don’t hire IT idiots, then something smells like a turd in the punchbowl.

      • merkwurdiglieber says:

        They are talking shit, hoping nobody will know enough tech to call them
        on it… hope it don’t work!

  22. Jeff says:

    Two quick comments. The only email traffic I have been able to find that was included as evidence at Libby’s trial that seems to have been part of the February 2006 recovered emails dates from . . . October 1, 2003. (It’s GX 418, and was produced under Addington’s certification apparently, having been printed out February 2, 2006.) But I’m not sure how that fits with what we’ve just learned today, since I’m not sure why an October email wouldn’t have been preserved and found in the first place.

    Also, on a related note to do with emails, though this time the RNC emails in part, let me just recall this odd detail, working from Waxman’s interim report that Marcy blogged back in June here. As I noted at the time, Waxman says:

    The first e-mail sent by Mr. Rove that the RNC has preserved is dated November 26, 2003.

    Hmm. Corn and Isikoff’s Hubris, p. 377-78:

    A hard copy of the Hadley-Rove e-mail turned over to Fitzgerald (which was independently obtained by the authors) showed that it had been printed out of Rove’s White House computer on November 25, 2003. One of Rove’s assistant’s, B.J. Goergen, had searched the computer that day at the request of Rove’s attorney, Robert Luskin.

    Got that? Rove’s office prints out emails to give to Luskin
    to give to Fitzgerald on November 25, 2003, apparently from his White
    House computer (but who knows if that includes his political laptop or
    his RNC account). Waxman tells us that the RNC has preserved no
    emails sent by Rove before November 26, 2003. (Obviously, Fitzgerald knows all this, so the point is not that this would be news to that investigation.) The day after Rove produced this July 2003 email, his RNC account appears to be wiped clean. What’s up with that? And how did they get the July 2003 email in the first place? Was it just that Rove had himself preserved it, even if it was not backed up (or rather the backup had been written over by then)?

  23. manys says:

    Seconding Rayne above. “Industry Best Practices” say that you put monthly and yearly backups into a vault or via a third-party data and document retention service, never to be overwritten lest there be an emergency or other need for old data. This is standard, across the board behavior, even at small companies.

  24. JimWhite says:

    Dan Eggen at WaPo has a story on the emails:
    http://www.washingtonpost.com/…..id=topnews
    Quote from Amy Weismann, chief counsel for CREW:

    “They didn’t have what any archival person would consider to be an electronic record-keeping system,” Weismann said. “These are not the steps of a White House committed to preserving records or meeting its obligations under the law.”

    Also, Eggen throws in this nugget, overcoming my fears @ 13:

    The back-ups are meant to preserve records in case of a disaster. They also serve a role in ensuring that federal record-keeping laws are met, according to administration officials and records management experts. Two separate statutes require the White House to preserve federal or presidential records.

  25. Mary says:

    I haven’t been following the CREW and Archives litigation, so can someone give a thumbnail on the posture/status?

    I thought it was just FOIA requests, but has it gone beyond that to someone asserting a claim that documents were destroyed (or not kept) in violation of statute? And if so, who has the right to press the claims under the statute(s) at issue?

    I always did think the 200+/- “missing” (from normal archives, but not from prosecution discovery files) emails was one of the most interesting, least discussed, issues from the Libby case.

    So, I guess any emails that might have tied in with the CIA IG’s start up of review of some aspects of the torture programs in “early 2003″ are among the missing?

    http://www.washingtonpost.com/…..id=topnews

    An internal probe of the interrogations by the CIA’s inspector general began in early 2003 for reasons that have not been disclosed. In February of that year, then-CIA General Counsel Scott W. Muller told lawmakers that the agency planned to destroy the tapes after the completion of the investigation.

  26. JodiDog says:

    There are no emails.
    There are no emails.
    There are no emails.

    There was a slight mention of the obvious but with the wrong conclusion. If there are emails, then some one sent it, someone got it, someone got a copy or two.

    But no one has come up with anything anywhere with anyone, which further proves:

    There are no emails.
    There are no emails.
    There are no emails.

    And also
    there are no torture tapes.
    There are no torture tapes.
    There are no torture tapes.

    It is sort of like UFOs, Kennedy’s assassination Martin Luther King’s assassination, yellow cake, 911, the two towers.
    People will talk, talk, talk, and never get anywhere except in their own feverish minds

    • bobschacht says:

      There are no emails.
      There are no emails.
      There are no emails.

      Someone appears to be under the impression that if you close your eyes, and repeat something 3 times, and wish real hard, that makes it true. I’m thinking maybe the Wizard of Oz? Or maybe you have to click your heels while your eyes are closed and you’re repeating fervently your wish.

      Doesn’t work.

      Bob in HI

      • JodiDog says:

        Ok, Bob.

        Show me the emails!

        (I’m waiting…)

        By the way, as I repeated the incantation, I turned around 3 times, and then back 3 times. I think that should have worked, and indeed it seems so.

        Actually I would like to see the emails that no longer exist also, because that would prove just like those who still exist that there is nothing there.

        (No one is satisfied.)

          • bigbrother says:

            Jodi is ad,itting he is a party to the obstruction as he KNOWS there are no tapes as do the colurker conspirators. SS by any other name is SS just the same.

          • JodiDog says:

            We have been over this time and time again from the Valerie Plame case.

            The emails are gone! People keep saying they are not, and yet no emails are forthcoming.

            If I say “the emails are gone” I am only stating the obvious. However some people like a little more glitz to the statement. I was just providing it.

  27. maryo2 says:

    April 13, 2007 WH Press Briefing
    Perino: “… we do not have any indication that there was any basis to conclude that there was any wrongdoing, intentional wrongdoing in the use of the RNC emails. You’re talking about the double-delete function, where you can delete your deleted files?”
    ****

    Notice that she does not say that there was no wrong doing, she says (rather blatantly) “Na na na boo boo, you don’t have proof that there was intentional wrong doing.”

    Then notice she uses the phrase “the double-delete function, where you can delete your deleted files” as if she knows EXACTLY what is meant and how it is done. Nobody else at the pres conference said “You know, Dana, the ‘double-delete’ function.” She used the phrase in a way that implies that she is VERY aware of what and how.

  28. merkwurdiglieber says:

    After a quick wiki of .pst it appears that the files are still on the hard drives from 2003 forward, and any backups made. Just guessing, but,
    looks like hard drives and tapes would have to be wiped or destroyed to
    eleminate files permanently.

  29. Mary says:

    42 –

    The day after Rove produced this July 2003 email, his RNC account appears to be wiped clean. What’s up with that?

    The price of oil? The number of American soldiers killed, maimed, wounded and disabled? The number of people who hate the US? The uncollected bodies in Iraqi morgues? The number of detainees in Iraqi, Afghan, Cuban and blacksite facilities? The “misstatements” on the Congressional and Court records by DOJ employees? The Iraqi refugees in Syria? …

    Oh – you just meant isn’t that timing kinda interesting, what with pull out what you want/need, then wipe the rest clean, eh?

  30. Mary says:

    11 & 30

    1) LHP @11 is right about “spoilation” of evidence and the inference against interest to the spoiling party. The PRA makes the matter pretty cut and dried.

    I guess it is because I haven’t followed the cases, but I’m kind of not following here. What is the right/remedy/standing posture on the cases? They seem to be civil actions and are they just to force FOIA releases or are they to pursue some kind of remedy under the PRA and if so, what is the remedy and are they the right guys to ask for it?

    IOW, are you looking at a criminal violation of statute or a statute that gives a civil right of action? I’m either out of the loop or looped or both.

  31. malcontent says:

    Rayne and WO are right. This can be best described as malicious obediance, not “consistent with industry best practices”…

    For those who don’t deal with this on a daily basis an excellent backgrounder on backup tape management is here

    http://en.wikipedia.org/wiki/G…..Son_Backup

    Once you understand the elegant simplicity of managing your tapes, it becomes perfectly clear that Helen is a)unqualified b)mendacious or c)both.

    Keeping your email in a PST file can get you fired at many companies because it obscures the IT department’s responsibility of providing communications paper trails to HR when the company faces sexual harrassment charges among other issues like Sarbanes Oxley compliance audits.

    You keep your mail in a PST when you want total control of it including the ability to destroy it quickly or take it with you when you leave. More PST’s are out there folks. More tapes have at least some of this data. Is there a Bastard Operator From Hell somewhere willing to share some of this data?

      • MadDog says:

        Can you say more, at least to hold us over until WO comes back?

        PST files are Personal Store folders that allow MS Outlook users to keep a copy of their email on, generally, their local hard drive.

        Theresa Payton’s declaration states that:

        … example, the snapshot captures all email information present on the EOP network in the journals, the .pst archives, and the customer mailboxes at the time the back-up is created…

        This means a couple of things:

        1. The WH EOP network using Microsoft Exchange server for email does journaling! See my # 73 above for Microsoft’s definition of journaling.

        2. The WH EOP network usage of Microsoft Outlook’s PST is strange because PST usage is old-style technology from the early days of Microsoft Mail (verus Microsoft Exchange. These are two different beasts) with the contemporaneous technology which provides much of the same capability (and less of the downsides of PST usage) is the use of OST files (Offline Storage files) for allowing users to access their email when they are offline from the Exchange server.

        PST usage is also still used with Outlook when one does not have a Exchange server. For example, I used to use Outlook as my email client on my home PCs.

        I’d have Outlook configured to get my email from Comcast’s email server. The mail I’d get would then be stored in my Outlook PST. It was available offline, and more importantly, it was configured to keep the email in my PST file, but delete it from the Comcast email servers.

        A number of years ago, I relented and just used Comcast’s own web interface for email access, and stopped using Outlook on my Home PC altogether.

        But, I still have copies of my PST file on my Home PC hard drive taking up space. *g*

        Note that while I said that my usage of Outlook with the Comcast email servers was setup to download email to my hard drive and delete them from the Comcast email servers, you can bet your bippy that Comcast was doing journaling/archiving/backup and that those emails still exist somewhere in Comcast’s offsite storage.

        • bmaz says:

          But, I still have copies of my PST file on my Home PC hard drive taking up space. *g*

          Thank you. Now we have them too.
          Best regards,
          D. Cheney

    • merkwurdiglieber says:

      ExPFC Wintergreen, where are you? The has to be a copy somewhere, there
      is always one kept for restoration at some future point.

  32. bmaz says:

    Okay. I raised this last week (I think?) when we were discussing the number of employees in the White House.

    I found this list, which I estimate to be well under 500 people. Check out all the names; this is one inbred group of wingnuts. The level of total nepotism among friends and family is beyond belief. This name caught my attention:

    Mitnick, John M Associate Counsel to the President
    Is he related to Kevin Mitnick? If he is, there is one source of potential computer tomfoolery.

    Anybody have any clue if these two Mitnicks are related?

  33. maryo2 says:

    Payton/Hong wrote “Additionally, the OCIO would then restore the data-type environment, applicable software, and/or information, and then conduct a search for the information requested.”

    Has the software used for creating and restoring from backup tapes changed since 2003? If it hasn’t changed, then you do’t have to restore the old product.

    Has the software used for email changed? If not, then then you don’t have to resotre it.

    Don’t worry about searching for the requested information. At this point, you just give the backup media to the courts and they will have professional techies see what is on it.

    • MrWhy says:

      Brief History of Microsoft Exchange Server & Outlook:

      Exchange Server 2000 (v6.0) released 29 Nov 2000
      Exchange Server 2003 (v6.5) released 28 Sep 2003
      Exchange Server 2007 (v8.0) released late 2006

      Outlook 97 (8.0) released 16 Jan 1997
      Outlook 98 (8.5) released 21 Jun 1998
      Outlook 2000 (9.0) released 7 Jun 1999
      Outlook 2002 (10) released 31 May 2001
      Outlook 2003 (11) released 21 Oct 2003
      Outlook 2007 (12) released 30 Nov 2006

      No government organization would roll out the application within 3 months of the initial product release except under threat of torture or dismissal extreme duress. Tradition suggests waiting till SP2, but there are always optimists and early adopters who feel they need the latest features. My office skipped Outlook 98.

      Educated guess, for the period in question, (post 9/11, pre Nov 2003) the Server version would likely be Exchange 2000, and the client Outlook 2000, possibly Outlook 2002. The desktop OS would most likely be Windows 2000 with a transition to Windows XP in late 2002, early 2003. I’d bet there was an archive snapshot before the transition to XP, and would expect a snapshot before the release of the Outlook 2002 client.

      Government archive policy dictates certain things, but we know that OVP doesn’t feel constrained by statutory policy.

  34. WilliamOckham says:

    Folks,

    The line about .pst files may not be what most of you are thinking. It says they saved “journaled emails” to .pst files. Journaled emails are captured at the Exchange Server level. There are different ways to set up journaling. There are valid ways to use this functionality for back ups of emails, but it seems a little strange.

    I’ll come back in a bit and explain how it works. If you feel really geeky, just google “Exchange journaling” and read the first microsoft.com page hit you see.

    • MadDog says:

      WO said:

      The line about .pst files may not be what most of you are thinking. It says they saved “journaled emails” to .pst files. Journaled emails are captured at the Exchange Server level. There are different ways to set up journaling. There are valid ways to use this functionality for back ups of emails, but it seems a little strange.

      I’ll come back in a bit and explain how it works. If you feel really geeky, just google “Exchange journaling” and read the first microsoft.com page hit you see.

      And since some of you can’t wait for WO’s return, here’s an MS definition of journaling as it applies to Exchange server email:

      “Journaling is the ability to record all communications in an organization. E-mail communications are one of many different communication mechanisms that you may be required to journal. Therefore, journaling in Exchange has been developed to enable the messaging administrator to feed messaging data into a larger journaling solution, while using minimum overhead.

      It is important to understand the difference between journaling and archiving. Journaling is the ability to record all communications; alternatively, archiving refers to reducing the strain of storing data by backing it up, removing it from its native environment, and storing it elsewhere. That said, you may use Exchange journaling as a tool in your e-mail retention or archival strategy.”

      • sojourner says:

        Quick drive by but there are a couple of things to remember… The WH migrated from Lotus Notes to Exchange (if I remember correctly) in 2002 — 2003. The .pst suffix is for a personal archive (although I guess it could also be a journal — I haven’t looked).

        Early on when it came to light that there were so many emails missing, I had the thought that maybe, just maybe, the people did not look for .pst files.

        There is one other thought, too… If they had a phased implementation of Exchange, did they search for Lotus Notes archives? I offer that up because I managed a project about 18 months ago in which we had to convert Lotus Notes archives to Exchange. It was not an easy process. I am willing to bet that there are a lot of Lotus Notes archives still sitting on some hard drives.

  35. Neil says:

    Backup tapes of servers – files and message stores (exchange) – is probably not the same “system” as “archiving” communications for the Presidential records act.

    I think WO is talking about the archiving process which would record all sent messages.

  36. maryo2 says:

    Theresa Payton is Chief Information Officer in the Office of Administration, but she didn’t write the response and she doesn’t speak techno-ese. Where was she employed prior to this job? How and when she did she get this job? How old is she? Where’d she go to college? What are her family’s political contributions? (You know, the usual Bush appointee run-down.)

  37. BlueStateRedHead says:

    She is an IT person, member of Women in Information Science and Engineering (WISE), and one with thoughts about leadership:

    Good leaders develop through a never-ending process of self-study, education, training, and experience. The discussion with our panelists and the attendees was true proof that this should be our credo.” — Theresa Payton

    http://www.localtechwire.com/b…..y/1161418/

    more as I collect it. I have more or less given up on doing work today other than the off with my head if not type.

  38. BlueStateRedHead says:

    Here is the job she was doing in Charlotte:

    Theresa Payton, senior vice president of retail and channel technology at Wachovia Corp., a financial-services firm in Charlotte, N.C., agrees. “As women, it is important that we acknowledge our responsibility to the future women leaders. We need to continue to evolve the workplace culture to advance this field as an attractive option for young women to enter.”

    after that, a senior vice president for Bank of America, in 2005.

  39. BlueStateRedHead says:

    and guess what, a donor to the RNC.

    Theresa Payton
    Senior Vice President
    Bank Of America

    RNC
    $750
    2004 907 E Worthington Ave
    Charlotte NC

  40. bmaz says:

    Theresa Payton has been a senior executive at Bank of America as well as senior vice president of retail and channel technology at Wachovia Corp., a financial-services firm in Charlotte, N.C.

    BUT GET THIS: the link takes you to a Google page; then click on the link that reads “CIO Boot Camp – November 8-9, 2006″. Wow. she not only went to a boot camp to learn her job, it was run by freaking Frederick of Hollywood Thompson!

    • bobschacht says:

      Theresa Payton has been a senior executive at Bank of America as well as senior vice president of retail and channel technology at Wachovia Corp., a financial-services firm in Charlotte, N.C.

      BUT GET THIS: the link takes you to a Google page; then click on the link that reads “CIO Boot Camp – November 8-9, 2006″. Wow. she not only went to a boot camp to learn her job, it was run by freaking Frederick of Hollywood Thompson!

      Is that where she learned how to sleep on the job and let others (like Helen “Kong” Hong do the work for her?

      Bob in HI

    • CTMET says:

      CIO bootcamp is not that unusual. The CIO job is notoriously difficult and most people don’t last in it. Some people come into it as technologists and can’t handle the business aspects, so people come in with a business background and can’t deal with technology. CIO is often referred to as “Career is Over”.

  41. manys says:

    Reading the filing:

    A.4. is just mind-numbing geekspeak. Can be ignored.

    A.5. She says .pst files are “maintained” on the EOP network, but not that they are backed up from there. She says that “part of” the OCIO’s responsibilities is an implementation of archiving so that, say, mails older than 30 days are saved there instead of cluttering up one’s inbox. However, this does not speak to backing up the server or where they are backed up from, which would be from the Exchange Server itself. Exchange does not use .pst’s to store its data and backups of Exchange servers do not convert its data files to pst’s. (Microsoft’s word on the issue). This section is pretty much fluff, as pertains to the missing email.

    B.6. Here she acknowledges only in the last part of the last sentence that “email databases” get backed up onto tape. These glossed over email databases are where the gold is.

    B.7. This is a general description of the purpose of backing up data. It’s fine, but seems to have an ulterior motive in foregrounding a concept of “limitation” in these backups. Leaving herself an out, as it were, but we’ll see if it gets used.

    B.8. This is an overview of a sample recovery scenario. I believe that when she says the email recovery process is “complex, labor intensive, and costly,” she means to say that recovering email requires someone to do something, somewhere. It’s not particularly difficult to recover email for a properly backed-up Exchange Server, and by way of anecdote I have in the past routinely recovered individual emails and .pst’s for executives in a 40 person (small!) company, people who think the delete key is shorthand for “let the IT guy work his magic.” Many, many IT people have done this and in fact is probably a leading source of specialized Exchange backup tool sales, since invariably the first thing that happens after an Exchange server is installed that someone wants to get something back from the past.

    C.9. OK
    C.10. Denigrates plaintiff’s chart-based rationale.
    C.11. More chart. OK, so she can’t verify the chart. What does that say about the emails? Nothing yet. Well, not so quick. She has started people on a (I’m guessing a complex, labor intensive and costly) project to compare the counts of emails. What they are probabaly doing here is getting backup statistics from the tapes themselves and comparing those with the backup logs that might be somewhere else. Maybe, but I’m guessing here. Suffice it to say that this is not putting people to work recovering the emails, they just comparing some numbers and hope to be finished “in the near term.”

    D.12(a) Here she turns the recovery process on its head, seemingly in an attempt to be confusing. She could just be a bureaucrat with no concept of logical paragraph structure though. The gist of this response is that the tapes themselves have barcodes but no date informating ON THE TAPE. But the bar code can be scanned to figure out which period the information on that tape covers, as well as being able to search the media database for particular files and having the database list the relevant bar codes, tape IDs, or other identifying characteristic for that data. They can go both ways, though: they can scan a particular tape to see what data it has, and they can search within their backup software to find the tapes that cover the data/dates desired.

    D.12(b) This one’s pretty funny. She’s saying there’s “a field” in the database of their backup software that identifies the “type” of server for that tape or backup session. Of course, what she doesn’t mention is that there are lots of other fields, too. She is focussing on the most minor bit of information in an identifying process, kind of like saying that you don’t know what kind of car you have because you only look at the tires. Of course this field seems only to say what kind of server is involved (possibly a “Type” or “Description” column in the interface). So what she’s saying here is that Exchange servers are identified as mail servers in their backup software. That’s all.

    D.12(c) “We reused tapes until October 2003.” Typically this would be daily tapes, since (as I posted above) it’s industry best practice to retain some periodic backups permanently, typically yearly and monthly. Some organizations will recycle monthlies once the yearly is vaulted, but since the EOP backup practice was modified in October they still would have had all monthlies at the end of the year when taking the annual backup. This is if they in fact save these monthlies as might be standard practice. The balance of the response says that they should still have everything from 10/2003 forward.

    D.12(d) This part seems to have been written by a lawyer. They don’t know if they have the emails, presumably because all of their backup engineers are still counting and comparing (cf. B.8 & C.11). “However,” as in the previous response, due to their post-10/2003 practices they “should” have the emails.

    That’s my take on it, anyway! If I boil it down in my head she’s saying that pre-10/2003 emails are of unknown existence and post-10/2003 “should” be there. Not exactly a clear-cut response.

    • BlueStateRedHead says:

      This is great stuff. Your stuff,not my googling. Surely there is a way to communicate all this technical expertise to the right people, be they the plaintiff or the appropriate congress critter. Lurkers take note!

    • Rayne says:

      BTW, most excellent assessment.

      I like B.8; I have the sense that she’s never actually done anything more than snapped her fingers and told some IT grunt to fetch a backed up copy of her own .PST in her lifetime. And of course the IT grunt told her, Boy, that’s going to be a while, are you absolutely certain you need it? in order to discourage yet another suit from acquiring the notion that backups are something we should recover every day at a whim.

      And D.12(b)clinched it for me that she’s a suit who hasn’t really gotten her hands dirty doing any IT work in the recent past. It’s like she never even took a freshman level class in databases. But perhaps I’m assuming she’s the one who wrote this and not a horde of lawyers.

    • CTMET says:

      There is always someone within the Netroots who can answer a question. My turn today….

      This is a person someone may want to talk to…

      http://www.evanta.com/details_…..038;id=246

      Carlos Solari
      Former CIO
      Executive Office of the President, The White House

      ——————————————————————————–

      Carlos Solari has most recently completed a successful assignment as the Chief Information Officer for the Executive Office of the President (EOP – The White House) – a position first created in August 2001. Solari has been directly involved in the field of information technology for over 24 years, including 6 plus years as a Senior Executive with the FBI and 13 years as an officer in the U.S. Army where he earned numerous commendations and Ranger Tab, Airborne Wings.

      While serving as CIO, Solari transformed the EOP IT infrastructure to a modern, survivable enterprise serving the White House and its 12 component offices – all in the midst of unprecedented cyber and physical security concerns. This transformation was comprehensive and conducted while concurrently delivering advanced business functionality. This overall 2-year strategy involved $60M in federal funds, the reengineering of major business processes and consolidation of systems. Solari also created a first-time system survivability for high assurance in crisis conditions.

      In 2004, Federal Computer Weekly select him as a Top 100 Executive for his work in bringing the EOP from a level 2 rating to a level 5 rating – in the Government Accountability Office enterprise architecture assessment.

      Solari has held positions in various private business enterprises to include launching an independent information systems consulting service – Solari Innovations, Inc. where he currently serves as President.

      • bmaz says:

        Yeah, well Solari ain’t that great of a whiz bang; his state of the art 60 million dollar system, all complete with “survivability for high assurance in crisis conditions” and everything somehow let slip 5-10 million emails. Unless it was intentional. Hmmm, you are right, I wonder what tune this Solari bell will sing…..

      • Rayne says:

        Now THAT’s a beauty of a comment.

        And it spells out exactly what I’d seen in GAO documentation, that the government’s infrastructure was hardened in the wake of 9/11 to prevent a loss of “business continuity” in the event of a catastrophic attack. Should have been a LOT more redundancy in the system than required by the so-called “industry best practices”.

  42. JohnJ says:

    Well I learned something; I always assumed the only risk to destroying evidence was a separate charge. I didn’t realize that there was a remedy for the effected case as well.

    This changes the level of wrong doing required to make the trade off of the destruction charges compared to what the evidence would prove. I would say that in this light, there is much more hidden in the “lost” data than I originally assumed. they are risking losing any case as well as destruction of evidence charges.

    For me this also applies to my take on the torture tapes as well.

    This is much more interesting than I thought!

    • bmaz says:

      JohnJ – The pleading and argument of lost/destroyed evidence issues can get fairly intricate. In a broad stroke nutshell, however, as LHP indicated, In civil cases the remedy is generally an inference against the party that has lost/destroyed the evidence; with that inference being reduced to a formal jury instruction. If that is the critical element of the case, too bad. In criminal cases; the general rule is that lost/destroyed evidence is sanctioned, with the sanction being dismissal of those specific parts of the case surrounding said evidence, but the remainder of the case survives. If the lost/destroyed evidence is found to have been exculpatory/potentially exculpatory, the remedy is dismissal of the entire case.

  43. earlofhuntingdon says:

    What credible information do we have that suggests the White House is even now complying with the requirements of the Presidential Records Act? What have they done to fix earlier admitted non-compliance? If little or nothing, haven’t we another case of continuing, ongoing illegal behavior by President Bush and those who conspire to prevent or delay curing such illegalities.

    If Congress does nothing in the face of serial violations of law, who would not predict that Bush/Cheney will claim that inaction as a positive affirmation that whatever the level of this administration’s compliance, Congress acquiesced in and, therefore, ratified it? Hence, all are immune, and we still ain’t got no tapes. I don’t know how a federal prosecutor can put away private citizens any more when their own bosses are so corrupt.

  44. earlofhuntingdon says:

    Regarding the writing style in the response Paynton signed, it is great art. It takes considerable skill to so obfuscate the facts one is obligated to reveal.

    My guess is that one of the new White House counsel used to be an SEC lawyer for a Fortune 100 company. It takes the same sort of skill to formally disclose company actions and a CEO’s paycheck without letting anyone know what the company did or how much their top exec is really paid.

  45. bigbrother says:

    Ok Thank you all for spending time and energy and expertise. Please answer one question for me. Does impeachment procedings give more subpeona power. Thus facilitating open government?

    • bmaz says:

      Not necessarily more subpoena power, but fewer and less onerous ability to restrict the subpoena powers through claims of privilege, secrets, etc.

  46. WilliamOckham says:

    I’ll disagree with manys about one thing. Paragraph a4 is interesting for what it doesn’t say. It doesn’t mention the PRA at all. That’s a mind-blowing omission in this context.

    The typical use of journaling in MS Exchange is to send copies of all your users email to a third-party archiving system. If someone told me that the EOP was using journaling, I would assume that they were using it as the interface into the the archiving system set up to satisfy the PRA. Saving .pst files of the journal mailboxes is weird. It almost makes sense, in a twisted sort of way. Backups of Exchange are almost worthless. If you regularly grab those to a .pst file, at least you could reconstruct the emails.

    I’m unbelievably curious about how they did it. I would guess that they probably used some custom CDO code (since running Outlook sessions would be insane). That would imply that they actually thought this through and decided it was a good idea. That’s the part that baffles me.

    • MadDog says:

      Saving .pst files of the journal mailboxes is weird. It almost makes sense, in a twisted sort of way. Backups of Exchange are almost worthless. If you regularly grab those to a .pst file, at least you could reconstruct the emails.

      I was still getting stuck on why in the world would these folks use PSTs instead of OSTs. OSTs are useful for folks with laptops so they can read their email offline and then get synched back up with their Exchange email servers once they get connected again (either via LAN or dial-up).

      But then an answer dawned on me.

      Using PSTs, the WH folks could connect and get their emails from not only from WH EOP Exchange servers, but from places like the RNC email servers, and other email accounts they had (Gmail, HotMail, etc.), and have all their emails available in one email client for ease of accessibility.

      And if that is true, then all those missing RNC emails that Karl Rove was hiding missing are hanging about in the PSTs.

      • WilliamOckham says:

        MadDog,

        It’s not clear from this document, but I don’t think the .pst files in question are for users. This thing implies that they were journaling emails to .pst files for archiving purposes.

        It is clear from the DOJ document dumps that many people in the Administration used local .pst files the way you are suggesting.

        I have to run off to a meeting right now, but the statements in this document don’t line up at all with the public claims of the WH press office (I know, you’re shocked).

      • Rayne says:

        Yes. Exactly.

        There were not one but TWO email accounts, at a minimum, that were going through the network for specific key individuals. They were getting intertwined, as we saw from the debris that Rove’s and Abramoff’s little pet Susan Ralston disclosed in the emails that Waxman obtained. There are more than one .PST out there, and one or more of them may be duplicated on parallel systems inside the firewall and out. Sure seems strange, doesn’t it, that the RNC AND the EOP network both had similar problems with different email accounts at the same time, doesn’t it?

        Oh, and jodidog (48) — nice mantra, but pretty ineffectual. You see, the Blackberry server likely logged the traffic it sent into and received out of the EOP Network, and the Blackberry server isn’t part of the EOP’s purview as part of an external service provider, meaning that all Blackberry archiving actually does meet so-called “industry best practices”. That’s why we all know that there are emails and the approximate number. That’s also why we know Ms. Payton hasn’t been hired for her IT acumen, but for her ability to obsfucate.

        Did you see the word “Blackberry” in that one-minute-to-midnight-filing from Payton? Yeah, me too. Yet another shadowy outline that points to a missing object.

        • JodiDog says:

          All sorts of theories and confidences have been advanced by people, claiming to know this or that part of program and computer technology, asserting that the emails are still available.

          I maintain simply that no one has yet presented them after all this time, and thus it is becomes safer and safer to simply say that they no longer are available.

          Like the movie. “Show me the money.”

          Rayne, show me the emails!

          • Rayne says:

            You’re being redundant and pointless.

            There are already plenty of clues that indicate the emails are there — emails don’t die — and that the administration is simply not providing the emails.

            If you can’t grok that part, maybe you need to sit this out.

            • JodiDog says:

              I begin to think this is just a lot of hoopla designed to keep the less fervent troops interested.

              “Show me the Emails!”

              • manys says:

                I begin to think this is just a lot of hoopla designed to keep the less fervent troops interested.

                Since you don’t criticize anything except that which you already oppose, there’s no reason to think that “beginning to think” carries any credibility to speak of.

  47. biggerbox says:

    I’m in no way an expert on system backups, but even I can detect bafflegab here. I notice Payton refers to industry best practices for “disaster recovery”, not archival storage, which is what they are required by law to provide, right? It sounds from these comments like they actually weren’t using best practices for any situation.

    I had to laugh at the extensive description of the “complex, labor-intensive, and costly” process to recover something. It essentially translates to, “Well, I’d look up where it was on the tapes, go pull the right tapes, and then get it off.” That’s pretty much how back-ups work. It’s done thousands of times a day in businesses around the nation, I’m sure.

    Of course, one could avoid having to use the backups if they didn’t improperly delete information in the first place.

  48. BayStateLibrul says:

    Is fucking around with the Presidential Records Act, in deadly pursuit of a crime, considered a “high crime and misdemeanor”?
    Yes.
    Let’s vote.
    Don’t let him back in the country.

  49. earlofhuntingdon says:

    Historical archiving and disaster recovery are two different things, especially for the federal government.

    Simply put, disaster recovery is directed at restoring the current operating system and current files so as to minimize the interruption in routine business. Historical archiving of qualifying materials (which should include ALL WH communications) relates to devising, implementing and maintaining permanent records. It would include a registry or indexing system so you knew what you’ve got and where it’s stored physically and virtually.

    Over-writing back-up tapes is archaic, especially with the dramatic declines in disc storage size and cost, though many legacy systems still use StorageTek or other digital tape systems. In any event, it ought to relate only to disaster recovery. Since the WH’s operations should include separate permanent archiving, it should be irrelevant whether these individual tapes were over-written.

    There should be separate copies of the material elsewhere. So, to some extent, this focus on individual back-up tapes may be a red herring, the classic defense lawyer’s dodge, a fully accurate statement that doesn’t reveal what was asked.

    • bmaz says:

      Looks like a good one to reserve judgment on. Some evidence the girl is trying to extort him. Man, I need to start getting paid for defending the legal rights of all these scumbag athletes.

      • Neil says:

        I learned my lesson with the Duke Lax case. Some prosecutors should not get the benefit of the doubt. I think that prosecutor got off easy. What he did was well beyond legal ethics, it was criminal.

        Any person can get (at least in MA) a temporary restraining order for ten days just by swearing out a complaint. For it to be extended, the restrained has a right to stand up in court and argue against it. Either way, it remains on the record. It has the potential for abuse while it also probably saves lives.

      • bmaz says:

        Ahem there randiego. Remember how they burned that Steeler dude who made fun of them? Ran a touchdown play right over him. After they already had a 120 point lead….

        • randiego says:

          Randy Moss has been a ticking timebomb since he joined the team. Anything that distracts him is fine with me.

          • bmaz says:

            I an sure the Frightning Bolts are in good shape; you have no Moss types on your team, only cool headed customers like Phil Rivers….. Oh and the Miami Ink contestant lineman who is spouting off….

            • randiego says:

              Ahem. well.

              Wait – what has Rivers done exactly? Mouth off to fans who were cheering him being hurt? What else? Rivers has yet to blast his teammates for not getting the ball enough, or squirting a REF! with his water bottle, or get caught up in a potential assault and battery situation, etc etc etc

              Igor Olshansky made his comments right after the Indy game ended. Norv got him back on message – pretty minor stuff really.

              AJ Smith doesn’t accept the ‘marginal citizen’ types. We had some problems last year, all those guys are gone.

              • bmaz says:

                Heh heh. What happened to that dude involved in a shooting driving an old Olds Cutlass or something along those lines? I can’t remember squat about it other than I thought he was getting kind of a bum rap and that the cops were jerks and mostly at fault….

                • randiego says:

                  yeah, that was Steve Foley. He was driving drunk, a second-year off-duty cop in an unmarked car from another jurisdiction a zillion miles away shot him in the leg instead of backing off and waiting for the locals to show up, and ended his career. haven’t heard anything from him lately – I think he has a civil suit pending.

                  The cop got cleared of any wrongdoing, of course. Welcome to San Diego. Oh wait – that was in Poway. (heh heh that was for Marcy)

  50. Mary says:

    96 – Actually, that’s kind of what I was trying to figure out. I took a quickie look at the PRA and http://www.archives.gov/about/……html#2202
    there don’t seem to be any enforcement mechanisms or remedies for violation.

    Which probably goes to a Congress that wanted to be vague-ish over how much they were saying they could direct vis a vis Presidential document retention, but I’m not really sure where remedial aspects like spoilation come in, under a direct FOIA or PRA action, to achieving a remedy.

    I do think that violation of the PRA, in connection with criminal cases where the defendant is alleging WH wrongdoing, or cases like FISA violations where there are statutory civil rights and remedies where plaintiffs may be alleging Exec branch wrongdoing, are impacted when you can show you want discovery of info from a particular time frame and the WH doesn’t have that info bc it violated the PRA. Or as a cost shifting mechanism. But I’m not really following a spoilation argument in the context of the CREW suit itself.

    mustbeabadbiorhythmnsday

    • earlofhuntingdon says:

      Yep, the PRA’s pretty toothless. Given the choice between admitting evidence of a high crime and misdemeanor, or non-compliance with a toothless statute about “files”, which would you choose? The trick is not to get caught in contempt or perjury charges, hence, the elegant obfuscation.

      Congress needs to put teeth in the PRA; certain violations should be felonies. The American people own that man-sized safe Cheney carts around, and all his and his colleagues other records, including his PDA’s, computers, etc. As it is, this Congress is likely to let all those records out the door with little chance of it being properly catalogued and archived before Bush sanitizes the hell out of it. And Payton and her predecessors are likely to get no more than a slap on the wrist and a permanent WingNut Welfare job at the AEI or Bush libarwee.

      We really will need a Truth and Reconciliation Commission.

      • pseudonymousinnc says:

        Congress needs to put teeth in the PRA; certain violations should be felonies.

        That’s hard to work out. The buck stops with the President. Even the Hatch Act is toothless, as Cookies Doan proved.

        The Archivist needs to be empowered, that’s for sure, with ongoing oversight powers. But I honestly think that the PRA is unenforceable against someone wanting to bury the archives unless it either a) is stipulated as an impeachable offense; b) allows an ex-president to be charged. And even b) can be negated by plenary pardon power.

    • bmaz says:

      Mary – I think the spoilation reference LHP made was in response to a general question, kind of like the general response I gave @84 above.

      On the punishment provision, or more appropriately lack thereof, my recollection is that you are right and there is none specified. It has been over two decades, but i would swear that there is a general penalty provision covering all crimes not otherwise specified. It was minimal punishment 6 months incarceration, small fine cap; but it was something. Might have been some provision to the effect of “any and all crimes specified under this title, or otherwise contained in the USC and/or CFR shall be punishable as a ____ level crime”. I found this looking for the smallest thing I could find to give a bone to a prosecutor I had cornered, but who for PR reasons, couldn’t dismiss the case entirely. It may have been under Arizona or California state law, but my recollection is that it was Federal. You heard of anything like this? I am alzheimered…..

  51. Neil says:

    I second what earloH said.

    I think CIO’s response is an attempt to obscure some facts while meeting the strictest interpretation of the judge’s order. Can CREW (or for that matter, could anybody via FOIA) request the ‘as built’ specification of the PRA as well as the ‘as built’ specifications for the disaster recovery plan (aka backup tapes) including what records exist and what records do not.

  52. earlofhuntingdon says:

    Short and sweet: Payton gave us the same practiced brush-off she’d give a retail bank customer who wanted a copy of last year’s checking account transactions.

    Kinda like being told your child’s pneumonia-related hospital bills aren’t covered expenses because they relate to a pre-existing condition (life) by a high school dropout who’s never read your health insurance contract.

  53. bmaz says:

    Hey EW, whats going on up there in Michigan Land?

    A former congressman and delegate to the United Nations was indicted Wednesday as part of a terrorist fundraising ring that allegedly sent more than $130,000 to an al-Qaida and Taliban supporter who has threatened U.S. and international troops in Afghanistan.

    The former Republican congressman from Michigan, Mark Deli Siljander, was charged with money laundering, conspiracy and obstructing justice for allegedly lying about lobbying senators on behalf of an Islamic charity that authorities said was secretly sending funds to terrorists.

    Is this another bogus DOJ charity persecution, or is there meat to it?

    • emptywheel says:

      Wow, I’ve been meaning to click on that headline, but I’m still working on the CIA propaganda that Pincus offered up.

      But it’s probably real enough. They have yet to manage a prosecution here (and let that Lebanese spy from our local ME restaurant scam into the CIA to spy. But they at least claim we’ve got our share of terrorist organizations. I’ll have to look in more detail later.

    • MadDog says:

      Sorry maryo2, but I don’t know.

      I stay far, far away from Blackberries. Our IT organization decided a couple of years back not to “officially” support PDAs because they were all over the map and not generally amenable to the dictates of IT Standards. *g*

      That hasn’t stopped users from getting them nor beating us up to support them, but IT has been fairly successful in holding back the barbarians from the gate…thusfar. *g*

        • MadDog says:

          LOL!

          In a Microsoft-centric organization such as mine, you are speaking heresy, unforgiveable heresy my friend!

          Apple stuff? That’s for tree-huggin’, sandal-wearin’, illegal vegetation-smokin’ DFHs.

          We be corporate flunkies of the highest order!

          Of course, my Unix colleagues here are similarly heretical, but for some reason our Engineers won’t let us burn them at the stake. *g*

          • bobschacht says:

            “Of course, my Unix colleagues here are similarly heretical, but for some reason our Engineers won’t let us burn them at the stake.”

            Perhaps because Unix people actually *know* stuff?

            My impression is that Unix people are the same type as the wonks that actually prefer the MS-DOS text interface, rather than the Windows GUI.
            “Yon Cassius has that lean and hungry look.”

            Bob in HI

    • Neil says:

      Dunno but a lot of law firms that use Exchange also use blackcrackberries. WHen you sync them to your open outlook interface, they do a good job of deleting what you deleted on the blackberry and adding what you added on the blachberry. My guess is that the underlying file structure, whether it be exhchange message store, pst or ost, is irrelevent.

      • bobschacht says:

        Dunno but a lot of law firms that use Exchange also use blackcrackberries. WHen you sync them to your open outlook interface, they do a good job of deleting what you deleted on the blackberry and adding what you added on the blachberry. My guess is that the underlying file structure, whether it be exhchange message store, pst or ost, is irrelevent.

        On my Treo “Smartphone,” the sync operation includes a backup, just in case you accidentally zapped your contacts file or whatever, without realizing what you were doing. Most people don’t pay any attention to this part of the sync operation, which I assume is standard. Does a Blackberry sync operation work the same way?

        Bob in HI

  54. maryo2 says:

    Do we know who the provider for the Blackberry accounts was – Verizon, Sprint, AT&T, QWEST?

    I have found the following (below) which makes me ask if the missing emails are on the servers where the Blackberry server software was installed (RNC servers)?

    http://searchmobilecomputing.t…..30,00.html
    “With BlackBerry deployments, most of the application is sitting on a server, with a small, lightweight client application on the mobile device. … In order to keep the battery life, the device remains light on the computing power, which means that the network must always be there.” BlackBerry generally isn’t meant to work offline, he added.

    BlackBerry doesn’t actually have an open operating system; instead, it uses a Java environment. “

      • manys says:

        No, Blackberry servers in organizations of this type (budget, basically) run Blackberry Enterprise Server, which is a piece of software that sits on top of Exchange. It just provides a proxy for the BB device to access Exchange data, it doesn’t store anything itself and doesn’t send the email anywhere else.

        It works like this: You have an Exchange server that you use with Outlook. You get a Blackberry, but this can not natively access the Exchange data like Outlook does, and BES provides the bridge. It’s kind of like the difference between http://www.google.com and http://www.google.com/m (the mobile interface).

        • bmaz says:

          No reason to doubt you on this, nor any knowledge from which to do so. I do recall there being discussion about these Crackberries being issued by the RNC as opposed to the White House and that they very well might be working off of the RIM backbone or whatever, which is/was in Canada.

          • manys says:

            It’s possible that the RNC is using a hosted BB service for their email, but those are typically for individuals and budget operations. A business of any size such as the RNC will already have the email and IT infrastructure in place to support an in-house Blackberry Enterprise Server.

  55. AZ Matt says:

    OT Ancestral Republican Discovered in South America!
    Looks kinda like Fred

    Scientists discover huge extinct rodent

    LONDON – Eeek! Imagine a rodent that weighed a ton and was as big as a bull. Uruguayan scientists say they have uncovered fossil evidence of the biggest species of rodent ever found, one that scurried across wooded areas of South America about 4 million years ago, when the continent was not connected to North America.

    A herbivore, the beast may have been a contemporary, and possibly prey, of saber-toothed cats — a prehistoric version of Tom and Jerry.

    For those afraid of rodents, forget hopping on a chair. Its huge skull, more than 20 inches long, suggested a beast more than eight feet long and weighing between 1,700 and 3,000 pounds.

    Although British newspapers variously described it as a mouse or a rat, researchers say the animal, named Josephoartigasia monesi, actually was more closely related to a guinea pig or porcupine.

    “These are totally different from the rats and mice we’re accustomed to,” said Bruce Patterson, the curator of mammals at the Field Museum in Chicago, adding that it was the biggest rodent he had ever heard of.

    An artist’s rendering showed a creature that looked like a cross between a hippopotamus and guinea pig.

  56. maryo2 says:

    I wonder if there is a log showing the time and date that a Blackberry handheld unit is synced. That log would give some indication of the volume of emails per unit.

  57. maryo2 says:

    The WH is always trying to change the framing of story. In this case there are two topics, but the WH keeps trying to act like there is only one topic. There is the topic of the emails – we want them and will have them regardless of where we have to go to find them. And there is the topic of the WH archiving process – is it criminally negligent or truly incompetent?

  58. bmaz says:

    Troll, do you try to be ignorant, or does it simply come naturally? Whatever your bag is, peddle it somewhere else. I was one of the few who voted to keep you; but I’m done. Go away and don’t come back.

  59. Neil says:

    Story hits the AP…

    Another private group suing the White House in the e-mail controversy sees the White House’s conduct as odd.

    ‘‘They changed their practice in 2003 to start saving the backups, but four-and-a-half years later they still have not yet figured out whether or what e-mails were deleted,’’ commented Meredith Fuchs, general counsel of the National Security Archive.

    ‘‘What has the White House been doing for two years?’’ asked Tom Blanton, director of the National Security Archive. ‘‘Two years after the special prosecutor publicly identified the problem, the White House still doesn’t seem to have a clue.’’ On the recycling issue, Fratto, the White House spokesman, said that despite the practice, some tapes should contain e-mails from before October 2003.

    ‘‘Of course the disaster recovery backup tapes were, at one time, recycled,’’ said Fratto. ‘‘However, since October 2003, the Office of Administration has retained and preserved its disaster recovery tapes. The disaster recovery system is set up to regularly back up everything on the network for the Executive Office of the President.’’

    CIA leak e-mail possibly erased
    January 16, 2008 FROM AP

  60. pseudonymousinnc says:

    ‘Industry best practices,’ indeed. Someone chez Atrios said that he kept tape backups for ten years in the Mississippi school system.

    Backup tape is cheap. The Presidential Records Act is clear. An ‘industry’ that ‘recycles’ its backup tapes is one working in the field of ass-coverage. If a Fortune 500 co. said this to investigators, people would be heading to jail.

Comments are closed.