Is Apparent US Conspiracy with Cisco about Wiretapping?

Canada has just discovered how much corporations own our legal system, how our legal system criminalizes whistleblowers, and our utter and total disdain for the rule of law.

At issue is the apparent conspiracy between Cisco and the US government to respond to an anti-trust lawsuit launched by Peter Alfred Adekeye, a former Cisco employee. He sued because of the way Cisco forced customers to buy a maintenance contract for things like bug fixes.

This lawsuit is about Cisco’s deliberate and continuing attempt to monopolize for itself (and its “partners” (Cisco-authorized resellers of Cisco equipment and services nationwide) with which it does not significantly compete) the service and maintenance of Cisco enterprise (Cisco networking equipment for all segments (e.g., internet service providers, government, academia, small, medium and large business, etc.) with the exception of home networking equipment) hardware, principally routers, switches and firewalls. Cisco possesses a market share of approximately 70% in the networking equipment industry.

[snip]

To protect its over $6 billion yearly stream of service and maintenance revenue, Cisco has cleverly and uniquely conditioned the provision of its software “updates” on the customer’s purchase of a hardware maintenance service agreement called “SMARTnet,”

[snip]

The effect of this leveraging of monopoly power and unlawful tie-in and/or bundling is to effectively preclude any non-Cisco affiliated Independent Service Organization (“ISO”) from competing for the business of servicing Cisco networking hardware, thus preserving for itself all but a pittance of that line of commerce which is separate and distinct from the “updates” of its software.

In response, Cisco counter-sued, accusing Adekeye of illegally accessing Cisco services. And Cisco either lied persuasively or got DOJ to conspire in the intimidation campaign, because DOJ then charged Adekeye with 97 violations that–the Canadian judge who just blew this up suggested–should have only amounted to one single violation.

The US also refused to allow Adekeye to enter the US after 2008, meaning he couldn’t testify in the litigation. Finally, in 2010, he flew to Canada to testify. At that point, the US had him arrested by the Mounties, based on false claims (among other things) that he was a shady Nigerian. He was held for four weeks, and then made to stay in Canada on restrictive bail conditions ever since as the US tried to have him extradited.

Justice [Ronald] McKinnon thought this case met the test and was flabbergasted by Adekeye’s “shocking” arrest during a judicial proceeding: “It is simply not done in a civilized jurisdiction that is bound by the rule of law.”

This was an egregious abuse of process and brought the administration of justice into disrepute, he concluded.

In his piece on this Sirota suggests that, if the US did conspire with Cisco, it probably did so in response to lobbying.

But I wonder if there’s not something more going on? Here’s how James Bamford described the government’s efforts to partner with Cisco on wiretapping in his book, The Shadow Factory.

One of the ways to covertly penetrate both the Internet and fiber-optic communications is to target their weakest point, the point where the systems interconnect–the routers.

[snip]

By discovering the weak spots and vulnerabilities of in this “postal service,” the NSA has the ability to target and intercept much of the electronic mail.

Thus, as [Deputy Director for Services Terry] Thompson further explained at the 1999 meeting, one of the NSA’s goals should be to hire away, on a short-term basis, people from key companies such as Cisco. Having hired them, the agency could use their knowledge and expertise to “reverse engineer” the systems and find ways to install back doors.

Just a gut level feel. If Adekeye’s initial suit hinted at something that played a key role in maintaining NSA’s access to all communications crossing Cisco’s routers, or if a successful suit would have made it harder to suck the worlds telecommunications off the network, that might explain the government’s seeming conspiracy with Cisco.

Alternately, maybe our government is just that fucking crazy.

Update: Here’s Cisco’s counterclaim against Adekeye. It claims, in part:

Adekeye, a former Cisco employee, founded Multiven in 2005. Under Adekeye’s direction, Multiven has, on multiple occasions, unlawfully accessed, downloaded, used, and distributed Cisco’s valuable proprietary information. Among other unlawful acts, Multiven improperly obtained a Cisco employee’s login credentials for password-protected areas of Cisco’s website. Adekeye and Multiven thereafter accessed these areas of Cisco’s website and, among other things, illegally downloaded Cisco’s copyrighted operating system software for use in its business and, on information and belief, for redistribution to others.

[snip]

During his five years as a Cisco employee, Adekeye acquired confidential inside knowledge regarding Cisco’s proprietary information, internal operations, security, and personnel. Adekeye’s employment with Cisco ended on May 6, 2005.

[snip]

Adekeye and Multiven used at least two improper means to learn about Cisco’s service techniques and proprietary information, including information regarding Cisco configurations and bug fixes. First, they illegally accessed Cisco’s password-protected website to view Cisco’s TAC services resources, some of which are contained in a database rich with technical guidance regarding network configurations and software-related information.

[snip]

Multiven and Adekeye concealed their illegal and otherwise improper conduct. As a result, Cisco, despite reasonable efforts and precautions, did not begin to discover Multiven’s and Aedkeye’s conduct until 2008.

Update: We don’t yet know when the criminal charges against Adekeye were filed. But as Mary noted to me via email, the current US Attorney in NDCA, Melinda Haag,a came from a firm–Orrick, Herrington & Sutcliffe–that does a lot of work for Cisco. Also, she was hired to be an AUSA by Robert Mueller.

Haag’s firm bio says she was recruited by then-U.S. Attorney Robert Mueller, now FBI Director, to join the Northern District office, based in San Francisco. (Mueller was U.S. Attorney there from 1998 to 2001.) Haag was chief of the office’s White Collar Crime Section, and in private practice, she has represented several corporate general counsels accused of improperly backdating stock options.

Update: Interesting. A portion of Adekeye’s May 20, 2010 deposition was entered into the civil suit docket on March 7, 2011, including the part where the Mounties come in and arrest him.

One of the things they appear to have been suggesting is that someone was paying Multiven to sue Cisco. In the unredacted parts, he is asked about individuals and/or corporations who funded the suit. But there’s a chunk redacted after that.

Update: Here’s a brief from Cisco objecting after Adekeye and Multiven suggested they had lured him to Canada to be arrested.

image_print
  1. MadDog says:

    …Just a gut level feel. If Adekeye’s initial suit hinted at something that played a key role in maintaining NSA’s access to all communications crossing Cisco’s routers, or if a successful suit would have made it harder to suck the worlds telecommunications off the network, that might explain the government’s seeming conspiracy with Cisco.

    Alternately, maybe our government is just that fucking crazy.

    Third choice? Both!

    • emptywheel says:

      Does my spidey sense suspicion make sense from a tech perspective, though?

      The idea being that the govt is doing something with bug fixes that helps it get our data. If other people were allowed a part of this maintenance business 1) it might mean they’d discover the NSA’s goodies and 2) it would make it harder to have near total access to the tubes (and therefore the data).

      So when someone threatens that monopoly legally, then the USG happily helps Cisco make sure he fails (after all, Cisco has the USG by the nuts, metaphorically).

      Heck, it’s even possible Adekeye’s intent was not just competitive.

      What do you think? Plausible?

      • MadDog says:

        More than just plausible!

        Cisco routers route almost the entire world’s Internet traffic. Think of that in relation to the traffic from…oh let’s say…China. Or Russia. Or Iran. And that’s just our putative “enemies”.

        We also do the same to our putative “friends” like France, and yes, even merry olde England.

        More than a few folks don’t realize the implications. Think of how our entire Foreign Policy is utterly dependent upon our National Security State and its tools.

        This is one of the reasons for the primacy of our National Security State thoughout our government. And the likeliest reason why no challenge to our National Security State, by friend or foe, will ever be tolerated.

        • emptywheel says:

          Note my update, with quotes from Cisco’s counterclaim.

          So Adekeye is hired in 2000. He moved to the US in 2003. He left in May 2005. And then allegedly starting breaking into Cisco’s databases in March 2006–so after the wiretap program was revealed.

          • MadDog says:

            Ta for the update!

            And the “happy ending” per the Vancouver Sun article:

            …Last August -two months after Adekeye was arrested -Cisco settled the suit and abandoned its previous service-contract practice.

            It dropped the allegation involving the illicit computer access.

            Justice McKinnon said he found it impossible to understand how a criminal charge now could be proven when Cisco, the supposed victim, says it didn’t suffer any loss…

            Sounds like effective greymail that again was meant to remove the entire issue from anyone’s radar.

        • mzchief says:

          The manufacturer zealously guard their switch technology (the 5ESS is the supposed de facto US Central Office standard by Y2k) as corporate secrets and so do the admins. Cisco is a big gorilla (many honor them with the Lucent salute) that got into the 5-tier telecomm equipment biz in edge devices in the late 1990s before the Dot Bomb fire sale/consolidation/lock-boxing. And, yes, Cisco’s position in the big network diagram is very strategic and in relation to all IP appliances.

      • MadDog says:

        …The idea being that the govt is doing something with bug fixes that helps it get our data. If other people were allowed a part of this maintenance business 1) it might mean they’d discover the NSA’s goodies and 2) it would make it harder to have near total access to the tubes (and therefore the data)…

        At the dawn of the computer age (hey, I was there *g*), updates, bug fixes, even new releases were provided in “source code” which meant the recipient could actually read in plain text what the update/bug fix was programmed to do.

        Rapidly after the dawn of the computer age (it wasn’t within nanoseconds, but nevertheless, happened quickly), computing vendors woke the fook up and realized that if they provided “source code”, they were making public their crown jewels. Their own fookin’ assets!

        Not bright. Not bright at all for profit-motivated corporations.

        Since that time, nobody in their right mind releases “source code”. Recipients get “object code“, and good luck trying to figure out what all those 1’s and 0’s are really doing/saying.

        If the US government were in bed with Cisco (highly probable), it would make mucho sense that code that ran in Cisco routers was “modified” to meet the needs of the US National Security State.

      • marksb says:

        This is a case of both hands scratching each other’s backs. Or something like that. I used to work in this field, TCP/IP packet routing….

        Cisco (and most other network and telecom equipment companies) use service contracts to enable updates to code and features. This is almost SOP in the industry. (At least it was seven or so years ago and there’s zero reason for it to have changed.) It’s a nice predicable revenue stream, and provides the sales guy with the chance to get in there regularly so they can sell the next system.

        And Cisco owns the code to almost all the core routing equipment throughout the world, giving it the unique ability to provide “back doors” to data access. This is complex code that controls every aspect of packet analysis and forwarding, doing it incredibly well and incredibly fast. The people who started Cisco basically wrote much of the core TCP/IP code that started all packet networks.

        It would be very difficult to detect the traffic leaving a router and determine what is going where–unless you installed a very expensive analysis machine on the pipe out of the router, and why would you do that? You get nothing out of it. If you are in the business of providing pipes or running traffic, you would have no reason to spend substantial money on additional analysis gear outside your routing system.

        Network providers buy from Cisco because, well, it works, it’s reliable, if there’s any problem at all they fix it *now*, and, in the words of every industry sales guy, “no one ever got fired for buying Cisco”.

        So it’s a dream come true for a government wanting to suck up packet traffic. The whole world of packet traffic is available if you can find a way of convincing the company that owns the code to help you, and there’s (almost) no way anyone is going to notice that you’re doing so.

        To convince them the Intel guys could use a combination of one-hand-shakes-the-other lobbying, revenue streams for the data, contracts for large government projects, blacker systems, and military pipes and data centers, “lawful intercept” legal arguments, and old-fashioned political pressure.

        I think that you are on to something here, EW, in spite of that nagging feeling of paranoia in the back of your mind…

        • PeasantParty says:

          EW has that nagging feeling and I have the same feeling only really in front of my mind!

          Now that brings me back to the 24/7 surveilance and a link to connections. Doesn’t that also mean they can hear you thru your cell phone, anywhere, in the car, standing in the middle of a forest?

          • marksb says:

            “They” can hear all of us, all the time, on any and all of our devices. The turning point happened around four or five years ago with new technology and specific government-funded analysis projects to run on this new super-fast hardware. Our constant-war mentality provided the money and the legal cover.

            It ain’t really paranoia if they really are trying to get us. Heh. That farm out in the middle of nowhere, the one without a network connection, is looking kinda good, isn’t it?

    • MadDog says:

      Too easy I think. Being inside Cisco routers sounds more like effective strongarming of a putative “US” technology company. Perhaps Cisco even went along willingly.

      The Thomas Drake/NSA technology question still seems to me to be something the government was breaking into rather getting the vendor’s assistance.

  2. Mary says:

    Not so much related to your spec, but just a comment on Obama’s DOJ, this part from the Vancouver Sun article that you linked above made me shake my head.

    Last August -two months after Adekeye was arrested -Cisco settled the suit and abandoned its previous service-contract practice.

    It dropped the allegation involving the illicit computer access.

    So the “injured party” with respect to the 97, or 1, claim(s) settled last year and dropped the civil claim that all the criminal charges were based upon in connection with that settlement.

    Justice McKinnon said he found it impossible to understand how a criminal charge now could be proven when Cisco, the supposed victim, says it didn’t suffer any loss.

    Sheez. Do we know for sure which US Attorney sent the letter to Canada with the allegations? The other thing that floored me is that, after all this – having the guy jailed, requesting that Canada exercise emergency powers because of the threat he supposedly represented, then watching the US civil suit be settled and the counter-plaintiff “nevermind” its allegations of wrongdoing against the guy, after causing Canada to spend oodles of time and resources on the jailing and the legal proceedings and after sending a judge ballistic up there, after watching the guy leave Canada for Zurich – DOJ’s only comment on the whole mess was that they have 30 days to appeal.

    • MadDog says:

      …DOJ’s only comment on the whole mess was that they have 30 days to appeal.

      I read that part the same as you, but then I wondered if “Justice Department” at the end of the article was Canada’s Department of Justice rather than the US DOJ.

      It’s hard to tell, and perhaps makes no difference anyways. A DOJ is a DOJ is a DOJ. *g*

      • Mary says:

        Eh -you’re right I think. The comments on the case came from the Canadian prosecutor and that’s probably right procedurally too (that it’s the CNDOJ)

              • MadDog says:

                And that reminds me of a embarassing experience I once had.

                I was in Zurich for a meeting to give a computer presentation to about a dozen of my European counterparts. I was one of the best presenters in my company (actually, the best if I say so myself *g*).

                My European counterparts were from Switzerland (German and French speaking cantons), Germany, France, Britain, Italy, the Netherlands, and so on. The European counterparts all spoke English as a second or third language most excellently and were all computer folks as well.

                I was rambling along and after stopping for a break, one of my European counterparts hesitantly broached his concern that he was having trouble understanding my English. The accent was throwing him. The rest of my European counterparts all nodded agreement with him.

                Me? I was stunned. I’ve got an accent? Me with a my upper Midwest English that is chosen by all the national TV networks because we have no accent? How could this be?

                I don’t have a Texas twang or a Georgia drawl, or even Boston brogue. How could this be?

                The lesson I learned is that we all have accents. And when making fun of someone else’s accent, it’s wise to remember that we too are the butt of someone else’s accent joke.

                • Kelly Canfield says:

                  European second language English speakers would learn English, mostly, with a British accent.

                  That is why your Midwest came off with a hitch (twang I imagine.)

                  I’ve always wondered how the British think my American English sounds. I’ve never really been able to fully wrap my head around that. I certainly can’t describe, much less reproduce, Cockney.

                  • MadDog says:

                    What was most disconcerting was that I had a second upper Midwestern American colleague with me, and my European counterparts had no problem with his accent.

                    Just shows to go ya’. *g*

                  • dopeyo says:

                    OT, but the funniest thing i ever heard was a lovely redheaded belle from alabama, imitating how midwesterners speak. must be all the corn.

            • Mary says:

              Comes from having relatives in MN. Cannuckian can even eventually diminish Kentuckian. I’d never have thunk it.

  3. MadDog says:

    Keep those updates coming EW! Mighty good stuff!

    And while the MSM happily plays with Weiner’s weiner, you might be uncovering a real bombshell of a story here. As big as the NYT’s NSA illegal wiretapping story!

    There is far more here than meets the eye. The stuff you’ve gotten thusfar may just be the tip of a very large National Security State iceberg.

  4. mzchief says:

    Canada has just discovered how much corporations own our legal system, how our legal system criminalizes whistleblowers, and our utter and total disdain for the rule of law.

    Well actually Canada (see VSNL International Canada [aka Teleglobe and Teleglobe USA was HQ’d in Reston, VA) had a role to play in setting this up as did the UK (British Telecomm) along with Orbital Sciences Corporation (bird shake ‘n bake next to Dulles, VA along with AT&T and )and Hughes. You also have to look at ill-fated Iridium Communications Inc.. All these are many of the top players pre-Y2k. Also note who’s on the I-270 corridor from Germantown, MD to Dulles, VA and out the Dulles Greenway.

  5. GulfCoastPirate says:

    Well, is there any doubt any longer that our government is that ‘fucking crazy’?

    I manage quite a few routers and associated ‘Smart-Net’ contracts. No one is forced to buy one. They not only cover software updates they also cover hardware failure of the router. The cost of the contract is related to the response time frame. For instance, I have a router somewhere at a remote location in Bumfuck, Mississippi and I manage it remotely. If it goes down and I want a new router at that location within a certain number of hours I pay more than if I want it there the NBD, 2 days or whatever. There are reasons for the contracts that have nothing to do with software. In fact, of all the reasons to have a contract, software updates are one of the last. I never update software on a working router unless there is a new feature that is advantageous for some reason. It’s that old ‘if it ain’t broke, don’t fix it’ reasoning.

    Also remember, routers mostly work on wires – if we consider the telco equipment brought to the customer’s site as part of a ‘wire’ between locations. In order for the government to siphon all the data for examination they would have to have an agreement with ‘someone’ that all data ultimtely passes through a single point (or multiple single points they are monitoring) and is then transmitted to its final destination. I believe you folks have talked about the old lawsuit in SF where they were supposedly doing exactly that with overseas traffic so I suppose it is possible but in order to do so there would have to be a lot of collusion.

    I remember working on an experiment back in the late 80’s/early 90’s at Brookhaven when the Internet was mostly a function of the university related scientific (physics) community and we would talk at night about exactly what governments could do in the future. IIRC Netscape wasn’t even out at the time but the rumors were out there and we knew eventually it was going to be brought to the masses. There was too much possible business potential for it not to be. It’s somewhat eerie to see some of this possibly coming true; however, in order for it to happen there has to be massive collusion between government and others.

    In case anyone doesn’t know and wants to trace the route a particular packet takes to a destination bring up a commmand prompt and type in:

    tracert xxx.xxx.xxx.xxx

    where the ‘x”s are the ip address you are trying to reach. It will tell you what routers your packet goes through to reach its destination. If you type in the same thing a second time you may take an entirely different route to the same destination. This means a couple of things.

    1. The software would have to be programmed to make sure the ‘collection point’ wasn’t announced in the traceroute. Otherwise, you would be able to tell that no matter which route your packet took it always passed through one particular point.

    2. There is a lot of collection going on at multiple locations. While it may be somewhat easy to make sure all traffic from overseas enters the country at one point I think it would be much more difficult to make sure all inter-country traffic goes through a collection point.

    3. The telcos would have to have some very smart people located in some back room that, to date, I’ve never come into contact with and I’ve spent hours on the phone with them over the years and solved many of their problems they couldn’t seem to solve themselves. :)

    I’m not saying it isn’t going on but I am somewhat skeptical.

    • MadDog says:

      A couple points in response to your excellent comment:

      1. The more I think about it, the less I believe that Cisco routers have been hijacked by the US National Security State to perform “collection” or “surveillance” stuff.

      Routers are for the most part pretty dumb boxes. They have very few tasks to perform (looking at packets and routing them). Addressing stuff mostly. They just need to do this quickly for a lot of packets.

      On the other hand, the Narus technology (now owned by Boeing) is optimized to scan within those packets for far more than destination IP addresses. Narus is looking within packets for text triggers.

      The Narus is a “surveillance” or “collection” platform by design while a Cisco router is not.

      2. If the above is true, then I’m leaning toward the Cisco/US National Security State relationship being more about using Cisco routers as “offensive/defensive weapons” rather than “surveillance platforms”.

      By that I mean that it wouldn’t surprise me that the US National Security State would be using the Cisco router technology to be able to block routing or shut down routing.

      Like shutting down China’s ability to communicate over the Net. Or blocking a state-sponsored or terrorist cyberhacking attack. This would be more in keeping with the capabilities of routers including Cisco routers.

      • marksb says:

        Excellent comment. While routers are not dumb boxes, they are designed to do one thing fast and well–route traffic–and not do Real Time analysis. It’s tough enough to route significant traffic at the speeds today’s networks demand.

        But you’ve got to obtain the packet stream to feed the Narus boxes, and there’s where the Cisco relationship would be invaluable. If Cisco routing systems are setup to send a mirrored stream to a secure location where a Narus system(s) is(are) doing analysis, there is no need to “bother” the actual network providers–they don’t know, and won’t know. Better to ask forgiveness than permission, right? (And lawful intercept via Patriot Act can cover various asses if anyone ever notices.)

        Brilliant, eh?

        • PeasantParty says:

          (And lawful intercept via Patriot Act can cover various asses if anyone ever notices.) Brilliant, eh?

          Absolutely! Especially when the lawful parts of that are known secrets, that we aren’t supposed to know what they entail.

        • MadDog says:

          …If Cisco routing systems are setup to send a mirrored stream to a secure location where a Narus system(s) is(are) doing analysis, there is no need to “bother” the actual network providers–they don’t know, and won’t know…

          It could be done that way though what AT&T did in San Francisco (and apparently multiple other AT&T sites) was simply use a Splitter (see Klein’s declaration here – 7 page PDF) where the fiber cable was simply split and a duplicate copy of its traffic sent to a Narus for total capture and deep packet inspection.

          Again, I’m still leaning toward the use of Cisco routers for stopping or blocking Net traffic via US National Security State commands. That would seem to be well within the native capabilities of the Cisco software itself.

          • marksb says:

            I’ll buy that. Probably a combo. If I were involved in the project I’d be pushing for All Of The Above. I don’t want the Narus box at the split location, so I want to secretly route that mirrored traffic to the analysis location.

        • MadDog says:

          …While routers are not dumb boxes, they are designed to do one thing fast and well–route traffic–and not do Real Time analysis. It’s tough enough to route significant traffic at the speeds today’s networks demand…

          I wanted to come back to this point. In using the phrase “dumb boxes”, I made a poor choice of words. A better choice would have been “simple, limited set of functions.”

          As you stated, routers are designed and optimized in both their hardware and software to do a relatively simple set of functions: Move packets in and move packets out.

          Their hardware and software is not designed to lollygag around doing deep packet inspection for text triggers, and then writing their text trigger captures to the comparatively molasses-slow hard drive.

          The Narus system, in contrast to routers, is designed to do exactly that deep packet inspection for text triggers, and leisurely write what it captures to its hard drive.

          The Narus system doesn’t have to worry about 1/2 the workload that routers do. The Narus system isn’t in the business of “moving packets out”.

          Again, this is one of the reasons that I doubt that Cisco routers are being used for “collection” or “surveillance”. Too big of a task for a system that is optimized for “in quick and out quick” processing.

          • GulfCoastPirate says:

            I agree with all this. If they’re doing it then they have a way to siphon (or duplicate) the traffic and move it elsewhere for analysis. How they are doing that is the big question.

      • GulfCoastPirate says:

        1. In how many different places do you think this ‘Narus’ technology has been placed? No matter what type of technology you have there is still the problem of routing it to central locations to be ‘monitored and inspected’. How many people would have to be involved?

        2. Well, I think they can block traffic coming ‘into’ the country but I’m still a little skeptical of their ability to block traffic outside the country. Again, I’m trying to get it though my head how many people this would involve and how they would keep it quiet.

        • MadDog says:

          1. In how many different places do you think this ‘Narus’ technology has been placed? No matter what type of technology you have there is still the problem of routing it to central locations to be ‘monitored and inspected’. How many people would have to be involved?

          Good questions! Based on that Mark Klein declaration (7 page PDF), it seemed like AT&T allowed the NSA to install Narus systems in a number of AT&T’s sites like Seattle, Los Angeles, San Jose, San Diego and others.

          As to number of AT&T people in the loop? Just a SWAG here but a couple dozen max?

          …2. Well, I think they can block traffic coming ‘into’ the country but I’m still a little skeptical of their ability to block traffic outside the country. Again, I’m trying to get it though my head how many people this would involve and how they would keep it quiet.

          I’m thinking that the stopping or blocking of Net traffic from a country would be a weapon only used in a war-like situation.

          For example, I’m betting that the majority, if not all, of the Net traffic exiting China is through Cisco routers. Even Cisco routers physically in China.

          If the US National Security State had the ability to command Cisco routers per my speculation, they would be able to shutdown China’s ability to communicate via the Net.

          Ditto for any other foreign country where Cisco routers are located. And since Cisco routers route the world’s Net traffic, they are physically located all over the world.

          • GulfCoastPirate says:

            1. Hmmm …. could be but only a couple of dozen? How will one of those ‘in the know’ always know when a router that may contain the ‘Narus’ is going to be replaced by some local technicians? Maybe since I’m on the more independent side of the business I’m not fully cognizant of exactly how much control there is in the larger corporations.

            2. Are the Chinese that stupid? Not only that, but would a corporation the size of Cisco take a chance on relying on the US government when all their manufacturing operations actually take place in Asia?

            • MadDog says:

              1. Hmmm…. could be but only a couple of dozen? How will one of those ‘in the know’ always know when a router that may contain the ‘Narus’ is going to be replaced by some local technicians? Maybe since I’m on the more independent side of the business I’m not fully cognizant of exactly how much control there is in the larger corporations.

              I spent a lot of years working with AT&T (before and after the breakup). The folks at AT&T have been in bed with the US National Security State since before World War II. Deeply in bed!

              If you read Mark Klein’s declaration, you’ll find that AT&T put a secret secure room together in their San Francisco location. A room that could only be entered by “duly authorized personel”. Such authorization to be had only from cleared AT&T management and the NSA.

              If you didn’t have the “need to know”, you’d not know what AT&T and the NSA had squirreled away in that room.

              2. Are the Chinese that stupid? Not only that, but would a corporation the size of Cisco take a chance on relying on the US government when all their manufacturing operations actually take place in Asia?

              What choice does China have? They don’t make routers themselves. As for Cisco, they are either willing or unwilling partners with the US National Security State, but partners they are!

              Like many nominally US corporations, when the US National Security State says you’re going to do this. The easy way or the hard way, but you are going to this, there are very few rare instances where corporations successfully “declined”.

    • bmaz says:

      Answer to your initial question – hell no.

      But more importantly, long time no see! I look forward to you being around here when we crank up for the Clemens trial here after the 4th of July. And we are very quietly optimistic about at least a decent year for the Devils next fall. I think they have settled on Orsweiller at QB, a big kid with a ton of raw talent; hopefully he can get the ball to Knust downfield some.

      • GulfCoastPirate says:

        bmaz wrote:

        ‘Answer to your initial question – hell no.

        LOL.

        But more importantly, long time no see! I look forward to you being around here when we crank up for the Clemens trial here after the 4th of July. And we are very quietly optimistic about at least a decent year for the Devils next fall. I think they have settled on Orsweiller at QB, a big kid with a ton of raw talent; hopefully he can get the ball to Knust downfield some.’

        I’ve been around and try to keep up but you guys go so fast that if you aren’t there at the immediate moment you’re off on another subject. I just happened to hit this one at the right time.

        They were talking about Clemens on the radio the other afternoon and I thought about you. A couple of the radio guys seem to think he’s getting railroaded.

        Does it look like Randy’s going to get to play? That would be good news. I haven’t seen his dad since the last golf tournament last fall so I haven’t been keeping up with his progress. We get the UCLA Bruins first game this year with Keenum back so maybe we can get them off on a losing streak for all you folks that get them later.

        How about the news about USC this afternoon? Then Ohio State? Plus, the college presidents look like they had enough of guys like Saban and Spurrier screwing those kids around and limited their scholarships. Are there actual, enforcable rules coming to the NCAA? Naw – no choice. Wake me up when they throw the book at the evil empire (UT).

        • bmaz says:

          Well, it is hard to follow bout now, and I didn’t pay very good attention during spring ball. Last i heard, he was expected to get some good looks, but likely not start. They are supposedly spreading out and opening up the offense for next year, which from what I gather could play into his strengths pretty well. Still a red shirt frosh though.

          • GulfCoastPirate says:

            I’m not sure that frosh designation means all that much if he’s familiar with the spread. Down here, lots of the high schools are going to it so when the kids get to college they are already familiar with the basic idea and just have to adjust to the increased speed of the college game. I think we’ll increasingly see young quarterbacks doing well as it continues to spread through the high schools.

    • marksb says:

      Hmm. Should have read your missive before, GCP.

      I’ve been thinking I could “mirror” the traffic stream, isolated from the packet stream you are analyzing. I could do this with an optical splitter on the output of the routing system, than pop another router on it to send it off to my blacker net and the analysis systems. (Nearly all pipes of any decent size are optical these days.) (This is the way I’ve envisioned it before when we’ve discussed data capture on EW’s blog.)

      Or I could split it internal to the router box, providing a “virtual” splitter that again is isolated from the main stream of packet traffic.

      Either way I don’t think anyone is going to see the split traffic, mirrored traffic, in the main packet stream.

      Make sense to you?

      • MadDog says:

        There is oodles of black fiber (“unused” – supposedly black) that could easily be used as a black transport (different kind of black – National Security State black) for shipping the traffic directly to central NSA computers.

      • GulfCoastPirate says:

        Where would you do this? I’m on Galveston Bay. Bmaz is in Arizona. Suppose we each wanted to send a packet to the same destination in Santa Fe. Unless you had splitter points all over the country or, you were funneling everything through a central location, our packets are each traveling separate paths that probably don’t come anywhere near each other. Could it be set up? Sure, but that’s a pretty big assed conspiracy that would involve a LOT of external people. If you tried to do it by controlling a central location and funneling out from there then Bmaz and I should each be able to trace our packets through the centralized routers where you are doing your splitting unless the IOS software in the CIsco routers is corrupted by the government.

        I’m not saying they aren’t doing it – I’m just trying to figure out based on my limited knowledge HOW they are doing it.

        • MadDog says:

          Where would you do this?

          Ahhhh! So this is what is holding you back. *g*

          You would be right in your analysis if we were talking about capturing all domestic US traffic. That would require the oodles and oodles of capture points all over the US.

          In the case of International Net traffic that transits the US, that is a horse of a different color.

          There are orders of magnitude less capture points for International Net traffic that hits the US ((see Mark Klein’s declaration here – 7 page PDF). That San Francisco AT&T office is one of the major hubs.

          In regard to International Net traffic that does not transit the US, while the NSA may have ability to capture traffic with the cooperation of certain friendly nations (Britain, Australia, etc.), the NSA has less capability these days with regard to fiber-based traffic overseas where they can’t work a deal or steal access to hubs.

          In addition, another factor to throw into the mix is that many of the email providers such as Google, Microsoft, Yahoo, etc. are US-based, so that if scanning emails is a high priority for the NSA, these US-based email providers are likely partners with the US National Security State.

          • GulfCoastPirate says:

            No, I agree about the international traffic. That would be much easier because of fewer entry/exit points. I was referring to domestic traffic.

            • MadDog says:

              Yeah, I don’t think the US National Security State is doing all the US domestic traffic…yet. *g*

              It is technically feasible, and perhaps even financially feasible, but if proof ever surfaced, I’d like to think that both Telco folks and US National Security State busybodies would be heading to jail.

              But give it a few years and it might just be politically acceptable. Many in this country already support the notion that “if you’re not doing anything wrong, you won’t have a problem.”

  6. PeasantParty says:

    MARCY!

    Zoom! Scoop! hoooWeee!

    Could this/these revelations be part of that UnPatriotic Acts of known unknown secrets of Super Special Secret Laws that are so top secret the people under those laws are not to know about the known unknown super secrety secrets?

    If every single line of every kind is being hacked, er, uh, listened to then, oh heck what about all those Corporate secrets that may be floating across fax lines?

  7. marksb says:

    Old-school ASCII diagrams…

    ==traffic in==>router box===>traffic out to next router
    same router===>copied traffic to analysis servers (blacker net)

    So Cisco is vital unless you are planning to demand access from every provider, with the chance that a few of those demands become public (remember the AT&T secret room)

  8. PeasantParty says:

    Now I am thinking about all those critters sitting on the Intelligence Committee. Marcy did some excellent reporting on the few members that were arguing for this last Patriot Act re-up. I’m ready to see which ones are up for re-elect!

  9. PeasantParty says:

    Doesn’t Canada have laws for the news agencies stating they cannot lie in their reporting? I believe they do, so the info Marcy has from Canada has to be as close to truth as they can get/publish.

  10. BayStateLibrul says:

    Bruins 4 Canucks zippo (end of second)…

    Canucks sent Horton to Mass General… pricks

  11. GulfCoastPirate says:

    I’ve read about the SF case – in fact, found most of the info I found interesting around here. :)

  12. bobschacht says:

    I’m posting before reading the comments, so someone else may have noticed, but…

    EW, in the first paragraph of the first quote block, there are no less than two sets of nested parentheses! Now, mathematicians are used to that, but it’s considered in poor taste among writers. Normally, the thing to do is to use brackets for one pair of open and closing parentheses, and one pair of normal parentheses for the other. But since this is a quote, I assume that the double set of nested parentheses was in the original, right? I had to reread that paragraph several times before understanding the nested parentheses appropriately.

    Bob in AZ

  13. bobschacht says:

    OK, my last comment sat for half an hour while I read through the comments, before I sent it. But now that I have, I’d like to ask all these tech-savvy commenters:

    If you were the U.S. gov. and were absolutely determined to break down Wikileaks and destroy it, ISTM the tools you’re discussing here would be very useful, no? One of the few things we know about Wikileaks is that it’s international, right?
    Has the gov’t been tracing back info from Wikileaks to find the source(s)? And is Assange still under house arrest? How much you wanna bet that everything Assange reads or writes on the Internet is subjected to intense analysis, including tracing?

    In the face of this technology, how can Wikileaks continue to hide?– or function?

    Thanks,
    Bob in AZ

    • GulfCoastPirate says:

      Has the gov’t been tracing back info from Wikileaks to find the source(s)? And is Assange still under house arrest? How much you wanna bet that everything Assange reads or writes on the Internet is subjected to intense analysis, including tracing?

      In the face of this technology, how can Wikileaks continue to hide?– or function?

      I think at this point they have Assange himself under constant surveillance. It would be fairly easy for them to monitor one person.

      As for Wikileaks that’s a little different but I’m sure they’re working on it. Suppose you are in Spain sending a packet to someone in Sweden. Normally, there would be no reason for that packet to travel through the US where it could be monitored but whether or not the NSA has control over foreign corporations like they do the US telcos is unknown to me.

  14. emptywheel says:

    Hey there

    Reading some more of the filings in the civil suit. We may not be able to figure out whether there’s any there there until the Canadian court releases more.

    It does sound like Multiven is a fly-by-night organization. Adekeye was denied entry to the US because his H1B was revoked (not sure yet whether that’s bc the USG decided Multiven was too fly-by-night to offer H1Bs or bc it was acquired from Cisco). And in his depo (on May 19, so the day before teh arrest–I’m wondering whether it was part of the evidence the govt used for the arrest warrant–Adekeye did admit to accessing Cisco’s system (though he claims the Cisco employee offered up his access).

    So it may well be what Cisco claims it is: a stupid suit, followed by a bullying counterclaim suit that Cisco was able to interest the Feds in.

    The big question there would be why the Canadian Judge is so upset. It may be just that he was pissed about the depo as a means to get Adekeye to Canada. But he also says the USA lied in the arrest warrant.

    One more thing: it seems they make a big deal of all the foreigners involved in Multiven. I still think they might be suggesting it was a front. If that’s right this might be as much about the US protecting Cisco’s dominance in the market as it is anything else.

    • GulfCoastPirate says:

      One more thing: it seems they make a big deal of all the foreigners involved in Multiven. I still think they might be suggesting it was a front. If that’s right this might be as much about the US protecting Cisco’s dominance in the market as it is anything else.

      If Cisco has certain code hidden in their IOS that is advantageous to US authorities wouldn’t it be logical for them to do this?

    • marksb says:

      …this might be as much about the US protecting Cisco’s dominance in the market as it is anything else.

      Yeah but I’m now aware of the potential to elegantly (and perhaps legally) solve the problem of acquiring the raw packet stream for all traffic on a pipe for further analysis at one of the big-ol’ Intel server farms. It’s easy, almost undetectable, it means all the providers can claim ignorance if it every blows up in their faces, and it might be legal.

      They’d be fools, mission-wise, if they didn’t go down this path…

  15. marksb says:

    (Back to this tread…)

    Assuming we have no idea if there’s a there, there, IRT Cisco collusion with any Intel agency;

    The point I want to make is that if a router was modified to mirror packet flow to an off-site location, almost no one would know, unless they specifically went to look.

    When a router receives a packet, it looks to the header, analyzes the source and terminal (eventual) destination, looks up the “best” next-gateway (router) to most efficiently get to the destination, re-wraps the packet with the next destination, and sends it out. Nobody anywhere other than the next router gateway sees that packet (unless you monitor the optic pipe along the way, but that’s another story).

    So you have the ability, in theory, to send a packet stream off to another destination without anyone/anywhere seeing that happen. It’s like I have two telephone lines, and I hook my phone to both, dial a number on both lines, and talk into the one phone: both lines get the message, but nobody on either line knows the other is privy to the message. It really could be independently mirrored.

    (It’s been a good number of years and my terms are rusty, but TCP functionality hasn’t changed significantly in a long time)

    • GulfCoastPirate says:

      Again, we get back to how many routers would have to be modified – a LOT. I would certainly know if they attempted to do this to any of my routers. The only way I can see this would work would be if all traffic is funneled through a few centralized locations. Those locations would have to be owned by the telcos and they would have to be in on the collusion – not to mention kept secret.

      Is this even legal?

      • ondelette says:

        It’s debatable. It isn’t legal for criminal purposes but it might be for national security purposes, because the government holds the view that they haven’t violated anyone’s rights under minimization rules until they’ve actually looked at the data.

        • marksb says:

          Geez. And if they are looking at the data at a secure Intel center, no one’s ever going to know. What a loophole. Plus isn’t the rule post-Patriot Act something along the lines of security measures to prevent terrorism trumps privacy, and if nothing is found, no harm/no foul?

  16. bobschacht says:

    There used to be talk of an “Internet backbone.” Is that metaphor obsolete now? i.e., has the network become more decentralized, or does a large amount of the national traffic follow certain pathways?

    Bob in AZ

    • marksb says:

      The “backbone” has been a virtual terms for many years. It describes a virtual series of pipes through a lot of different providers that makes up the path that the packets take to get from here to there.

    • GulfCoastPirate says:

      It’s mostly decentralized. If you Google ‘Internet Backbone’ you can find maps of the major exchange/access points.

  17. ondelette says:

    I would suspect that Cisco machines that were actually used by the NSA to collect information were pretty specialized and centralized, given that the NSA “hoovers” all traffic and then sorts. So odds are that Adekeye isn’t routinely working on such machines or maybe never works on such machines. Keep in mind, though that under CALEA, all the machines that are for internet service providers are supposed to be ready to accept taps from the government for criminal investigations, so this is undoubtedly written into the Cisco software at least “in principle.”

    The spooks put together and ran a TIA-style gathering system in Singapore in 2007 after being told not to do so here, and submerging the system here into about 50 start-ups and contractors. One can assume that’s in place here now.

    I would suspect this Cisco case is lobbying and ordinary corruption, an attempt to get the government to enforce their monopolistic and predatory business practices.

  18. marksb says:

    GCP, I understand it’s legal, under lawful intercept rules, though how legal is another whole ball of wax. Once (~2000) you had to show a warrant, now I think the whole thing can be a secret notification, and I think the notification can cover a general search (“hoover”, I love that image) over a period of time. Further, if you ran a major access or backbone provider site, you might just be told that this is a required blade on the system, and it might feed a separate OC-192 line…

    Ondlette, our equipment (major telecom equipment vendor) was designed to allow lawful intercept with a console command. Every major bid we responded to from the mid-90’s to 2002 (when I left) required this “feature”.

  19. robspierre says:

    My TCP/IP knowledge is modest and really old now, but I’d like to amplify on Ondelette’s comment about the traffic. The same thought occurred to me: if the switch is duplicating the data stream and then resending the copy on a separate path to a new, secret address, you have to do twice the work with the same number of switching resources. This would be visible as a performance hit.

    The basic problem is that, for this scheme to work, the outbound traffic through the switch would have to be double the inbound traffic. Bandwidth on the downstream side of the switch would thus be half that on the upstream side. That seems like it would be hard to conceal.

    Worse, if we believe that duplication and rerouting is a general feature of all switches (rather than of targeted, central switches), then it seems to me that the bandwidth losses would increase exponentially. Each hop would duplicate the traffic duplicated on the previous hop. Given the nature of TCP/IP as I understand it and the conceptually simple, highly optimized nature of the equipment, a given switch wouldn’t have any way of knowing that some other switch had already cloned a given bunch of packets. Every switch would have to clone every packet it got. So, while the traffic for the node would double as Ondellette points out, the traffic on the network would increase by 2 squared for 2 hops, 2 cubed for three, and so on. The numbers could get big fast. I’d think that the internet would slow down so much that everyone would notice.

    Finally, I have to wonder whether a broadcast attack on the whole internet would yield any intelligence at all. Clever targeting is usually what makes data analysis worthwhile–you start with an idea of what you should find and where, so you do not waste so much time chasing irrelevancies that you miss the one real data point. If this scheme were implemented, the sheer volume of totally uninteresting packets would baffle the watchers and hide the real evidence, if any. Effectively, the Security state would be spamming itself into complete paralysis. This is why I suspect that what broadcast surveillance is being done is being done as cover for illicit, targeted surveillance, such as the discrediting of undesirable politicians or the harvesting of insider stock information. You could hide your targeted, criminal operations under the futile broadcst program. Finding out that a well-known governor and former prosecutor of corporations employed prostitutes would be as easy as data-mining all the internet’s packets would be hard: you know his name, his ISP, and probably his IP address in advance.

    • ondelette says:

      The solution is to optically tap a copy and assemble the packets yourself from a trunk line that is likely to contain a lot of relevant packets for a different reason. Hence the Narus machines on the trunk lines for the pacific cables, for instance (and at other points). Beam splitting doesn’t cause extra traffic in the duplication, and the Narus machines reconstruct the traffic directly from optical. Since Asia wants to mirror a lot of U.S. web sites at edge servers and local mirrors, problem solved, even for most email. That is now a much smaller problem there are only a few big carriers for the majority of the people, and they mirror their traffic on websites as well. I can go to China and access an email from one of my ISPs on their website which undoubtedly is carried by a mirror. Which means it was transmitted over the trans-pacific cable at some point.

      • marksb says:

        That’s what we figured during out last discussion on this sort of thing; split the optical feed and use private optical net to get to the Narus farm.
        I think the whole Cisco thing sort of threw me off…I could see how it would work, adding blades and running copies of the packet traffic, but there would be significant traffic load impact. Of course, if it was all to the next-gateway that removed it to a private optical pipe, then it would be less impact…
        All speculation. But I did dream a bit over the top here.

        robspierre’s thoughts are interesting that maybe the router has a reference table with target IP source addr’s and that traffic gets fwd.