The Other Servers and Laptops FBI Never Investigated: VR Systems and North Carolina Polling Books

Ron Wyden had a lot to say in his minority views to the SSCI Report on election security released yesterday, mostly arguing that there need to be national standards and assistance and that no one can make any conclusions about the effects of Russia’s efforts in 2016 because no one collected the data to make such conclusions.

But there’s one line in his section raising questions about the 2016 conclusions I find particularly interesting, pertaining to VR Systems (which he doesn’t name).

Assessments about Russian attacks on the administration of elections are also complicated by newly public information about the infiltration of an election technology company.

Since the Mueller Report came out, Wyden has been trying to chase down this reference in the report to the VR Systems hack.

Unit 74455 also sent spear-phishing emails to public officials involved in election administration and personnel a~ involved in voting technology. In August 2016, GRU officers targeted employees of [redacted; VR Systems], a voting technology company that developed software used by numerous U.S. counties to manage voter rolls, and installed malware on the company network.

In May, he sent a letter to VR Systems President Mindy Perkins, asking how the company could claim, in March 2018, that it had not experienced a security breach when the report said it had been infected with malware in August 2016. In response, the company told Wyden (according to a letter he and Amy Klobuchar sent FBI Director Chris Wray) that they had alerted the FBI that they found suspicious IPs in their logs in real time, but that FBI had never explained the significance of that.

In a May 16, 2019, letter to Senator Wyden, VR Systems described how it participated in an August 2016 conference call with law enforcement. Participants in that call were apparently asked by the FBI to “be on the lookout for certain suspicious IP addresses.” According to VR Systems, the company examined its website logs, “found that several of the IP addresses had, in fact, visited our website” and as a result, the company “notified the FBI as we had been directed to do.” VR Systems indicates they did not know that these IP addresses were part of a larger pattern until 2017, which suggests the FBI may not have followed up with VR Systems in 2016 about the nature of the threat they faced.

The implication from Wyden’s letters is that VR Systems only hired FireEye to conduct an assessment of what happened after Reality Winner leaked an NSA document making it clear they had been targeted by GRU in 2017. [Update: Kim Zetter actually reported this here.]

In their June 12 letter, Wyden and Klobuchar asked Wray whether the FBI followed up on VR Systems’ report.

  1. What steps, if any, did the FBI take to examine VR Systems’ servers for evidence of a successful cyber breach after the company alerted the FBI, in August of 2016, to the presence of suspicious IP addresses in its website logs? If the FBI did not examine VR Systems’ servers or request access to those servers, please explain why.
  2. Several months after VR Systems first contacted the FBI, electronic pollbooks made by the company malfunctioned during the November 8 general election in Durham County, North Carolina. In the two and a half years since that incident in Durham County, has the FBI requested access to the pollbooks that malfunctioned, and the computers used to configure them, in order to examine them for evidence of hacking? If not, please explain why.
  3. VR Systems contracted FireEye to perform a forensic examination of its systems in the summer of 2017. Has the FBI reviewed FireEye’s conclusions? If so, what were its key findings?

It’s unclear how Wray answered (or didn’t). But just before Wyden sent this letter, the WaPo reported that no one had yet conducted a forensic examination of the laptops used in the VR Systems polling books in North Carolina. After Democrats took over control, they finally persisted in getting DHS to agree to check the laptops.

On Tuesday, the Department of Homeland Security told The Washington Post it will conduct a forensic analysis of the laptops used in Durham County elections in 2016. Lawson said North Carolina first asked the department to conduct such a review more than 18 months ago, though he added that DHS has generally been a “good partner” on election security.

“We appreciate the Department of Homeland Security’s willingness to make this a priority so the lingering questions from 2016 can be addressed in advance of 2020,” said Karen Brinson Bell, the newly appointed executive director of the State Board of Elections.

After the election, Durham County hired a firm called Protus3 to dig into what happened. The security consultant said it appeared the problems were caused by user error but ended its 12-page report with a list of recommendations that included examining computers in a lab setting and interviewing more election workers.

Durham County elections director Derek Bowens said he is comfortable with the report’s conclusions. Even so, in 2017, the county switched to electronic poll books created by the state. Bowens said in an interview that the state’s software would save money and is, in his view, better.

But for North Carolina officials, concerns resurfaced in June 2017 when the website Intercept posted a leaked National Security Agency report referencing “cyber espionage operations against a . . . U.S. company in August 2016.” The NSA report said that “it was likely that at least one account was compromised.”

VR Systems soon acknowledged that hackers had targeted the company but insisted that its network had not been breached.

North Carolina officials weren’t so sure.

“This was the first leak that indicated anything like a nation-state actor targeting a voting systems vendor,” Lawson said.

The state elections board soon launched its own investigation, seizing 40 laptops from Durham in July. And it suspended the certification that allowed more than 20 North Carolina counties to use VR Systems’ poll books during elections, an action that would later land in court. “Over the past few months there has been a considerable change in the election security landscape and the level of scrutiny we receive,” the board wrote in a letter explaining its decision to VR Systems.

No one working for the board had the technical expertise to do a forensic examination of the machines for signs of intrusion. Staffers asked DHS for technical help but did not get a substantive answer for a year and a half, Lawson said.

As noted, FireEye appears to have done an assessment at VR Systems itself in the wake of the Winner disclosure. The WaPo reports that FireEye declared VR Systems hadn’t been hacked, but wouldn’t share any information with Wyden or–apparently–DHS.

VR Systems said a cybersecurity firm it hired to review its computer network in 2017 found no evidence of a hack. A subsequent review by DHS also found no issues, the company said. VR Systems declined to give Wyden documentation of those reviews, citing the need to protect proprietary information.

Wyden in a statement to The Post accused VR Systems of “stonewalling congressional oversight.”

A senior U.S. official confirmed DHS’s review of VR Systems’s network to The Post and noted that by the time agency investigators arrived, a commercial vendor had already “swept” the networks. “I can’t tell you what happened before the commercial vendor came in there,” the official said, speaking on the condition of anonymity to discuss a sensitive matter.

The same day as the WaPo report, Kim Zetter reported that VR Systems used remote updates for their software, opening up a possible point of compromise for hackers.

For two years, GRU hack denialists have thought it was the most important thing that the DNC provided FBI Crowdstrike’s forensic images of the hacked laptops, rather than providing the servers themselves.

But that step has, apparently, not been done yet with VR Systems. And the laptops that failed on election day are only now being forensically examined.  Which is why, I presume, that Wyden believes it’s premature to claim no vote totals were affected on election day 2016.

Cyber-Goggles: When China’s Tool Box Looks Like a Pile of Cyber-Hammers

Last week, the cybersecurity firm FireEye released a report largely declaring victory over Chinese cyberspying. The report itself is suspect. It spends two pages talking about internal issues — such as Xi Jinpeng’s efforts to consolidate power in China — then throws in a timeline designed to suggest actions the US has done has led to a decline in spying.

Screen Shot 2016-07-01 at 1.43.45 PM

The timeline itself is problematic as it suggests both indictments — of some People’s Liberation Army hackers targeting industrial companies and one union, and of Chinese businessman Su Bin — as IP hacks.

In May 2014, the U.S. Department of Justice indicted five PLA officers, marking the first time that the U.S. Government has charged foreign government personnel with crimes related to commercial cyber espionage. Although China warned that the move “jeopardizes China U.S. cooperation,” the Department of Justice indicted another Chinese national, Su Bin, the following August for allegedly orchestrating a cyber-enabled economic espionage operation targeting U.S. defense companies.

Neither should be classified so easily (though the press has irresponsibly done so, especially with respect to the PLA indictment). As I have laid out, with one exception the PLA indictment treated the theft of information pertaining to ongoing trade negotiations — something the US engages in aggressively — with the exception being the theft of trade information that China might have gotten anyone as part of a long-standing nuclear technology transfer deal with the target, Westinghouse. And while Su personally profited off his spying (or that’s what he said as part of pleading guilty), the targeted items all have a military purpose.

Without any internal evidence to back the case, FireEye declares that these indictments (the former of which, at least, relies on intelligence shared by FireEye division Mandiant) had an effect in China.

In 2014, the U.S. Government began taking punitive measures against China, from indicting members of the PLA to raising the possibility of sanctions. These unprecedented measures, though met with skepticism in the U.S., have probably been taken much more seriously in Beijing.


Screen Shot 2016-07-01 at 1.42.23 PMIn 2013, when we released the APT1 report exposing a PLA cyber espionage operation, it seemed like a quixotic effort to impede a persistent, well-resourced military operation targeting global corporations. Three years later, we see a threat that is less voluminous but more focused, calculated, and still successful in compromising corporate networks. Rather than viewing the Xi-Obama agreement as a watershed moment, we conclude that the agreement was one point amongst dramatic changes that had been taking place for years. We attribute the changes we have observed among China-based groups to factors including President Xi’s military and political initiatives, the widespread exposure of Chinese cyber operations, and mounting pressure from the U.S. Government.

The report then shows an impressive decline of perceived attacks. But even there, there’s no granularity given about where FireEye is seeing the decline (or whether these numbers might rise as it response to attacks on companies that will call FireEye in for hacks that started months or years ago). Again, in its description of the ongoing attacks, FireEye includes a lot of things that every country but the US would consider to be clear national defense hacks.

In the wake of the report, there has been some even more overheated victory laps about the success of the US-Chinese agreement in 2015, as well as this utterly absurd piece insisting that the US doesn’t engage in economic espionage. The piece is particularly nonsensical for how it uses evidence from Snowden.

More importantly, the U.S. does not steal information to give to its companies, as a rule. That none of the documents released from the vast trove of material pilfered by Edward Snowden points to this kind of commercial espionage is indicative. Those who control the Snowden documents are eager to release anything that would harm the U.S., yet they have not yet produced an example of information being given to a U.S. company.


What we know of American espionage against foreign companies (thanks to Snowden) is that the intent of the espionage against commercial targets is to support other American policies: non-proliferation, sanctions compliance, trade negotiations, foreign corrupt practices, and perhaps to gain insight into foreign military technologies.  The U.S. as well as other nations who care about such things regard these as legitimate targets for spying—legitimate in the sense that this kind of espionage would be consistent with international law and practice.  This spying supports foreign policy goals shared by many countries, in theory if not always in practice.

I say that because there’s no evidence from most domestic companies that NSA interacts with — not the Defense contractor targeted in a cyber powerpoint, and certainly not any of the telecoms that partner with the government. You would, by definition, not see evidence of what you’re claiming. Moreover, ultimately, this is retreat back to a fetish, the description of certain things to be a national good (like the trade negotiations we’ve indicted China for), but not others.

Ultimately, American commentators on cybersecurity continue to misunderstand the degree to which our corporations — especially out federal partners — cannot and are not in practice separated from a vision of national good. Though discussions about the degree to which tech companies should be wiling to risk overseas customers to spy without bound is one area where that’s assumed, even to the detriment of the tech company bottom lines.

Here’s what all this misses. There is spying of the old sort: spying on official government figures. And then there are decisions supporting national well-being (largely economics) that all countries engage in, pushing the set of rules that help them the most.

Discussions of China’s cyberspying have always been too isolated for discussions of China’s other national economic decisions. China steals just as much from US corporations located in China, but no one seems to care about that as a national security issue. And China buys a great deal, and has been buying a lot more of the things that it used to steal. The outcome is the same, yet we fetishize the method.

Which is why I find this so ironic.

A Chinese billionaire with party connections last year purchased the company, Wright USA, that insures a lot of national security officials in case they get sued or criminally investigated.

The company, Wright USA, was quietly acquired late last year by Fosun Group, a Shanghai-based conglomerate led by Guo Guangchang, a billionaire known as “China’s Warren Buffett” who has high-level Communist Party connections.

The links between Guo and Wright USA came under scrutiny by the Treasury Department’s Committee on Foreign Investment in the United States, as well as the Office of Director of National Intelligence, the coordinating body of all U.S. spy agencies, soon after Fosun announced the purchase of Wright’s parent company last November. The FBI has also launched a criminal probe into whether the company made “unauthorized disclosures of government data to outsiders,” according to a well-placed source, who like others, spoke to Newsweek on condition of anonymity because the information was sensitive.

(The FBI declined to comment, and Fosun denies the FBI has asked it for any documents.)

U.S. officials are concerned that the deal gave Chinese spy agencies a pipeline into the names, job titles, addresses and phone numbers of tens of thousands of American intelligence and counterterrorism officials—many working undercover—going back decades.

This happened after the Chinese acquired via the kind of cybertheft everyone seems to agree is old-fashioned spying the medical records and clearance records of most of Americans cleared personnel. And yet a Chinese firm was able to buy something equally compromising right out from underneath the spooks who oversee such things.

China will get what it wants via a variety of means: stealing domestically when Americans come to visit, stealing via hack, or simply buying. That we treat these differently is just a fetish, and one that seems to blind us to the multiple avenues of threat.