SPIES HACKED COMPUTERS THANKS TO SWEEPING SECRET WARRANTS, AGGRESSIVELY STRETCHING U.K. LAW

BY ANDREW FISHMAN AND GLENN GREENWALD @AndrewDFish@ggreenwald
https://firstlook.org/theintercept/2015/...-warrants/

YESTERDAY AT 1:03 PM
POPULAR


British spies have received government permission to intensively study software programs for ways to infiltrate and take control of computers. The GCHQ spy agency was vulnerable to legal action for the hacking efforts, known as "reverse engineering," since such activity could have violated copyright law. But GCHQ sought and obtained a legally questionable warrant from the Foreign Secretary in an attempt to immunize itself from legal liability.
GCHQ's reverse engineering targeted a wide range of popular software products for compromise, including online bulletin board systems, commercial encryption software and anti-virus programs. Reverse engineering "is essential in order to be able to exploit such software and prevent detection of our activities," the electronic spy agency said in a warrant renewal application.
But GCHQ's hacking and evasion goals appear to have led it onto dubious legal ground and, at times, into outright non-compliance with its own procedures for staying within the bounds of the law. A top-secret document states that a GCHQ team lapsed in following the agency's authorization protocol for some continuous period of time. Meanwhile, GCHQ obtained a warrant for reverse engineering under a section of British intelligence law that does not explicitly authorize and had apparently never been used to authorize the sort of copyright infringement GCHQ believed was necessary to conduct such activity.
The spy agency instead relied on the Intelligence Services Commissioner to let it use a law pertaining only to property and "wireless telegraphy," a law that had never been applied to intellectual property, according to GCHQ's own warrant renewal application. Eric King, deputy director of U.K. surveillance watchdog Privacy International said, after being shown documents related to the warrant, "The secret reinterpretation of powers, in entirely novel ways, that have not been tested in adversarial court processes, is everything that is wrong with how GCHQ is using their legal powers."
GCHQ may have also circumvented a restriction on using the type of warrant it obtained for domestic purposes; the agency said in one memo that it has used reverse engineering to support "police operations" and the domestic policing-focused National Technical Assistance Centre.
The agency also described efforts to cozy up to dozens of government staffers it believed could help obtain further warrants.
The agency's slippery legal maneuvers to enable computer hacking call into question U.K. government assurances about mass surveillance. To assuage public concern over such activity, the government frequently says spies are subject to rigorous oversight, including an obligation to obtain warrants. As it turns out, such authorizations have, at times, been vague and routine, as demonstrated by top-secret memos prepared by GCHQ in connection with the reverse engineering warrant.
The controversial path GCHQ took to authorize reverse engineering also seems likely to lend momentum to an ongoing push to reform the way surveillance warrants are issued in the U.K. Earlier this month, the U.K.'s independent reviewer of terrorism legislation, David Anderson, issued areport recommending that "all warrants should be judicially authorised" and describing the current regulatory system as "undemocratic, unnecessary and in the long run intolerable."
This story is based on 22 documents from NSA whistleblower Edward Snowden, linked below. None have been published before. One was briefly described in a January story in The Guardian.
[B]Widely used commercial software is targeted[/B]

One document describing the warrant, a 2008 warrant renewal application, identifies numerous commercially available products in which GCHQ identified vulnerabilities through reverse engineering. These include widely used encryption software such as Exlade's CrypticDisk and Acer's eDataSecurity. Exlade's products are used by "thousands of companies and government agencies," including tech giants IBM, Intel, GE, HP and Seagate, according to the company's website. Also successfully targeted were popular web forum services vBulletin and Invision Power Board. VBulletin says its users include Sony Pictures, NASA, Electronic Arts and Zynga. Invision Power Services, the maker of Invision Power Board, saidaround the time of the warrant renewal application that its users included Yahoo, AMD and Sony. GCHQ also targeted CPanel, software used by large hosting companies like GoDaddy for configuring servers, and PostfixAdmin, used to manage Postfix, popular email server software.
Invision Power Services said in a written statement that it monitors its software and external sources closely for information on vulnerabilities and issues fixes quickly. "There are currently no open vulnerabilities in our software of which we are aware," it added. vBulletin and Acer did not provide comment by press time. The maker of CPanel did not respond to a request for comment.
Particularly important to GCHQ was the ability to hack anti-virus programs, an offensive operation that would typically come after using reverse engineering to discover vulnerabilities. Interfering with such programs would allow the opportunity to breach a computer's defenses in order to exploit the computer without detection. GCHQ cited as a particular target Kaspersky Labs, a prominent Moscow-based maker of anti-virus software that claims more than 270,000 corporate clients. (For details on the targeting of Kaspersky, see this accompanying piece by Andrew Fishman and Morgan Marquis-Boire.)
"Personal security products such as the Russian anti-virus software Kaspersky continue to pose a challenge to GCHQ's CNE [computer network exploitation] capability and SRE [software reverse engineering] is essential in order to be able to exploit such software and to prevent detection of our activities," the 2008 document says.
Also targeted by the agency's warrants are hardware products such as large computer network routers, critical pieces of infrastructure. Hacking Cisco routers "has been good business for us and our 5-eyes partners for some time now," boasts a 2012 NSA document previously published by The Intercept.
The warrant memo describes GCHQ's "capability against Cisco routers," specifically that "GCHQ's [hacking] operations against in-country communications switches (routers) have also benefited from SRE." That has enabled the agency not only to access "almost any user of the internet" inside the entire country of Pakistan but also "to re-route selective traffic across international links toward GCHQ's passive collection systems." The Guardian previously described, but did not publish, this memo.
Cisco did not comment specifically on the warrant document, saying in a written statement only that its products are securely developed and tested, that the company has a "robust" process for handling vulnerabilities, and that "Cisco does not work with any government, including the U.K. Government, to weaken or compromise our products."
[B]Stretching the law[/B]

To support its efforts to probe and compromise software systems, GCHQ appears to have aggressively stretched Britain's Intelligence Services Act, failed to comply with its own guidelines based on that law for a continuous period, and even intentionally cozied up to staff in the Foreign and Commonwealth Office, or FCO, to get warrants approved. The apparent success of these efforts highlights the illusory nature of surveillance oversight, despite repeated government statements that the U.K. spy machine is tightly controlled.
GCHQ needed warrants, according the documents, to protect itself from potential claims of copyright infringement or of breaching a licensing agreement. The practice of reverse engineering is frequently barred in the terms and conditions attached to the copying and use of particular software by the makers of that software.
"In 2008, there was no real authority on this issue in the EU or the U.K.," says Indra Bhattacharya, a U.K. solicitor with the firm Jones Day who specializes in intellectual property law. A 2012 EU court ruling and a related 2013 U.K. court ruling allow greater latitude toward specific reverse engineering practices as long as there is no copying of code, he explains, but case law is "very fact-specific" and "deals mostly with commercial situations," making it difficult to determine how it might apply to a government agency and whether it would obviate the need for GCHQ's warrant.
But at the time of the warrant renewal application, GCHQ was clear on its legal position. "Reverse engineering of commercial products needs to be warranted in order to be lawful," one agency memo states. "There is a risk that in the unlikely event of a challenge by the copyright owner or licensor, the courts would, in the absence of a legal authorisation, hold that such activity was unlawful." Even if warrants shielded GCHQ from domestic law, the agency believed the warrant would not protect it under international law, noting that such warrant-based immunity would be "limited," given that "it only covers us under U.K. law."
GCHQ obtained its warrant under section 5 of the 1994 Intelligence Services Act, which covers interference with property and "wireless telegraphy" by the Security Service (MI5), Secret Intelligence Service (MI6) and GCHQ. Section 5 of the ISA does not mention interference in intellectual property, which the intelligence agency believed was necessary to reverse engineer software, but a top-secret memo states that the intelligence services commissioner approved such use in 2005.
This stretching of the law was dubious, says King, of Privacy International.
"It is not the Commissioner's function to provide the authoritative interpretation of any law," King says.
GCHQ did not need to go to an independent court or focus the scope of the warrant on a specific target to obtain the reverse engineering authorization. The warrant, like many surveillance warrants in the U.K., was granted by a cabinet minister, a practice harshly criticized in a just-issued report by the U.K.'s "terrorism watchdog."
The warrant renewal request for reverse engineering published today was addressed to the official that oversees GCHQ, the foreign secretary, then David Miliband, as well as two other FCO officials. The warrant is subject to renewal twice a year.
Cozying up to the Foreign and Commonwealth Office

While it was trying to hack software, GCHQ actually had efforts targeting FCO as well. Documents reveal the spy agency made a concerted effort tobuild personal relationships with key FCO staff with the goal of getting GCHQ warrants approved. One GCHQ document marked "Restricted" stated, under the heading "FCO," that "top five objectives in 08-09" included moves to provide a "greater level of routine contact between GCHQ and FCO seniors, and map members of FCO SLF [Senior Leadership Forum] to their SI/IA [Signals Intelligence/Information Assurance] interests." Another objective was to "ensure that GCHQ and FCO warrantry and submission procedures are fit for purpose given increasing complexity and need for pace in our work."
Then followed a list of dozens of named FCO staff members and a corresponding list of "major issues and targets for 09-10" for each, with goals like "win confidence by following his diary and briefing at key times," "build strong relationship with successor," "Positive about intelligence, build relationship," "Colin is new Build relationship," and "Generally supportive of submissions but could be more so."
DOCUMENT
PAGES

Zoom






















«

Page 1 of 2

»





Oversight issues

For all its efforts to win aggressive warrants clearing its reverse engineering as legal, GCHQ may well have failed to stay even with the broad boundaries it was given. When Snowden first came forward, he said part of his motivation was that there was so little monitoring of the searches NSA analysts could conduct, ensuring that abuse would often go undetected. GCHQ documents indicate there are similar problems of oversight at the British agency.
One agency memo about the reverse engineering warrants notes that, for a length of time that can't be ascertained from the document, internal authorization procedures were not adhered to by the Intrusion Detection team. When the error was discovered, the actions were simply retroactively approved.

Previously published news accounts have shown that the intelligence services commissioner works only part-time, and as of last year, had a staff of one. It was the ISC who approved the stretching of the Intelligence Services Act section 5 for use in GCHQ's software reverse engineering warrant. The ISC is also responsible for "independent external oversight" of the intelligence community. The current ISC, Sir Mark Waller, told the House of Commons' Home Affairs Committee that in 2012 he saw approximately 6 percent of more than 2,800 total warrants, with the percentage rising to roughly 12 percent the following year.
In a detailed and scathing 2014 report, the committee challenged the rigor of the ISC's oversight, citing as evidence Waller's own words:

The committee's report concluded, in boldface type: "We do not believe the current system of oversight is effective and we have concerns that the weak nature of that system has an impact upon the credibility of the agencies accountability, and to the credibility of Parliament itself."
Did GCHQ improperly use the warrant to "enable police operations?"

GCHQ may have improperly used the reverse engineering warrant for certain police-related activities, judging from language in the renewal document.
The reverse engineering warrant appears to have been used by GCHQ to support domestic law enforcement agencies and also appears to mirror existing authorizations for "activities where the effect is overseas," as one GCHQ memo put it.
The GCHQ warrant renewal application states that a number of the software exploitation efforts conducted "under the terms of this warrant … enable police operations."
The application also indicates that the warrant was used to subvert software on behalf of the National Technical Assistance Centre, or NTAC. NTAC is much more focused on domestic and law enforcement matters than on GCHQ's wider intelligence and security mission. The application says that GCHQ, on behalf of NTAC, reverse engineered Acer eDataSecurity encryption and unlocked "material relating to a high profile police case." It says it similarly thwarted CrypticDisk for NTAC, "allowing for the decryption of material relating to a child abuse investigation."
The GCHQ memo on the warrant renewal states:

The full extent of how GCHQ has applied the section 5 warrant authority to "enable police operations" is unknown. But the limitations of ISA are clear: GCHQ and MI6 cannot directly use a section 5 warrant to interfere with "property in the British Islands" if their function is "in support of the prevention or detection of serious crime," which falls under the purview of traditional law enforcement. "GCHQ should not be obtaining section 5 warrants if the purpose of the warrant is to prevent serious crime domestically," says King. The citation of police cases right in the application to justify renewal of the warrant would seem to make it difficult for GCHQ to argue that use by the police is incidental.
GCHQ refused to comment on the record about any of these matters, instead providing its boilerplate response about how it complies with the law.
____
Documents published with this article:

• GCHQ Application for Renewal of Warrant GPW/1160
  
• U.K. Ministry Stakeholder Relationships Spreadsheets (13 documents merged)
  • Foreign & Commonwealth Office Relationships & Goals
    
  
  
• Software Reverse Engineering
  
• Reverse Engineering Wiki
  
• Malware Analysis & Reverse Engineering ACNO Skill Levels
  
• TECA Product Centre Wiki
  
• Intrusion Analysis
  
• TSI Legal Authorisation Flowcharts: Targeting & Collection (2 documents merged)
  
• Operational Legalities Powerpoint Presentation
  

They nick their own stuff too:
Hundreds of secret police files go missing
ABC | 21.07.2014 15:51
Full article from The Times
Hundreds of Home Office files on a secret undercover police unit appear to have been lost or destroyed.

Theresa May's department has not handed a single document about its role in setting up and financing the Special Demonstration Squad (SDS) to Operation Herne, the long-running police investigation into the unit's activities, or to the independent legal review led by Mark Ellison, QC.

The situation echoes the 114 files on historical child abuse that the department admitted had been "lost or destroyed". Its inability to find the latest batch of documents is potentially more embarrassing because the home secretary has previously criticised the poor state of records on the unit.

At the publication of the Ellison Review, which was fiercely critical of the SDS and the lack of control by Scotland Yard, Mrs May said there was "real concern" over Metropolitan police record-keeping and referred to the alleged "mass shredding" of evidence in 2003.

The squad has been the focus of scrutiny after revelations that officers used dead children's names to create identities, were deployed in political protest groups and had long-term relationships with female activists, sometimes marrying and fathering children.

Although run from Scotland Yard, the SDS was created in 1968, when James Callaghan was home secretary, in response to the Vietnam war protests outside the American embassy in London. The Home Office financed it directly and its hand-picked members were trained by MI5. An external review was set up to report on the Home Office's involvement in the unit, but it is understood that no papers have been made available to either the police investigation into the SDS or to Mr Ellison's review.

A source with knowledge of the reviews said: "The police took a hammering over poor record-keeping and loss of documents, but it turns out the Home Office is far, far worse."

The Met has continued to find and hand over thousands of files to the police investigation into the SDS. The Met has also interviewed 85 people regarding the allegation of "mass shredding" but said that only one person remembered the incident. That witness claimed that four bin bags of papers were destroyed in 2001 after the information was computerised.

The Met has appointed an assistant commissioner to head the trawl for documents before a promised public inquiry into the SDS, particularly the deployment of officers in groups involved in the campaign for an investigation into the murder of Stephen Lawrence. The Ellison Review found evidence of a "spy in the camp" who reported back to Scotland Yard on the Lawrence family.

The SDS was disbanded in 2008, and undercover deployments are now overseen by the Surveillance Commissioner. A Home Office spokesman would not comment on the loss of documents. He said: "In the interests of transparency, the home secretary announced in March that the permanent secretary would commission a forensic external review in order to establish the full extent of the Home Office's knowledge of the SDS so we could understand the role the department played."

A Met assistant commissioner, Martin Hewitt, acknowledged the force's failings: "With the amount of information generated in everyday policing, effective record management can present challenges and the Met fully accepts that it has not kept good records of what was retained or destroyed in the past."
ABC
They can do it all by 'Polar Breeze', remotley too.

I used to have a game, 'STALKER, Clear Sky', singleplayer - offline, which suddenly changed - was hacked; there were gates to other parts of the bigmap, gates were guarded. Soon after the Monarch program or whatever it's called set-in, the game started running badly, very glitchy, and the guards at the gates would just shoot you - very different AI, pretty much killing the game as playable. Don't know if it was done remotely or was done when I was out. c.October2011.

I have come to the conclusion that the Guardian newspaper is schizophrenic and requires both medication and medical supervision. It doesn't seem to know if it's fish nor fowl, independent or state servant, a newspaper or a propaganda tool. Because from day to day it's all those things.

Witness what it has published on the Panama Papers and the piece below (albeit not exactly a new revelation but non-the-less an important story).

For me it suggests that there are journo's still working there who try to get real news out to the public, but they are in the minority. The majority simply bend over and have their prostrates felt.

Quote:

Revealed: how US and UK spy agencies defeat internet privacy and security

NSA and GCHQ unlock encryption used to protect emails, banking and medical records
$250m-a-year US program works covertly with tech companies to insert weaknesses into products
Security experts say programs 'undermine the fabric of the internet'

Q&A: submit your questions for our privacy experts






Through covert partnerships with tech companies, the spy agencies have inserted secret vulnerabilities into encryption software. Photograph: Kacper Pempel/ReutersJames Ball, Julian Borger and Glenn Greenwald
Friday 6 September 2013 11.24 BSTLast modified on Thursday 28 January 2016 20.19 GMT

• [URL="https://www.facebook.com/dialog/share?app_id=180444840287&href=http%3A%2F%2Fgu.com%2Fp%2F3thvv%2Fsfb&redirect_uri=http%3A%2F%2Fgu.com%2Fp%2F3thvv%2Fsfb"]
  [/URL]
  
• [URL="https://twitter.com/intent/tweet?text=Revealed%3A%20how%20US%20and%20UK%20spy%20agencies%20defeat%20internet%20privacy%20and%20security&url=http%3A%2F%2Fgu.com%2Fp%2F3thvv%2Fstw"]
  [/URL]
  
• [EMAIL="?subject=Revealed%3A%20how%20US%20and%20UK%20spy%20agencies%20defeat%20internet%20privacy%20and%20security&body=http%3A%2F%2Fgu.com%2Fp%2F3thvv%2Fsbl"]
  [/EMAIL]
  
• [URL="http://www.pinterest.com/pin/find/?url=http%3A%2F%2Fgu.com%2Fp%2F3thvv"]Share on Pinterest
  [/URL]
  
• [URL="http://www.linkedin.com/shareArticle?mini=true&title=Revealed%3A+how+US+and+UK+spy+agencies+defeat+internet+privacy+and+security&url=http%3A%2F%2Fgu.com%2Fp%2F3thvv"]Share on LinkedIn
  [/URL]
  
• [URL="https://plus.google.com/share?url=http%3A%2F%2Fgu.com%2Fp%2F3thvv%2Fsgp&hl=en-GB&wwc=1"]Share on Google+
  [/URL]
  

This article is 2 years old

Shares

45k

[URL="http://www.theguardian.com/world/2013/sep/05/nsa-gchq-encryption-codes-security#comments"]

Comments

4,142[/URL]

[URL="https://profile.theguardian.com/save-content?INTCMP=DOTCOM_ARTICLE_SFL&returnUrl=http%3A%2F%2Fwww.theguardian.com%2Fworld%2F2013%2Fsep%2F05%2Fnsa-gchq-encryption-codes-security&shortUrl=/p/3thvv&platform=web:Safari:desktop"]

Save for later[/URL]


US and British intelligence agencies have successfully cracked much of the online encryption relied upon by hundreds of millions of people to protect the privacy of their personal data, online transactions and emails, according to top-secret documents revealed by former contractor Edward Snowden.
The files show that the National Security Agency and its UK counterpart GCHQhave broadly compromised the guarantees that internet companies have given consumers to reassure them that their communications, online banking and medical records would be indecipherable to criminals or governments.
The agencies, the documents reveal, have adopted a battery of methods in their systematic and ongoing assault on what they see as one of the biggest threats to their ability to access huge swathes of internet traffic "the use of ubiquitous encryption across the internet".
Those methods include covert measures to ensure NSA control over setting of international encryption standards, the use of supercomputers to break encryption with "brute force", and the most closely guarded secret of all collaboration with technology companies and internet service providers themselves.

The stories you need to read, in one handy email

Read more



Through these covert partnerships, the agencies have inserted secret vulnerabilities known as backdoors or trapdoors into commercial encryption software.
The files, from both the NSA and GCHQ, were obtained by the Guardian, and the details are being published today in partnership with the New York Times and ProPublica. They reveal:
A 10-year NSA program against encryption technologies made a breakthrough in 2010 which made "vast amounts" of data collected through internet cable taps newly "exploitable".
The NSA spends $250m a year on a program which, among other goals, works with technology companies to "covertly influence" their product designs.
The secrecy of their capabilities against encryption is closely guarded, with analysts warned: "Do not ask about or speculate on sources or methods."
The NSA describes strong decryption programs as the "price of admission for the US to maintain unrestricted access to and use of cyberspace".
A GCHQ team has been working to develop ways into encrypted traffic on the "big four" service providers, named as Hotmail, Google, Yahoo and Facebook.


This network diagram, from a GCHQ pilot program, shows how the agency proposed a system to identify encrypted traffic from its internet cable-tapping programs and decrypt what it could in near-real time. Photograph: GuardianThe agencies insist that the ability to defeat encryption is vital to their core missions of counter-terrorism and foreign intelligence gathering.
But security experts accused them of attacking the internet itself and the privacy of all users. "Cryptography forms the basis for trust online," said Bruce Schneier, an encryption specialist and fellow at Harvard's Berkman Center for Internet and Society. "By deliberately undermining online security in a short-sighted effort to eavesdrop, the NSA is undermining the very fabric of the internet." Classified briefings between the agencies celebrate their success at "defeating network security and privacy".
"For the past decade, NSA has lead [sic] an aggressive, multi-pronged effort to break widely used internet encryption technologies," stated a 2010 GCHQ document. "Vast amounts of encrypted internet data which have up till now been discarded are now exploitable."
An internal agency memo noted that among British analysts shown a presentation on the NSA's progress: "Those not already briefed were gobsmacked!"
The breakthrough, which was not described in detail in the documents, meant the intelligence agencies were able to monitor "large amounts" of data flowing through the world's fibre-optic cables and break its encryption, despite assurances from internet company executives that this data was beyond the reach of government.
The key component of the NSA's battle against encryption, its collaboration with technology companies, is detailed in the US intelligence community's top-secret 2013 budget request under the heading "Sigint [signals intelligence] enabling".


Classified briefings between the NSA and GCHQ celebrate their success at 'defeating network security and privacy'. Photograph: GuardianFunding for the program $254.9m for this year dwarfs that of the Prism program, which operates at a cost of $20m a year, according to previous NSA documents. Since 2011, the total spending on Sigint enabling has topped $800m. The program "actively engages US and foreign IT industries to covertly influence and/or overtly leverage their commercial products' designs", the document states. None of the companies involved in such partnerships are named; these details are guarded by still higher levels of classification.

Advertisement

Among other things, the program is designed to "insert vulnerabilities into commercial encryption systems". These would be known to the NSA, but to no one else, including ordinary customers, who are tellingly referred to in the document as "adversaries".
"These design changes make the systems in question exploitable through Sigint collection … with foreknowledge of the modification. To the consumer and other adversaries, however, the systems' security remains intact."

The document sets out in clear terms the program's broad aims, including making commercial encryption software "more tractable" to NSA attacks by "shaping" the worldwide marketplace and continuing efforts to break into the encryption used by the next generation of 4G phones.

Among the specific accomplishments for 2013, the NSA expects the program to obtain access to "data flowing through a hub for a major communications provider" and to a "major internet peer-to-peer voice and text communications system".
Technology companies maintain that they work with the intelligence agencies only when legally compelled to do so. The Guardian has previously reported that Microsoft co-operated with the NSA to circumvent encryption on the Outlook.com email and chat services. The company insisted that it was obliged to comply with "existing or future lawful demands" when designing its products.

The documents show that the agency has already achieved another of the goals laid out in the budget request: to influence the international standards upon which encryption systems rely.

Independent security experts have long suspected that the NSA has been introducing weaknesses into security standards, a fact confirmed for the first time by another secret document. It shows the agency worked covertly to get its own version of a draft security standard issued by the US National Institute of Standards and Technology approved for worldwide use in 2006.

Advertisement

"Eventually, NSA became the sole editor," the document states.
The NSA's codeword for its decryption program, Bullrun, is taken from a major battle of the American civil war. Its British counterpart, Edgehill, is named after the first major engagement of the English civil war, more than 200 years earlier.
A classification guide for NSA employees and contractors on Bullrun outlines in broad terms its goals.
"Project Bullrun deals with NSA's abilities to defeat the encryption used in specific network communication technologies. Bullrun involves multiple sources, all of which are extremely sensitive." The document reveals that the agency has capabilities against widely used online protocols, such as HTTPS, voice-over-IP and Secure Sockets Layer (SSL), used to protect online shopping and banking.

The document also shows that the NSA's Commercial Solutions Center, ostensibly the body through which technology companies can have their security products assessed and presented to prospective government buyers, has another, more clandestine role.
It is used by the NSA to "to leverage sensitive, co-operative relationships with specific industry partners" to insert vulnerabilities into security products. Operatives were warned that this information must be kept top secret "at a minimum".
A more general NSA classification guide reveals more detail on the agency's deep partnerships with industry, and its ability to modify products. It cautions analysts that two facts must remain top secret: that NSA makes modifications to commercial encryption software and devices "to make them exploitable", and that NSA "obtains cryptographic details of commercial cryptographic information security systems through industry relationships".

The agencies have not yet cracked all encryption technologies, however, the documents suggest. Snowden appeared to confirm this during a live Q&A with Guardian readers in June. "Encryption works. Properly implemented strong crypto systems are one of the few things that you can rely on," he said before warning that NSA can frequently find ways around it as a result of weak security on the computers at either end of the communication.
The documents are scattered with warnings over the importance of maintaining absolute secrecy around decryption capabilities.


A slide showing that the secrecy of the agencies' capabilities against encryption is closely guarded. Photograph: GuardianStrict guidelines were laid down at the GCHQ complex in Cheltenham, Gloucestershire, on how to discuss projects relating to decryption. Analysts were instructed: "Do not ask about or speculate on sources or methods underpinning Bullrun." This informaton was so closely guarded, according to one document, that even those with access to aspects of the program were warned: "There will be no 'need to know'."

Advertisement

The agencies were supposed to be "selective in which contractors are given exposure to this information", but it was ultimately seen by Snowden, one of 850,000 people in the US with top-secret clearance.A 2009 GCHQ document spells out the significant potential consequences of any leaks, including "damage to industry relationships".

"Loss of confidence in our ability to adhere to confidentiality agreements would lead to loss of access to proprietary information that can save time when developing new capability," intelligence workers were told. Somewhat less important to GCHQ was the public's trust which was marked as a moderate risk, the document stated.
"Some exploitable products are used by the general public; some exploitable weaknesses are well known eg possibility of recovering poorly chosen passwords," it said. "Knowledge that GCHQ exploits these products and the scale of our capability would raise public awareness generating unwelcome publicity for us and our political masters."
The decryption effort is particularly important to GCHQ. Its strategic advantage from its Tempora program direct taps on transatlantic fibre-optic cables of major telecommunications corporations was in danger of eroding as more and more big internet companies encrypted their traffic, responding to customer demands for guaranteed privacy.
Without attention, the 2010 GCHQ document warned, the UK's "Sigint utility will degrade as information flows changes, new applications are developed (and deployed) at pace and widespread encryption becomes more commonplace." Documents show that Edgehill's initial aim was to decode the encrypted traffic certified by three major (unnamed) internet companies and 30 types of Virtual Private Network (VPN) used by businesses to provide secure remote access to their systems. By 2015, GCHQ hoped to have cracked the codes used by 15 major internet companies, and 300 VPNs.

Another program, codenamed Cheesy Name, was aimed at singling out encryption keys, known as 'certificates', that might be vulnerable to being cracked by GCHQ supercomputers.
Analysts on the Edgehill project were working on ways into the networks of major webmail providers as part of the decryption project. A quarterly update from 2012 notes the project's team "continue to work on understanding" the big four communication providers, named in the document as Hotmail, Google, Yahoo and Facebook, adding "work has predominantly been focused this quarter on Google due to new access opportunities being developed".
To help secure an insider advantage, GCHQ also established a Humint Operations Team (HOT). Humint, short for "human intelligence" refers to information gleaned directly from sources or undercover agents.
This GCHQ team was, according to an internal document, "responsible for identifying, recruiting and running covert agents in the global telecommunications industry."
"This enables GCHQ to tackle some of its most challenging targets," the report said. The efforts made by the NSA and GCHQ against encryption technologies may have negative consequences for all internet users, experts warn.
"Backdoors are fundamentally in conflict with good security," said Christopher Soghoian, principal technologist and senior policy analyst at the American Civil Liberties Union. "Backdoors expose all users of a backdoored system, not just intelligence agency targets, to heightened risk of data compromise." This is because the insertion of backdoors in a software product, particularly those that can be used to obtain unencrypted user communications or data, significantly increases the difficulty of designing a secure product."

Advertisement

This was a view echoed in a recent paper by Stephanie Pell, a former prosecutor at the US Department of Justice and non-resident fellow at the Center for Internet and Security at Stanford Law School.
"[An] encrypted communications system with a lawful interception back door is far more likely to result in the catastrophic loss of communications confidentiality than a system that never has access to the unencrypted communications of its users," she states.
Intelligence officials asked the Guardian, New York Times and ProPublica not to publish this article, saying that it might prompt foreign targets to switch to new forms of encryption or communications that would be harder to collect or read.
The three organisations removed some specific facts but decided to publish the story because of the value of a public debate about government actions that weaken the most powerful tools for protecting the privacy of internet users in the US and worldwide.