What Yahoo and Google did not think the NSA could see

The details

Click to see the related section of the document.
[URL="http://apps.washingtonpost.com/g/page/world/what-yahoo-and-google-did-not-think-the-nsa-could-see/555/#document/p1/a130006"]

What are Special Source Operations?

The SSO group, the insignia of which includes an eagle grasping fiber optic cables that span the globe, collects intelligence with the help of U.S. and foreign companies. Documents suggest it accounts for the largest fraction of all NSA collection.
[/URL][URL="http://apps.washingtonpost.com/g/page/world/what-yahoo-and-google-did-not-think-the-nsa-could-see/555/#document/p2/a130010"]

Yahoo data formats

When NSA systems ingest a stream of data, they send it through many layers of filtering and sorting. The TUDDS tool applies "selectors" (in effect, "keep this") and "defeats" ("discard this"). This slide shows defeat signatures for information that the NSA does not want. Any data matching a signature are blocked "at router," the collection point. Further selection is done at later stages of processing. This slide is significant because the signatures specified refer to proprietary Yahoo data formats that do not generally travel on the public Internet.
[/URL][URL="http://apps.washingtonpost.com/g/page/world/what-yahoo-and-google-did-not-think-the-nsa-could-see/555/#document/p4/a130013"]

What is NSA blocking with these "defeats"?

Each rule is meant to stop a specific kind of Google traffic. [adwords] is Google's web advertising network. [bigtable] is a proprietary Google database system that is 'not distributed outside Google'. [teragoogle] is a proprietary process used by Google to index Web sites in order to deliver search results quickly.
[/URL][URL="http://apps.washingtonpost.com/g/page/world/what-yahoo-and-google-did-not-think-the-nsa-could-see/555/#document/p5/a130014"]

Inside Google's network

This is a 'packet capture,' or a stream of unprocessed data passing through NSA collection systems. This slide shows one of Google's warehouse-sized data centers confirming, or authenticating, that it is talking securely to another, probably thousands of miles away. Engineers familiar with Google's systems said the NSA should not see this traffic from anywhere outside Google's internal network. Gaia is the authentication system used inside Google's internal network. Marina is a principal NSA database for Internet metadata.
[/URL][URL="http://apps.washingtonpost.com/g/page/world/what-yahoo-and-google-did-not-think-the-nsa-could-see/555/#document/p6/a130015"]

Google's internal traffic

This pie chart shows different types of internal Google network traffic, by volume. Some of the data types, including "Google Authorization" and "gaia//permission_whitelist," are available only inside Google's private cloud.
[/URL]


DOCUMENT
PAGES
NOTES
TEXT




Zoom

p. 1

p. 2

p. 3

«

Page 1 of 6

»







GRAPHIC: Barton Gellman and Matt DeLong - The Washington Post.

Back in the 1990s, security researchers and privacy watchdogs were alarmed by government demands that hardware and software firms build "backdoors" into their products, the millions of personal computers and cell phones propelling communication flows along the now-quaint "information superhighway."
Never mind that the same factory-installed kit that allowed secret state agencies to troll through private communications also served as a discrete portal for criminal gangs to loot your bank account or steal your identity.
To make matters worse, instead of the accountability promised the American people by Congress in the wake of the Watergate scandal, successive US administrations have worked assiduously to erect an impenetrable secrecy regime backstopped by secret laws overseen by secret courts which operate on the basis of secret administrative subpoenas, latter daylettres de cachet.
But now that all their dirty secrets are popping out of Edward Snowden's "bottomless briefcase," we also know the "Crypto Wars" of the 1990s never ended.
Documents published by The Guardian and The New York Times revealed that the National Security Agency "actively engages the US and IT industries" and has "broadly compromised the guarantees that internet companies have given consumers to reassure them that their communications, online banking and medical records would be indecipherable to criminals or governments."

"Those methods include covert measures to ensure NSA control over setting of international encryption standards," The Guardian disclosed, along with "the use of supercomputers to break encryption with brute force', andthe most closely guarded secret of allcollaboration with technology companies and internet service providers themselves."

According to The New York Times, NSA "had found ways inside some of the encryption chips that scramble information for businesses and governments, either by working with chipmakers to insert back doors or by surreptitiously exploiting existing security flaws, according to the documents."
In fact, "vulnerabilities" inserted "into commercial encryption systems" would be known to NSA alone. Everyone else, including commercial customers, are referred to in the documents as "adversaries."
The cover name for this program is Project BULLRUN. An agency classification guide asserts that "Project BULLRUN deals with NSA's abilities to defeat the encryption used in specific network communication technologies. BULLRUN involves multiple sources, all of which are extremely sensitive. They include CNE [computer network exploitation], interdiction, industry relationships, collaboration with other IC entities, and advanced mathematical techniques."
In furtherance of those goals, the agency created a "Commercial Solutions Center (NCSC) to leverage sensitive, cooperative relationships with industry partners" that will "further NSA/CSS capabilities against encryption used in network communications technologies," and already "has some capabilities against the encryption used in TLS/SSL. HTTPS, SSH, VPNs, VoIP, WEBMAIL, and other network communications technologies."
Time and again, beginning in the 1970s with the publication of perhaps the earliest NSA exposé byRamparts Magazine, we learned that when agency schemes came to light, if they couldn't convince they resorted to threats, bribery or the outright subversion of the standard setting process itself, which destroyed trust and rendered all our electronic interactions far less safe.
Tunneling underground, NSA, telcos and corporate tech giants worked hand-in-glove to sabotage what could have been a free and open system of global communications, creating instead the Frankenstein monster which AT&T whistleblower Mark Klein denounced as a "Big Brother machine."
The Secret State and the Internet
Five years after British engineer Tim Berners-Lee, Belgian computer scientist Robert Cailliau and their team at CERN developed a system for assembling, and sharing, hypertext documents via the internet, which they dubbed the World Wide Web, in 1994 the Clinton administration announced it would compel software and hardware developers to install what came to known as the "Clipper Chip" into their products.
The veritable explosion of networked communication systems spawned by the mass marketing of easy-to-use personal computers equipped with newly-invented internet browsers, set off a panic amongst political elites.
How to control these seemingly anarchic information flows operating outside "normal" channels?
In theory at least, those doing the communicatingacademics, dissidents, journalists, economic rivals, even other spies, hackers or "terrorists" (a fungible term generally meaning outsider groups not on board with America's imperial goals)were the least amenable users of the new technology and would not look kindly on state efforts to corral them.
As new communication systems spread like wildfire, especially among the great unwashed mass of "little people," so too came a stream of dire pronouncements that the internet was now a "critical national asset" which required close attention and guidance.
President Clinton's Commission on Critical Infrastructure Protection released a report that called for a vast increase in funding to protect US infrastructure along with one of the first of many "cyberwar" tropes that would come to dominate the media landscape.
"In the cyber dimension," the report breathlessly averred, "there are no boundaries. Our infrastructures are exposed to new vulnerabilitiescyber vulnerabilitiesand new threatscyber threats. And perhaps most difficult of all, the defenses that served us so well in the past offer little protection from the cyber threat. Our infrastructures can now be struck directly by a variety of malicious tools."
And when a commercial market for cheap, accessible encryption software was added to the mix, security mandarins at Ft. Meade and Cheltenham realized the genie would soon be out of the bottle.
After all they reasoned, NSA and GCHQ were the undisputed masters of military-grade cryptography who had cracked secret Soviet codes which helped "win" the Cold War. Were they to be out maneuvered by some geeks in a garage who did not share or were perhaps even hostile to the "post-communist" triumphalism which had decreed America was now the world's "indispensable nation"?
Technological advances were leveling the playing field, creating new democratic space in the realm of knowledge creation accessible to everyone; a new mode for communicating which threatened to bypass entrenched power centers, especially in government and media circles accustomed to a monopoly over the Official Story.
US spies faced a dilemma. The same technology which created a new business model worth hundreds of billions of dollars for US tech corporations also offered the public and pesky political outliers across the political spectrum, the means to do the same.
How to stay ahead of the curve? Why not control the tempo of product development by crafting regulations, along with steep penalties for noncompliance, that all communications be accessible to our guardians, strictly for "law enforcement" purposes mind you, by including backdoors into commercially available encryption products.
Total Information Awareness 1.0
Who to turn to? Certainly such hush-hush work needed to be in safe hands.
The Clinton administration, in keeping with their goal to "reinvent government" by privatizing everything, turned to Mykotronx, Inc., a California-based company founded in 1983 by former NSA engineers, Robert E. Gottfried and Kikuo Ogawa, mining gold in the emerging information security market.
Indeed, one of the firm's top players was Ralph O'Connell, was described in a 1993 documentpublished by Computer Professionals for Social Responsibility (CPSR) as "the father of COMSEC" and the "Principle NSA Technical Contact" on Clipper and related cryptography projects.
A 1993 Business Wire release quoted the firm's president, Leonard J. Baker, as saying that Clipper was "a good example of the transfer of military technology to the commercial and general government fields with handsome cost benefits. This technology should now pay big dividends to US taxpayers."
It would certainly pay "big dividends" to Mykotronx's owners.
Acquired by Rainbow Technologies in 1995, and eventually by Military-Industrial-Surveillance Complex powerhouse Raytheon in 2012, at the time the Los Angeles Times reported that "Mykotronx had been privately held, and its owners will receive 1.82 million shares of Rainbow stockmaking the deal worth $37.9 million."
The Clipper chip was touted by the administration as a simple device that would protect the private communications of users while also allowing government agents to obtain the keys that unlocked those communications, an early manifestation of what has since become know as law enforcement's alleged "going dark" problem.
Under color of a vague "legal authorization" that flew in the face of the 1987 Computer Security Act (CSA), which sought to limit the role of the National Security Agency in developing standards for civilian communications systems, the administration tried an end-run around the law through an export ban on Clipper-free encryption devices overseen by the Commerce Department.
This wasn't the first time that NSA was mired in controversy over the watering down of encryption standards. During the development of the Data Encryption Standard (DES) by IBM in the 1970s, the agency was accused of forcing developers to implement changes in the design of its basic cipher. There were strong suspicions these changes had weakened the algorithm to such a degree that one critical component, the S-box, had been altered and that a backdoor was inserted by NSA.
Early on, the agency grasped CSA's significance and sought to limit damage to global surveillance and economic espionage programs such as ECHELON, exposed by British and New Zealand investigative journalists Duncan Campbell and Nicky Hager.
Before the 1987 law was passed however, Clinton Brooks, a Special Assistant to NSA Director Lieutenant General William Odom, wrote a Top Secret Memorandum which stated: "In 1984 NSA engineered a National Security Decision Directive, NSDD-145, through the Reagan Administration that gave responsibility for the security of all US information systems to the Director of NSA, removing NBS [National Bureau of Standards] from this."
Conceived as a follow-on to the Reagan administration's infamous 1981 Executive Order 12333, which trashed anemic congressional efforts to rein-in America's out-of-control spy agencies, NSDD-145 handed power back to the National Security Agency and did so to the detriment of civilian communication networks.
Scarcely a decade after Senator Frank Church warned during post-Watergate hearings into government surveillance abuses, that NSA's "capability at any time could be turned around on the American people, and no American would have any privacy left, such is the capability to monitor everything: telephone conversations, telegrams, it doesn't matter . . . there would be no place to hide," the agency was at it with a vengeance.
"This [NSDD-145] also stated," Brooks wrote, "that we would assist the private sector. This was viewed as Big Brother stepping in and generated an adverse reaction" in Congress that helped facilitate passage of the Act.
Engineered by future Iran-Contra felon, Admiral John Poindexter, President Reagan's National Security Adviser who would later serve as President George W. Bush's Director of DARPA's Information Awareness Office, the Pentagon satrapy that brought us the Total Information Awareness program, NSDD-145 stated that the "Director, National Security Agency is designated the National Manager for Telecommunications and Automated Information Systems Security."
NSA's new mandate meant that the agency would "act as the government focal point for cryptography, telecommunications systems security, and automated information systems security."
Additionally, NSA would "conduct, approve, or endorse research and development of techniques and equipment for telecommunications and automated information systems security for national security information."
But it also authorized the agency to do more than that, granting it exclusive authority to "review and approve all standards, techniques, systems and equipments for telecommunications and automated information systems security." As well, NSA was directed to "enter into agreements for the procurement of technical security material and other equipment, and their provision to government agencies, where appropriate, to private organizations, including government contractors, and foreign governments."
In other words, NSA was the final arbiter when it came to setting standards for all government and private information systems; quite a coup for the agency responsible for standing-up Project MINARET, the Cold War-era program that spied on thousands of antiwar protesters, civil rights leaders, journalists and members of Congress, as recently declassified documents published by the National Security Archive disclosed.
NSA Games the System
Although the Computer Security Act passed unanimously by voice vote in both Houses of Congress, NSA immediately set-out to undercut the law and did so by suborning the National Bureau of Standards, now the National Institute of Standards and Technology (NIST).
The battle over the Clipper Chip would be the template for future incursions by the agency for the control, through covert infiltration, of regulatory bodies overseeing civilian communications.
According to the Clinton White House, Clipper "would provide Americans with secure telecommunications without compromising the ability of law enforcement agencies to carry out legally authorized wiretaps."
Neither safe nor secure, Clipper instead would have handed government security agencies the means to monitor all communications while giving criminal networks a leg up to do the same.
In fact, as the Electronic Privacy Information Center (EPIC) discovered in documents unearthed through the Freedom of Information Act, the underlying algorithm deployed in Clipper, Skipjack, had been developed by NSA.
Cryptography expert Matt Blaze wrote a now famous 1994 paper on the subject before the algorithm was declassified, Protocol Failure in the Escrowed Encryption Standard: "The EES cipher algorithm, called Skipjack', is itself classified, and implementations of the cipher are available to the private sector only within tamper-resistant modules supplied by government-approved vendors. Software implementations of the cipher will not be possible. Although Skipjack, which was designed by the US National Security Agency (NSA), was reviewed by a small panel of civilian experts who were granted access to the algorithm, the cipher cannot be subjected to the degree of civilian scrutiny ordinarily given to new encryption systems."
This was precisely as NSA and the Clinton administration intended.
A partially declassified 1993 NSA memo noted that "there will be vocal public doubts expressed about having a classified algorithm in the device we propose for the US law enforcement problem, the CLIPPER chip, we recommend the following to address this." We don't know what those agency recommendations were, however; more than 20 years after the memo was written they remain secret.
The memo continued: "If such people agree to this clearance and non disclosure process, we could go over the algorithm with them to let them develop confidence in its security, and we could also let them examine the detail design of the CLIPPER chip made for the US law enforcement problem to assure themselves that there were no trapdoors or other techniques built in. This would likely require crypto-mathematicians for the algorithm examination and microelectronics chip design engineers for the chip examination."
But the extreme secrecy surrounding Skipjack's proposed deployment in commercial products wasthe problem. Even if researchers learned that Clipper was indeed the government-mandated backdoor they feared, non-disclosure of these facts, backed-up by the threat of steep fines or imprisonment would hardly assure anyone of the integrity of this so-called review process.
"By far, the most controversial aspect of the EES system," Blaze wrote, "is key escrow."

"As part of the crypto-synchronization process," Blaze noted, "EES devices generate and exchange a Law Enforcement Access Field' (LEAF). This field contains a copy of the current session key and is intended to enable a government eavesdropper to recover the cleartext."
"The LEAF copy of the session key is encrypted with a device-unique key called the unit key,' assigned at the time the EES device is manufactured. Copies of the unit keys for all EES devices are to be held in escrow' jointly by two federal agencies that will be charged with releasing the keys to law enforcement under certain conditions."

What those conditions were however, was far from clear. In fact, as we've since learned from Snowden's cache of secret documents, even when the government seeks surveillance authorization from the FISA court, the court must rely on government assurances that dragnet spying is critical to the nation's security. Such assurances, FISA court judge Reggie B. Walton noted, were systematically "misrepresented" by secret state agencies.
That's rather rich considering that Walton presided over the farcical "trial" that upheld Bush administration demands to silence FBI whistleblower Sibel Edmonds under the state secrets privilege. Edmonds, a former contract linguist with the Bureau charged that top FBI officials had systematically covered-up wrongdoing at its language division and had obstructed agents' attempts to roll-up terrorist networks before and after the 9/11 provocation, facts attested to by FBI whistleblower Coleen Rowley in her 2002 Memo to then-FBI Director Robert Mueller.
In 2009, Walton wrote that "The minimization procedures proposed by the government in each successive application and approved and adopted as binding by the orders of the FISC have been so frequently and systemically violated that it can fairly be said that this critical element of the overall BR regime has never functioned effectively."
"The Court," Walton averred, "must have every confidence that the government is doing its utmost to ensure that those responsible for implementation fully comply with the Court's orders. The Court no longer has such confidence."
Predating those critical remarks, a heavily-redacted 1993 Memo to then-Special Assistant to the President and future CIA chief, George Tenet, from FBI Director William Sessions noted that NSA "has developed a new encryption methodology and computer chip which affords encryption strength vastly superior to DES [Digital Encryption Standard], yet which allows for real time decryption by law enforcement, acting pursuant to legal process. It is referred to as Clipper'."
[Two redacted paragraphs] "if the devices are modified to include the Clipper' chip, they would be of great value to the Federal, state and local law enforcement community, especially in the area of counter narcotics, investigations, where there is a requirement to routinely communicate in a secure fashion."
But even at the time Sessions' memo was written, we now know that AT&T provided the Drug Enforcement Administration "routine access" to "an enormous AT&T database that contains the records of decades of Americans' phone calls," The New York Times reported, and had done so since 1987 under the auspices of DEA's Hemisphere Project.
Furthermore, in the wake of Snowden revelations we also learned that listening in on the conversations of drug capos is low on NSA's list of priorities. However, programs like X-KEYSCOREand TEMPORA, which copies all data flowing along fiber optic cables, encrypted and unencrypted alike, at petabyte scales, is supremely useful when it comes to building profiles of internet users by intelligence agencies.
This was an implicit goal of Clinton administration maneuvers to compel developers to insert Clipper into their product designs.
According to Sessions, "the Clipper' methodology envisions the participation of three distinct types of parties." [Redacted] It is proposed that the second party, the two custodians of the split' key infostructure [sic], be comprised of two disinterested and trustworthy non law enforcement Government agencies or entities. Although, such decision and selection are left for the Administration, a list of reccommended [sic] agencies and entities has been prepared (and included in the text), [redacted]. This party would administer and oversee all facets of the Clipper' program and methodology."
Based on NSDD-145′s mandate, one can assume "this party" would be NSA, the agency that designed the underlying algorithm that powered Clipper.
The Sessions memo averred: "The Clipper chip provides law enforcement access by using a special chip key, unique to each device. In the AT&T TSD 3600, a unique session key is generated, external to the Clipper chip for each call."

"This session key," the memo explained, "is given to the chip to control the encryption algorithm. A device unique chip key' is programmed into each Clipper at the time of manufacture. When two TSD 3600s go to secure operation, the device gives out its identification (ID) number and the session key encrypted in its chip key."

Underlining a key problem with Clipper technology Sessions noted, "Anyone with access to the chip key for that identified device will be able to recover the session key and listen to the transmission simultaneously with the intended receiver. This design means that the list of chip keys associated with the chip ID number provides access to all Clipper secured devices, and thus the list must be carefully generated and protected. Loss of the list would preclude legitmate [sic] access to the encrypted information and compromise of the list could allow unauthorized access."
In fact, that "anyone" could include fabulously wealthy drug gangs or bent corporations with the wherewithal to buy chip keys from suborned government key escrow agents!
Its ubiquity would be a key selling-point for universal deployment. The memo explained, "the NSA developed chip based Clipper' solution works with hardware encryption applications, such as those which might be used with regard to certain telecommunications and computers devices," which of course would allow unlimited spying by "law enforcement."
Such vulnerabilities built into EES chip keys by design not only enabled widespread government monitoring of internet and voice traffic, but with a few tweaks by encryption-savvy "rogues" could be exploited by criminal organizations.
In his 1994 paper Blaze wrote that "a rogue system can be constructed with little more than a software modification to a legal system. Furthermore, while some expertise may be required to install and operate a rogue version of an existing system, it is likely that little or no special skill would be required to install and operate the modified software."
"In particular," Blaze noted, "one can imagine patches' to defeat key escrow in EES-based systems being distributed over networks such as the Internet in much the same way that other software is distributed today."
In the intervening years since Blaze observed how easy it would be to compromise key escrow systems by various bad actors, governments or criminals take your pick, the proliferation of malware powered botnets that infect hundreds of thousands of computers and smart phones every dayfor blanket surveillance, fraud, or bothis a fact of life.
It didn't help matters when it emerged that "escrow agents" empowered to unlock encrypted communications would be drawn from the National Institute of Standards and Technology and the Automated Services Division of the Treasury Department, government outposts riddled with "No Such Agency" moles.
As EPIC pointed out, "Since the enactment of the Computer Security Act, the NSA has sought to undercut NIST's authority. In 1989, NSA signed a Memorandum of Understanding (MOU) which purported to transfer back to NSA the authority given to NIST."
The MOU required that NIST request NSA's "assistance" on all matters related to civilian cryptography. In fact, were NIST and NSA representatives on the Technical Working Group to disagree on standards, the ultimate authority for resolving disputes would rest solely with the Executive Branch acting through the President, the Secretary of Defense and the National Security Council, thus undercutting the clear intent of Congress when they passed the 1987 Computer Security Act.
EPIC noted:

"The memorandum effectively returned to NSA many of the powers rejected by the Computer Security Act. The MOU contained several key goals that were to NSA's benefit, including: NSA providing NIST with technical security guidelines in trusted technology, telecommunications security, and personal identification that may be used in cost-effective systems for protecting sensitive computer data;' NSA initiating research and development programs in trusted technology, telecommunications security, cryptographic techniques and personal identification methods'; and NSA being responsive to NIST in all matters related to cryptographic algorithms and cryptographic techniques including but not limited to research, development, evaluation, or endorsement'."

A critique of the Memorandum in 1989 congressional testimony by the General Accounting Office (GAO) emphasized: "At issue is the degree to which responsibilities vested in NIST under the act are being subverted by the role assigned to NSA under the memorandum. The Congress, as a fundamental purpose in passing the act, sought to clearly place responsibility for the computer security of sensitive, unclassified information in a civil agency rather than in the Department of Defense. As we read the MOU, it would appear that NIST has granted NSA more than the consultative role envisioned in the act."
Five years after the GAO's critical appraisal, NSA's coup was complete.
"In 1994," EPIC noted,

"President Clinton issued Presidential Decision Directive (PDD-29). This directive created the Security Policy Board, which has recommended that all computer security functions for the government be merged under NSA control."

Since PDD-29 was issued matters have only gotten worse. In fact, NIST is the same outfit exposed in Snowden documents published by The Guardian and The New York Times that allowed NSA to water down encryption and build backdoors into the Dual EC DRBG standard adopted by the Institute in 2006.
"Eventually, NSA became the sole editor."
Besieged by widespread opposition, the Clinton administration was out maneuvered in the court of public opinion and by 1996 had abandoned Clipper. However, this proved to be a pyrrhic victory for security-minded researchers and civil libertarians as we have since learned from Edward Snowden's revelations.
Befitting a military-intelligence agency, the dark core of America's deep state, NSA was fighting a long warand they were playing for keeps.
http://www.globalresearch.ca/the-u-s-secret-state-and-the-internet-dirty-secrets-and-crypto-wars-from-clipper-chip-to-prism/5357623

David Guyatt Wrote:The British government response to the Guardian coverage does't surprise me at all. It's simply bending over to US pressure, wanting to be seen to be willing.

a quite horrible thought in reality, but standard fare these days.

Quote:

Surveillance technology out of control, says Lord Ashdown

Former Lib Dem leader says it is time for high-level inquiry to address fundamental questions about privacy in 21st century

• Nick Hopkins and Matthew Taylor
  
  
• The Guardian, Tuesday 19 November 2013 06.10 AEST
  

Paddy Ashdown is the latest senior politician to demand a review of the powers of Britain's intelligence agencies and the laws and oversight which underpin their activities. Photograph: Adrian Dennis/AFP/Getty Images

The technology used by Britain's spy agencies to conduct masssurveillance is "out of control", raising fears about the erosion of civil liberties at a time of diminished trust in the intelligence services, according to the former Liberal Democrat leader Lord Ashdown.
The peer said it was time for a high-level inquiry to address fundamental questions about privacy in the 21st century, and railed against "lazy politicians" who frighten people into thinking "al-Qaida is about to jump out from behind every bush and therefore it is legitimate to forget about civil liberties". "Well it isn't," he added.
Ashdown talks frequently to the deputy prime minister, Nick Clegg, and is chair of the the Liberal Democrats' general election team. Though he said he was speaking for himself, his views are understood to be shared by other senior members of the Liberal Democrats in government, who are also keen for some kind of broad inquiry into the subject.
This idea is also supported by Sir David Omand, a former director of GCHQ. He told the Guardian he was in favour of an inquiry and thought it would be wrong to "dismiss the idea of a royal commission out of hand". It was important to balance the need for the agencies to have powerful capabilities, and the necessity of ensuring they did not use them in a way parliament had not intended, Omand added.
Ashdown is the latest senior politician to demand a review of the powers of Britain's intelligence agencies GCHQ, MI5 and MI6 and the laws and oversight which underpin their activities.
In an interview with the Guardian, Ashdown said surveillance should only be conducted against specific targets when there was evidence against them. Dragnet surveillance was unacceptable, he added.
Ashdown made clear revelations in the Guardian about GCHQ and its American counterpart, the National Security Agency, had raised important issues that "could not be ignored or swept aside in a barrage of insults".
He also criticised the Labour party, which was in power when the agencies began testing and building many of their most powerful surveillance capabilities. Labour's former home secretary Jack Straw was responsible for introducing the Regulation of Investigatory Power Act 2000 (Ripa), which made the programmes legal.
"Ripa was a disgraceful piece of legislation," Ashdown said. "Nobody put any thought into it. Labour just took the words they were given by the intelligence agencies. I don't blame the intelligence agencies.
"We charge them with the very serious business of keeping us secure and of course they want to have powers. But it's the duty of government to ensure those powers don't destroy our liberties and Labour utterly failed to do this."
One consequence of Labour's negligence was the development of surveillance techniques that could damage civil liberties and erode privacy, said Ashdown.
He said that he was "frightened by the erosion of our liberties" and while accepting that there was a need to keep the nation safe it was the "habit of politicians who are lazy about the preservation of our liberties or don't mind seeing them destroyed, to play an old game.
"They tell frightened citizens: 'If you give me some of your liberties, I will make you safer'".
Ashdown said that as a young man in 1960s he was taken to a vast Post Office shed in central London where spies were steaming open letters. Recalling being met by "a deep fog of steam" after entering the room, he said that the place was "filled with diligent men and women, each with a boiling kettle on their desk, steaming open letters". It was appropriate for the state to intervene in the private communications of its citizens, but the peer added "only in cases where there is good evidence to believe the nation's security is being threatened, or arguably, when a really serious crime has been committed".
The former party leader said that intercepting communications needed to be "targeted on an individual and not classes of individuals or, as at the moment, the whole nation" and argued that ought to be sanctioned by a third-party, preferably by a judge, or if not a member of the cabinet.
Ashdown said he did not believe Britain's intelligence agencies were out of control, but he said the same was not true of technology.
"We need a proper inquiry to decide what liberties and privacies ought to be accorded in the new interconnected world, and what powers of intrusion ought to be given to the state. The old laws that applied in the age of the steaming kettle will no longer do. The old protections are no longer good enough," he said.
Ashdown said the Guardian's reporting of the NSA files had been "helpful because it had raised this important issue to the point where sensible people understand this inquiry is now necessary".
An inquiry also needed to be set in the context of people's privacy expectations, he added, noting: "People today seem more casual about their privacy than they used to be. They don't seem to mind when their privacy is breached when they use Google, Facebook and other social media."
He added that he hoped this had not "changed the public's attitude towards the state's power to intrude into their privacy" but argued this was the fundamental question that needed to be addressed.
Ashdown said he thought the agencies would welcome an inquiry too, saying that they "recognise the mechanisms are no longer sufficient" and he doubted whether such an exercise would be "inimical to the heads of the secret services".
The Lib Dem also dismissed the parliamentary Intelligence and Security Committee, chaired by Sir Malcolm Rifkind, which is supposed to scrutinise the agencies.
He said that it was an institution "wholly incapable of coping" with the new circumstances.
Although he was careful to be respectful of its Conservative chair, Ashdown argued that "we are no longer in the age when a grandee's emollient words are enough to assure us that our liberties are safe" and concluded that the committee was "past its time".
Ashdown defended the Guardian's reporting of the issues over the last five months, and the paper's right to publish material that it deemed in the public interest.
He said: "I am not going to back every single thing the Guardian has done. But overall, in my view, the Guardian has done a very important in job exposing a really important issue that must now be properly considered."
But he also criticised Edward Snowden, the former NSA contractor who leaked files to the Guardian, the Washington Post and Der Spiegel.
"When Snowden first broke cover, I had quite a lot of admiration for him. Here was a whistleblower breaking surface on an issue that is certainly important. But I have to say that the way he has behaved since has diminished that admiration enormously. It seems to me this is becoming more about vanity."
Meanwhile, Omand said the ISC had to be given a chance to review the work of the agencies in an inquiry that it announced last month.

Quote:N.S.A. Report Outlined Goals for More Power

By JAMES RISEN and LAURA POITRAS

Published: November 22, 2013


WASHINGTON Officials at the National Security Agency, intent on maintaining its dominance in intelligence collection, pledged last year to push to expand its surveillance powers, according to a top secret strategy document.

In a February 2012 paper laying out the four-year strategy for the N.S.A.'s signals intelligence operations, which include the agency's eavesdropping and communications data collection around the world, agency officials set an objective to "aggressively pursue legal authorities and a policy framework mapped more fully to the information age."
Written as an agency mission statement with broad goals, the five-page document said that existing American laws were not adequate to meet the needs of the N.S.A. to conduct broad surveillance in what it cited as "the golden age of Sigint," or signals intelligence. "The interpretation and guidelines for applying our authorities, and in some cases the authorities themselves, have not kept pace with the complexity of the technology and target environments, or the operational expectations levied on N.S.A.'s mission," the document concluded.
Using sweeping language, the paper also outlined some of the agency's other ambitions. They included defeating the cybersecurity practices of adversaries in order to acquire the data the agency needs from "anyone, anytime, anywhere." The agency also said it would try to decrypt or bypass codes that keep communications secret by influencing "the global commercial encryption market through commercial relationships," human spies and intelligence partners in other countries. It also talked of the need to "revolutionize" analysis of its vast collections of data to "radically increase operational impact."
The strategy document, provided by the former N.S.A. contractor Edward J. Snowden, was written at a time when the agency was at the peak of its powers and the scope of its surveillance operations was still secret. Since then, Mr. Snowden's revelations have changed the political landscape.
Prompted by a public outcry over the N.S.A.'s domestic operations, the agency's critics in Congress have been pushing to limit, rather than expand, its ability to routinely collect the phone and email records of millions of Americans, while foreign leaders have protested reports of virtually unlimited N.S.A. surveillance overseas, even in allied nations. Several inquiries are underway in Washington; Gen. Keith B. Alexander, the N.S.A.'s longest-serving director, has announced plans to retire; and the White House has offered proposals to disclose more information about the agency's domestic surveillance activities.
The N.S.A. document, titled "Sigint Strategy 2012-2016," does not make clear what legal or policy changes the agency might seek. The N.S.A.'s powers are determined variously by Congress, executive orders and the nation's secret intelligence court, and its operations are governed by layers of regulations. While asserting that the agency's "culture of compliance" would not be compromised, N.S.A. officials argued that they needed more flexibility, according to the paper.
Senior intelligence officials, responding to questions about the document, said that the N.S.A. believed that legal impediments limited its ability to conduct surveillance of suspected terrorists inside the United States. Despite an overhaul of national security law in 2008, the officials said, if a terrorism suspect who is under surveillance overseas enters the United States, the agency has to stop monitoring him until it obtains a warrant from the Foreign Intelligence Surveillance Court.
"N.S.A.'s Sigint strategy is designed to guide investments in future capabilities and close gaps in current capabilities," the agency said in a statement. "In an ever-changing technology and telecommunications environment, N.S.A. tries to get in front of issues to better fulfill the foreign-intelligence requirements of the U.S. government."
Critics, including some congressional leaders, say that the role of N.S.A. surveillance in thwarting terrorist attacks often cited by the agency to justify expanded powers has been exaggerated. In response to the controversy about its activities after Mr. Snowden's disclosures, agency officials claimed that the N.S.A.'s sweeping domestic surveillance programs had helped in 54 "terrorist-related activities." But under growing scrutiny, congressional staff members and other critics say that the use of such figures by defenders of the agency has dramatically overstated the value of the domestic surveillance programs in counterterrorism.
Agency leaders believe that the N.S.A. has never enjoyed such a target-rich environment as it does now because of the global explosion of digital information and they want to make certain that they can dominate "the Sigint battle space" in the future, the document said. To be "optimally effective," the paper said, "legal, policy and process authorities must be as adaptive and dynamic as the technological and operational advances we seek to exploit."
Intent on unlocking the secrets of adversaries, the paper underscores the agency's long-term goal of being able to collect virtually everything available in the digital world. To achieve that objective, the paper suggests that the N.S.A. plans to gain greater access, in a variety of ways, to the infrastructure of the world's telecommunications networks.
Reports based on other documents previously leaked by Mr. Snowden showed that the N.S.A. has infiltrated the cable links to Google and Yahoo data centers around the world, leading to protests from company executives and a growing backlash against the N.S.A. in Silicon Valley.
Yet the paper also shows how the agency believes it can influence and shape trends in high-tech industries in other ways to suit its needs. One of the agency's goals is to "continue to invest in the industrial base and drive the state of the art for high performance computing to maintain pre-eminent cryptanalytic capability for the nation." The paper added that the N.S.A. must seek to "identify new access, collection and exploitation methods by leveraging global business trends in data and communications services."
And it wants to find ways to combine all of its technical tools to enhance its surveillance powers. The N.S.A. will seek to integrate its "capabilities to reach previously inaccessible targets in support of exploitation, cyberdefense and cyberoperations," the paper stated.
The agency also intends to improve its access to encrypted communications used by individuals, businesses and foreign governments, the strategy document said. The N.S.A. has already had some success in defeating encryption, The New York Times has reported, but the document makes it clear that countering "ubiquitous, strong, commercial network encryption" is a top priority. The agency plans to fight back against the rise of encryption through relationships with companies that develop encryption tools and through espionage operations. In other countries, the document said, the N.S.A. must also "counter indigenous cryptographic programs by targeting their industrial bases with all available Sigint and Humint" human intelligence, meaning spies.
The document also mentioned a goal of integrating the agency's eavesdropping and data collection systems into a national network of sensors that interactively "sense, respond and alert one another at machine speed." Senior intelligence officials said that the system of sensors is designed to protect the computer networks of the Defense Department, and that the N.S.A. does not use data collected from Americans for the system.
One of the agency's other four-year goals was to "share bulk data" more broadly to allow for better analysis. While the paper does not explain in detail how widely it would disseminate bulk data within the intelligence community, the proposal raises questions about what safeguards the N.S.A. plans to place on its domestic phone and email data collection programs to protect Americans' privacy.
N.S.A. officials have insisted that they have placed tight controls on those programs. In an interview, the senior intelligence officials said that the strategy paper was referring to the agency's desire to share foreign data more broadly, not phone logs of Americans collected under the Patriot Act.
Above all, the strategy paper suggests the N.S.A.'s vast view of its mission: nothing less than to "dramatically increase mastery of the global network."
Other N.S.A. documents offer hints of how the agency is trying to do just that. One program, code-named Treasure Map, provides what a secret N.S.A. PowerPoint presentation describes as "a near real-time, interactive map of the global Internet." According to the undated PowerPoint presentation, disclosed by Mr. Snowden, Treasure Map gives the N.S.A. "a 300,000 foot view of the Internet."
Relying on Internet routing data, commercial and Sigint information, Treasure Map is a sophisticated tool, one that the PowerPoint presentation describes as a "massive Internet mapping, analysis and exploration engine." It collects Wi-Fi network and geolocation data, and between 30 million and 50 million unique Internet provider addresses code that can reveal the location and owner of a computer, mobile device or router are represented each day on Treasure Map, according to the document. It boasts that the program can map "any device, anywhere, all the time."
The documents include addresses labeled as based in the "U.S.," and because so much Internet traffic flows through the United States, it would be difficult to map much of the world without capturing such addresses.
But the intelligence officials said that Treasure Map maps only foreign and Defense Department networks, and is limited by the amount of data available to the agency. There are several billion I.P. addresses on the Internet, the officials said, and Treasure Map cannot map them all. The program is not used for surveillance, they said, but to understand computer networks.
The program takes advantage of the capabilities of other secret N.S.A. programs. To support Treasure Map, for example, the document states that another program, called Packaged Goods, tracks the "traceroutes" through which data flows around the Internet. Through Packaged Goods, the N.S.A. has gained access to "13 covered servers in unwitting data centers around the globe," according to the PowerPoint. The document identifies a list of countries where the data centers are located, including Germany, Poland, Denmark, South Africa and Taiwan as well as Russia, China and Singapore.
Despite the document's reference to "unwitting data centers," government officials said that the agency does not hack into those centers. Instead, the officials said, the intelligence community secretly uses front companies to lease space on the servers.
Despite the N.S.A.'s broad surveillance powers, the strategy paper shows that N.S.A. officials still worry about the agency's ability to fend off bureaucratic inertia while keeping pace with change. "To sustain current mission relevance," the document said, Signals Intelligence Directorate, the N.S.A.'s signals intelligence arm, "must undertake a profound and revolutionary shift from the mission approach which has served us so well in the decades preceding the onset of the information age."
James Risen reported from Washington, and Laura Poitras from Berlin.