07-12-2009, 09:47 PM
http://erratasec.blogspot.com/2009/11/ha...rming.html
Friday, November 20, 2009
Hacker exposes global warming researcher (Climategate)
Posted by Robert Graham at 4:47 PM
Hackers broke in and revealed the private e-mails of Phil Jones (NYTimes, BBC ), a famous climatologist. This is going to be one of the most politically relevant hacks of the last few years. When hackers broke into Sarah Palin's e-mails during the presidential campaign, they failed to find any interesting dirt. Phil Jones' e-mails, though, are full of dirt. There's no proof of a "conspiracy" or "cover-up", but a lot of the e-mails look bad for Jones and some of his fellow researchers.
As a cybersecurity expert and a climate skeptic, I thought I'd give some background on what happened.
THE CLIMATE DEBATE
Climate skeptics accept the fact that CO2 is a major greenhouse gas and that mankind has produced a lot more of it. However, the effect is logarithmic, suffering decreasing marginal returns, which means that even when we double CO2 in the atmosphere around the year 2100, by itself, CO2 will only increase temperatures by 1 degree. Global warming alarmism is based on the idea that the atmosphere is unstable, with reinforcing feedbacks, and "sensitive" to changes. Warmer air holds more water vapor which holds in more heat, which in turn warms the air further. Ice reflects heat, and when some melts do to slight warming, it exposed rock which absorbs heat, causing even more warming.
That's the debate in a nutshell. Alarmists think climate sensitivity is large, skeptics think it's small.
There are two major proofs for alarmism: computer models and historic reconstructions.
Computer models attempt to reproduce the entire climate. They are hugely complex, trying to incorporate everything we know about the climate. Models show warmer air holding more water vapor causing a positive feedback. Models require supercomputers to run. It is experimentation with computer models that "proves" climate sensitivity is large. Climate skeptics think computer models are bogus, that they do not replace experimental evidence, and that the billions of dollars spent experimenting with models would be better spent experimenting with the atmosphere. I'm a skeptic because I understand computer models, how they can be deceptive, and that the IPCC's reliance on them is unwarranted.
Historic reconstructions try to figure out what the temperature has been in the past compared to the present. They try to answer the question whether the current warming is normal or unprecedented. During the "Medieval Warming Period" a thousand years ago, Greenland was green, and Europe was warmer than it is now. Was that a localized climate phenomenon, or was it global? Skeptics think it was global, alarmists think it was local, and therefore the planet is warmer now than any time in the last several thousand years.
Historic temperatures are constructed from a wide variety of sources, such as ratios of isotopes, widths of tree rings, contents of sediments, and so on. Climatologists rely upon statistics to reconstruct useful information from what is otherwise a chaotic jumble of data.
The problem with these reconstructions is that the authors do not release their data to the public. Skeptics can't review the raw data or methodology and challenge the results. This is not unique to climate science, other scientific disciplines have similar data sharing problems. Collecting the data, such as measuring the rings of thousands of trees, takes a lot of work, and a scientist can make many discoveries from the data set. They only want to release the data after they've spent a few years writing papers based on it. Another problem is that it takes a lot of work to archive and maintain the data for critics who want to challenge it years later.
So, the lack of reproducibility doesn't mean there's a conspiracy or malfeasance, but it does mean that the science isn't "settled" (as Al Gore claims). It cannot be settled until critics have had a chance to rebut the claims. The lack of reproducibility gives the skeptics too much credibility (if they are wrong), or not enough credibility (if they are right).
At the center of the reproducibility debate is Stephen McIntyre and his website ClimateAudit. McIntyre documents his struggles to obtain the raw data and reproduce the results of famous temperature reconstructions. When he finally gets his hands on the data, often years later, he's often successful at finding important flaws.
The targets of McIntyre's campaign have their own website, RealClimate. They stress that their website is by "climate researchers", and important point because a lot of critics are not (McIntyre is a statistician).
Phil Jones, the guy whose e-mails were hacked, is part of the RealClimate crowd, and the target of McIntyre's attempts to reproduce results. Phil Jones runs that UK's University of East Anglia Climate Research Unit (CRU). CRU maintains global temperature records from the present going back to the 1800s. It also holds the data for a lot of reconstructions, especially the reconstructions used by the UN's IPCC (Intergovernmental Panel on Climate Change). McIntyre wants to challenge the IPCC's conclusions, but he can't, because Phil Jones refuses to release the data.
THE HACK
Somebody(s), we don't know who, stole a thousand confidential e-mails of Phil Jones, head the CRU. They also stole some important raw data used in many climate research papers. They put it in a 61-megabyte ZIP file and posted it to an anonymous FTP server in Russia. The posting was accompanied with the note:
We feel that climate science is, in the current situation, too important to be kept under wraps.
We hereby release a random selection of correspondence, code, and documents
All evidence points to this as being genuine. Phil Jones has admitted that one of pieces of dirt, about "hiding" a warming trend, is indeed genuine, although misinterpreted and out of context.
CRU has canceled everyone's passwords, forcing everyone to choose new passwords. This hints they have logs showing a specific account accessing the data (possibly Jones' own account).
Nobody knows how it happened. We are unlikely to figure out how the hack occurred unless we discover who did it.
It's a roughly 80% chance it was done by some sort of "insider", by somebody who has at least partial access to one of the internal computers. There is only a 20% chance it was done by an outside hacker breaking in. (This is my gut feel as a security researcher, there is no robustness in this estimate). Universities are more easily hacked by outsiders than most networks, ironically because Universities have a culture of sharing data.
UPDATE: A wag suggested this: Phil Jones and crew were likely logging onto their accounts using an open wifi at a climate conference. If you wanted to break into climatologist's e-mail, that'd be the easiest way to do it.
UPDATE: The hacker used open proxies to post the content, hiding his/her IP address.
UPDATE: Phil Jones is quoted as saying "It was a hacker. We were aware of this about three or four days ago". He's referring to the first attempt by the hacker to post the data to RealClimate.
The data is oddly specific. Only Phil Jones e-mails were copied, and a lot of the data that was hacked is specific to certain climate controversies. If it wasn't an insider, it was certainly somebody familiar with the central debate about reproducibility of climate reconstructions.
The fact that they posted the data to an anonymous FTP site in Russia also points to somebody who is active in the hacking community. This narrows things down. I suspect that at the end of the data, they'll find some sort of computer administrator working for CRU.
This hack is not simply about global warming, but the ethics of hacking.
This is similar to the Palin hack. If you'll remember, Alaska had rules that all e-mails must be archived, for the purposes of making government transparent. Palin conducted official business through her Yahoo account, evading these rules. It wasn't done maliciously, the hack actually proved there was nothing being covered up. Yet, it was still a violation of the rules.
In much the same way, Phil Jones is hiding data. It's bad science, it's bad politics. Again, there is no conspiracy or cover up here. Jones passionately hates McIntyre, he doesn't like skeptics, and he doesn't want to go through a lot of work to help skeptics. (Actually, I feel for Jones: we often come across virus samples, we send them around to people we like who ask, but we are too lazy to make them more widely available).
I think hackers do the world a favor by making things publicly available that should have been available in the first place. I believe "transparency" is fundamental to our political system, and if they politicians fail to be transparent, hackers should force the issue.
On the other hand, I'd like to see the hacker come forward publicly and admit the deed. It's a bad principle for hackers to decide for themselves when it's right or wrong to hack. This because hackers always have a justification for their hacks, only rarely are they going to find that others agree with them. It's like the question whether you'd kill Hitler before he could cause WW II and the Holocaust. My answer is that I would - but I'd expect to be convicted of murder and sent to jail. It's like how Henry David Thoreau practiced civil disobedience: he refused to pay taxes, and expected to go to jail (and was annoyed when his friends released him from jail by paying his taxes). People who hide from the government in order to avoid taxes are douchebags, people who stand up for principles are heroes.
Friday, November 20, 2009
Hacker exposes global warming researcher (Climategate)
Posted by Robert Graham at 4:47 PM
Hackers broke in and revealed the private e-mails of Phil Jones (NYTimes, BBC ), a famous climatologist. This is going to be one of the most politically relevant hacks of the last few years. When hackers broke into Sarah Palin's e-mails during the presidential campaign, they failed to find any interesting dirt. Phil Jones' e-mails, though, are full of dirt. There's no proof of a "conspiracy" or "cover-up", but a lot of the e-mails look bad for Jones and some of his fellow researchers.
As a cybersecurity expert and a climate skeptic, I thought I'd give some background on what happened.
THE CLIMATE DEBATE
Climate skeptics accept the fact that CO2 is a major greenhouse gas and that mankind has produced a lot more of it. However, the effect is logarithmic, suffering decreasing marginal returns, which means that even when we double CO2 in the atmosphere around the year 2100, by itself, CO2 will only increase temperatures by 1 degree. Global warming alarmism is based on the idea that the atmosphere is unstable, with reinforcing feedbacks, and "sensitive" to changes. Warmer air holds more water vapor which holds in more heat, which in turn warms the air further. Ice reflects heat, and when some melts do to slight warming, it exposed rock which absorbs heat, causing even more warming.
That's the debate in a nutshell. Alarmists think climate sensitivity is large, skeptics think it's small.
There are two major proofs for alarmism: computer models and historic reconstructions.
Computer models attempt to reproduce the entire climate. They are hugely complex, trying to incorporate everything we know about the climate. Models show warmer air holding more water vapor causing a positive feedback. Models require supercomputers to run. It is experimentation with computer models that "proves" climate sensitivity is large. Climate skeptics think computer models are bogus, that they do not replace experimental evidence, and that the billions of dollars spent experimenting with models would be better spent experimenting with the atmosphere. I'm a skeptic because I understand computer models, how they can be deceptive, and that the IPCC's reliance on them is unwarranted.
Historic reconstructions try to figure out what the temperature has been in the past compared to the present. They try to answer the question whether the current warming is normal or unprecedented. During the "Medieval Warming Period" a thousand years ago, Greenland was green, and Europe was warmer than it is now. Was that a localized climate phenomenon, or was it global? Skeptics think it was global, alarmists think it was local, and therefore the planet is warmer now than any time in the last several thousand years.
Historic temperatures are constructed from a wide variety of sources, such as ratios of isotopes, widths of tree rings, contents of sediments, and so on. Climatologists rely upon statistics to reconstruct useful information from what is otherwise a chaotic jumble of data.
The problem with these reconstructions is that the authors do not release their data to the public. Skeptics can't review the raw data or methodology and challenge the results. This is not unique to climate science, other scientific disciplines have similar data sharing problems. Collecting the data, such as measuring the rings of thousands of trees, takes a lot of work, and a scientist can make many discoveries from the data set. They only want to release the data after they've spent a few years writing papers based on it. Another problem is that it takes a lot of work to archive and maintain the data for critics who want to challenge it years later.
So, the lack of reproducibility doesn't mean there's a conspiracy or malfeasance, but it does mean that the science isn't "settled" (as Al Gore claims). It cannot be settled until critics have had a chance to rebut the claims. The lack of reproducibility gives the skeptics too much credibility (if they are wrong), or not enough credibility (if they are right).
At the center of the reproducibility debate is Stephen McIntyre and his website ClimateAudit. McIntyre documents his struggles to obtain the raw data and reproduce the results of famous temperature reconstructions. When he finally gets his hands on the data, often years later, he's often successful at finding important flaws.
The targets of McIntyre's campaign have their own website, RealClimate. They stress that their website is by "climate researchers", and important point because a lot of critics are not (McIntyre is a statistician).
Phil Jones, the guy whose e-mails were hacked, is part of the RealClimate crowd, and the target of McIntyre's attempts to reproduce results. Phil Jones runs that UK's University of East Anglia Climate Research Unit (CRU). CRU maintains global temperature records from the present going back to the 1800s. It also holds the data for a lot of reconstructions, especially the reconstructions used by the UN's IPCC (Intergovernmental Panel on Climate Change). McIntyre wants to challenge the IPCC's conclusions, but he can't, because Phil Jones refuses to release the data.
THE HACK
Somebody(s), we don't know who, stole a thousand confidential e-mails of Phil Jones, head the CRU. They also stole some important raw data used in many climate research papers. They put it in a 61-megabyte ZIP file and posted it to an anonymous FTP server in Russia. The posting was accompanied with the note:
We feel that climate science is, in the current situation, too important to be kept under wraps.
We hereby release a random selection of correspondence, code, and documents
All evidence points to this as being genuine. Phil Jones has admitted that one of pieces of dirt, about "hiding" a warming trend, is indeed genuine, although misinterpreted and out of context.
CRU has canceled everyone's passwords, forcing everyone to choose new passwords. This hints they have logs showing a specific account accessing the data (possibly Jones' own account).
Nobody knows how it happened. We are unlikely to figure out how the hack occurred unless we discover who did it.
It's a roughly 80% chance it was done by some sort of "insider", by somebody who has at least partial access to one of the internal computers. There is only a 20% chance it was done by an outside hacker breaking in. (This is my gut feel as a security researcher, there is no robustness in this estimate). Universities are more easily hacked by outsiders than most networks, ironically because Universities have a culture of sharing data.
UPDATE: A wag suggested this: Phil Jones and crew were likely logging onto their accounts using an open wifi at a climate conference. If you wanted to break into climatologist's e-mail, that'd be the easiest way to do it.
UPDATE: The hacker used open proxies to post the content, hiding his/her IP address.
UPDATE: Phil Jones is quoted as saying "It was a hacker. We were aware of this about three or four days ago". He's referring to the first attempt by the hacker to post the data to RealClimate.
The data is oddly specific. Only Phil Jones e-mails were copied, and a lot of the data that was hacked is specific to certain climate controversies. If it wasn't an insider, it was certainly somebody familiar with the central debate about reproducibility of climate reconstructions.
The fact that they posted the data to an anonymous FTP site in Russia also points to somebody who is active in the hacking community. This narrows things down. I suspect that at the end of the data, they'll find some sort of computer administrator working for CRU.
This hack is not simply about global warming, but the ethics of hacking.
This is similar to the Palin hack. If you'll remember, Alaska had rules that all e-mails must be archived, for the purposes of making government transparent. Palin conducted official business through her Yahoo account, evading these rules. It wasn't done maliciously, the hack actually proved there was nothing being covered up. Yet, it was still a violation of the rules.
In much the same way, Phil Jones is hiding data. It's bad science, it's bad politics. Again, there is no conspiracy or cover up here. Jones passionately hates McIntyre, he doesn't like skeptics, and he doesn't want to go through a lot of work to help skeptics. (Actually, I feel for Jones: we often come across virus samples, we send them around to people we like who ask, but we are too lazy to make them more widely available).
I think hackers do the world a favor by making things publicly available that should have been available in the first place. I believe "transparency" is fundamental to our political system, and if they politicians fail to be transparent, hackers should force the issue.
On the other hand, I'd like to see the hacker come forward publicly and admit the deed. It's a bad principle for hackers to decide for themselves when it's right or wrong to hack. This because hackers always have a justification for their hacks, only rarely are they going to find that others agree with them. It's like the question whether you'd kill Hitler before he could cause WW II and the Holocaust. My answer is that I would - but I'd expect to be convicted of murder and sent to jail. It's like how Henry David Thoreau practiced civil disobedience: he refused to pay taxes, and expected to go to jail (and was annoyed when his friends released him from jail by paying his taxes). People who hide from the government in order to avoid taxes are douchebags, people who stand up for principles are heroes.
"History records that the Money Changers have used every form of abuse, intrigue, deceit and violent means possible to maintain their control over governments by controlling money and its issuance." --James Madison