Saturday, November 21, 2009

An Analysis of the Hacked Global Warming E-mails

I'm sure everyone has heard about the hacked emails of the scientists are RealClimate and the University of East Anglia, and the charges that they reveal some sort of conspiracy to manufacture a crisis and hide the truth. However, the emails I've seen do nothing of the sort. They simply show ignorance of the process of analyzing and presenting data on complicated subjects, and attempt to portray out-of-context quotes in a manner that belies their actual meaning. As an actuary who deals with this sort of thing all the time, I recognize the language well. Let's go through some examples touted by a commenter here as smoking guns:

"I’ve attached a cleaned-up and commented version of the matlab code that I wrote for doing the Mann and Jones (2003) composites. I did this knowing that Phil and I are likely to have to respond to more crap criticisms from the idiots in the near future, so best to clean up the code and provide to some of my close colleagues in case they want to test it, etc. Please feel free to use this code for your own internal purposes, but don’t pass it along where it may get into the hands of the wrong people."

I must say, after the intro to this as clear documentation as fraud, I was very let down. What exactly is the issue here: cleaning up code, testing it, anticipating criticisms, preparing for same, sharing information. All part and parcel of data analysis and presentation. I can only guess that the problem is with the last sentence, which the conspiratorial-minded might automatically assume means "hide the truth from all but those in on the scam". In actuality, it's far more likely to mean something like "don't let morons who won't understand what they are looking at get ahold of this, or we'll be answering stupid assed ignorant questions til the cows come home, and we won't get anything productive done." Contrary to the worldview of the Palinists, some subjects are far too complicated for untrained laymen, and data analysis is one of them. Making sure one's data is properly limited and presented properly is par for the course. It's no evidence of anything except people doing their job.

"Perhaps we'll do a simple update to the Yamal post, e.g. linking Keith/s new page--Gavin t? As to the issues of robustness, particularly w.r.t. inclusion of the Yamal series, we actually emphasized that (including the Osborn and Briffa '06 sensitivity test) in our original post! As we all know, this isn't about truth at all, its about plausibly deniable accusations."

All I can say about this one is that it doesn't do much for the charge that conservatives have no sense of humor. Anyone who can't see that last sentence is sarcastic is seriously deluded. It's no surprise that the preceding context is left out, it no doubt makes that even more clear than it already is.

"Anyway, I wanted you guys to know that you’re free to use RC [] Rein any way you think would be helpful. Gavin and I are going to be careful about what comments we screen through, and we’ll be very careful to answer any questions that come up to any extent we can. On the other hand, you might want to visit the thread and post replies yourself. We can hold comments up in the queue and contact you about whether or not you think they should be screened through or not, and if so, any comments you’d like us to include."

Again, I ask, "where's the beef?" Surely no rightwinger is going to claim that moderating a site is ipso facto evidence of dishonesty, given that conservative sites that heavily moderate dwarf those of the left by an order of magnitude. There's nothing here to indicate fraud or suppression of data. Now of course if one is already convinced that is the case, then it will look damning, but otherwise it looks like nothing more than a discussion of how to handle responding to comments, and being careful to properly screen them. I could see myself saying the exact same things.

"The Korttajarvi record was oriented in the reconstruction in the way that McIntyre said. I took a look at the original reference – the temperature proxy we looked at is x-ray density, which the author interprets to be inversely related to temperature. We had higher values as warmer in the reconstruction, so it looks to me like we got it wrong, unless we decided to reinterpret the record which I don’t remember. Darrell, does this sound right to you?"

This one is even more of a mystery to me. This looks like a standard (and rather dull) discussion of results and modelling of data. I can only surmise that this is considered revealing by those who are completely ignorant of the process of guess and test that is the core of science.

"I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) amd from 1961 for Keith’s to hide the decline."

As has been explained in many places already, "trick" in this context means "useful technique", not "deception". Another word that springs to mind is "reach".

"The skeptics seem to be building up a head of steam here! ... The IPCC comes in for a lot of stick. Leave it to you to delete as appropriate! Cheers Phil
PS I’m getting hassled by a couple of people to release the CRU station temperature data. Don’t any of you three tell anybody that the UK has a Freedom of Information Act!"


"If FOIA does ever get used by anyone, there is also IPR to consider as well. Data is covered by all the agreements we sign with people, so I will be hiding behind them."

I know this may come as a shock to people who don't work in information-related, politically controversial, regulated fields, but all this FOIA stuff is easy to explain: no one, at any job that deals with complicated subject matter, willingly gives up all of their data. It makes life harder, even when you are 100% correct and proper. Overzealous regulators, politicians, or activists, will misrepresent or outright lie about what the data is/says, and you get stuck cleaning up the mess. This fiasco over what we are seeing is really nothing is a perfect example! This is just creationist quote mining techniques applied to global warming.

"The fact is that we can’t account for the lack of warming at the moment and it is a travesty that we can’t. The CERES data published in the August BAMS 09 supplement on 2008 shows there should be even more warming: but the data are surely wrong. Our observing system is inadequate."

I find it amusing that scientists are on the one hand condemned for supposedly thinking they know everything, and then when they make qualifying statements about their limited knowledge, or openly admit when data is missing or contradictory, they are expected to pretend that means they know nothing. What is this quote but such an admission, in a very limited sense? Again, there is no there there.

All these emails prove is that scientists are human. The deniers and their ilk see all this as damning because they begin with the assumption that there is some sort of fraud going on, so to them anything like screening of comments is automatically assumed to be to keep the truth out, rather than, say, keeping morons and trolls from dragging the discussion down. Let's also keep in mind that this all comes as far back as 12 years ago, from a second-rate university better known for its literary programs than its science. It's akin to arguing that we should reject evolution because some researchers somewhere said doubtful things about it and their analysis of it. That doesn't change all the evidence. Bottom line, there is nothing here that will persuade the ideologically unencumbered or the statistically educated. Sadly, that isn't who they are playing to.

I found the Realclimate response very well written, and I'll close with what I thought was the crux of the issue as they put it:

"More interesting is what is not contained in the emails. There is no evidence of any worldwide conspiracy, no mention of George Soros nefariously funding climate research, no grand plan to ‘get rid of the MWP’, no admission that global warming is a hoax, no evidence of the falsifying of data, and no ‘marching orders’ from our socialist/communist/vegetarian overlords. The truly paranoid will put this down to the hackers also being in on the plot though.

Instead, there is a peek into how scientists actually interact and the conflicts show that the community is a far cry from the monolith that is sometimes imagined. People working constructively to improve joint publications; scientists who are friendly and agree on many of the big picture issues, disagreeing at times about details and engaging in ‘robust’ discussions; Scientists expressing frustration at the misrepresentation of their work in politicized arenas and complaining when media reports get it wrong; Scientists resenting the time they have to take out of their research to deal with over-hyped nonsense. None of this should be shocking."

Indeed, none of it should be. The deniers would have us believe they've scored big points because they've shown scientists have all the same foibles as everyone else: biases, self-interests, emotional attachment to ideas, and a desire to avoid conflict. What they don't understand is that the scientific community has always assumed this to be the case, which is why they so emphasize public publishing of data and research, so others can have at one's biases. So in the end, the AGW deniers have proved only one thing: they don't understand science in the abstract, or the modern scientific enterprise as practiced.


Anonymous said...

Way to go Mr. Avenger. Sorry you had to take on our token climate denier over at 538. It's actually a lost cause. But good effort.

ScienceAvenger said...

Thanks. Amazing what people can see when its all they want to see, eh? The mind is a terrible thing to waste on conspiracies.

Miranda said...

I'm a climate denier, Mr. Universe. I look outside my window and see no climate at all!

Troublesome Frog said...

Some of these lines sound like things that I write all the time in my work. I do software R&D in a field that involves probability and statistics for security systems.

I freely pass data back and forth with my colleagues, but the moment somebody from marketing shows up and asks for the same thing, I freeze up. Nothing gets the ball rolling in the wrong direction faster and more dangerously than somebody with only half a clue about the data sending emails everywhere.

The fact that I'm reluctant to give semi-technical people my data and calculations has nothing to do with my being uncertain about my work. It has everything to do with the fact that when somebody sends out an alarmist email, the people who decide whether or not to panic don't necessarily have the background to distinguish between misinterpretation of the data and a genuine problem.

Of course, I'm not working in an environment where people just want my data so they can convict me in the court of public opinion and then cancel my program. If it was that bad, I'd be even harder to pin down.

hector said...

"George Monbiot, a writer and environmental activist many consider to be Great Britain's Al Gore" says: "there's "no use pretending that this isn’t a major blow. The emails extracted by a hacker from the climatic research unit at the University of East Anglia could scarcely be more damaging"

ScienceAvenger said...

As is typical of dishonest quoteminers, you leave off the complete quote that shows he's talking in terms of PR, not science:

"But do these revelations justify the sceptics' claims that this is 'the final nail in the coffin' of global warming theory? Not at all. They damage the credibility of three or four scientists. They raise questions about the integrity of one or perhaps two out of several hundred lines of evidence. To bury man-made climate change, a far wider conspiracy would have to be revealed."

He then goes on to show what a truly damning email would look like. I encourage everyone to go look at it, and see by contrast how lame the hacked emails are, and how desperate the deniers are to paint this as some sort of major victory.

hector said...

I was talking in terms of PR, too. It was obvious that Monbiot wouldn't change his opinion on GW. Too obvious to mention. You didn't even go that far (i.e. mentioning PR) in your initial post.

ScienceAvenger said...

That's because this is a site about science, not PR, and because the PR problem is only because of the ignorance and deception going on out there in the denialist world. It's all contrived, and unworthy of serious thought.

Veranda said...

""I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) amd from 1961 for Keith’s to hide the decline."

As has been explained in many places already, "trick" in this context means "useful technique", not "deception". Another word that springs to mind is "reach"."

Hmmm, a "useful technique" to HIDE THE DECLINE. Explain that, please.

ScienceAvenger said...

Explain what? Without the full context of the comments, there's really nothing to explain, because there's no way to be clear on what they are talking about. That's the big point here.

Veranda said...

No way to be clear? Then why are you so confident that "trick" merely means "useful technique" and not deception?

ScienceAvenger said...

For the same reasons I am confident that when a football coach says "fake" he doesn't mean deception. That just isn't the way the words are used in the broader context in either case.

As usual the deniers have the burdon of proof backwards. No one out here has to "explain" anything. If you think you have a case that there is some sort of deception going on here, and that somehow this casts doubt on the entirety of the AGW hypothesis, then by all means make the argument and reference the supporting data/commentary in its entirety. Merely tossing around out-of-context quotes and barking "explain that" doesn't cut it.

Veranda said...

Excerpts of New Senate Climategate Report