Thursday, August 09, 2007

Schmidt Responds To "Problem" In GISTEMP U.S. Temperature Data

Gavin Schmidt, a climate modeller at the Goddard Institute for Space Studies, and a regular contributor to the Real Climate blog, has given a quick response to Steve McIntyre's discovery of issues with the Global Historical Climatology Network's data. As noted here, this discovery has led the GHCN to re-organize its U.S. temperature rankings so that a number of recent years come out cooler than originally recorded, while several years in the 1930s come out hotter. What effect does this adjustment have on the overall case for AGW? Mr. Schmidt writes in the comments:

McIntyre noticed that there was an odd offset in the GISTEMP analysis in 2000 which turned out to be related to the transition between USHCN data to the GHCN data. The offset occurred because the USHCN corrections (for Time of Observation bias mainly) affect the more recent values in USHCN but not GHCN (as opposed to only affecting earlier values). Once notified of the problem, GISS investigated immediately, found the error, and added an extra step to the analysis to remove any jump at the transition. This only affected the US temperatures (reducing the mean by about 0.15 ºC in 2000-2006), but since the US is such a small part of the world, it doesn’t effect the global temperatures. Note that this wasn’t a problem with the USHCN data - rather in how the different data sources are melded. It also had nothing to do with any micro-site issues. - gavin]

Furthermore, when a commentator stated:

With this error, it isn’t a small percentage of the world’s temperature that is the issue. It is that the US data is a large percentage of the data available.

Schmidt replied:

[Response: But the global average isn’t simply an average of all the stations divided by the number of stations. You need to adjust for area so that a high concentration of stations in one spot doesn’t bias the mean. If the US is 2% of the area, then a 0.15 ºC correction there implies only a 0.15*0.02=0.003 ºC correction to the global mean (though it’s actually a little higher because of incomplete global coverage). These things should obviously be fixed, but the implications need to be kept in perspective. - gavin]

I suspect that Mr. Schmidt is correct in his assessment, and that the adjustment to U.S. data will only marginally effect the global mean, making the "blade" of the hockey-stick graph infinitesimally less steep.

Which means that Steve Ms work is less a problem for the Science of Global Warming and more a problem for the Politics and PR end of the issue. And here it is a fairly serious concern, because who knows how many more months of delay and inaction it might give rise to? The most obvious argument his work will inspire is: how can we trust ANY of the data, now that we have discovered this particular flaw in in this particular data-set (you might notice how people are already trying to conflate McIntyre's research with the "study" of microsite issues being undertaken by Anthony Watts and the gang at Surface Stations). Secondly, while the U.S. temperature data is only a small slice of the global pie, the fact that U.S. temperatures have not risen as much as those elsewhere may well inspire an "I'm alright Jack" response from American Pols and citizens. Who cares, in other words, if its only Asians and Africans that get fried?

And of course that would be a bad thing.

32 comments:

Ti-Guy said...

But the global average isn’t simply an average of all the stations divided by the number of stations.

I'm kind of wondering how many of the garden-variety denier trolls approach the measurement of temperature (or the measurement of anything, really) with just this kind of misconception about scales of measurement for different types of data.

Anonymous said...

== BCL said: ==
=". . . who knows how many more months of delay and inaction it might give rise to? The most obvious argument his work will inspire is: how can we trust ANY of the data, now that we have discovered this particular flaw in in this particular data-set."=

That is a dishonest comment. He improved the data. And for that you now try and turn it around to blame him for any possible delays on action on AGW.

Very sour grapes I think.

- Paul G

bigcitylib said...

Paul,

I think there is no question, whatever his work actually shows,that it will be used to make this argument. Look at his comments section and you'll see it is already being used in this manner.

Mike said...

BCL,

Nice demonstration of the scientific method in action in this series of posts. I wonder how many of those congratulating you in the previous post got it.

Anonymous said...

It is increasingly fashionable to denigrate the USA not least, I suppose, because being such a very large and powerful nation your faults, too, are writ large. Happily, as my wife will confirm, I am the least fashionable man you could ever wish to brush by quickly in the street, so I do not join that particular chorus. However, the excellent Messrs. Watts and McIntyre have demonstrated beyond peradventure that the temperature measuring sites in your country are bloody awful, and the boffins who juggle the figures have dropped a fairly enormous bollack! Well, as you Americans put it so accurately and succinctly, shit happens - over there, just like over here.

The point is, if such a 'snafu' can occur in God's sophisticated little acre, why on earth would I trust any statistics from China, Indonesia, Russia or, dare I say it, Britain?

Please, give me a reason some one, because the earth just moved for me - or at least its temperature just did!

bigcitylib said...

David,

I'm actually Canadian, as are most of the people who visit this blog. We're just a bit to the North.

Thanks for the kind words on your own blog, by the way.

Anonymous said...

The Warmongers view of the Scientific method is "I don't have to and won't show you my data, my methods or my processes. Just my Results".

"Trust me"


The rest of the world calls that Faith Based Science

Anonymous said...

Of course, I saw that but then forgot it. Still, any nation that contains both yourself and Steve McIntyre is a happy nation!

To be slightly more serious, it is important that the debate continues and that arguments on *both sides* are demonstrated with enthusiasm but courtesy, if only for the benefit of know-nothings like me who try to keep up with the controversy.

Anonymous said...

I think that it's important to move past or to have never gotten into the politicization of science. Unfortunately, I see both sides guilty of this. Both deniers who cackle about Kyoto being dead, when this result does not show that, and warmers who respond to this finding only with the numerical, it doesn't matter argument, not guving the discovery it's due (for instance this was detective work where the NASA scientists refused to share code, but Steve still dsicovered a mistake...shouldn't we be open?)

bigcitylib said...

TCO,

McIntyre runs around "demanding" all sorts of data from all sorts of people. He even inadvertantly launched a DOS attack of his own on NASA servers. For the most part, Hansen et all were quite forthcoming in their response.

magnus said...

What if one starts to scrutinize *all* GISS computation in the world? I mean, gavins sais it's just US, but in science it must be important to prove that an error doesn't occur in any data included (here defining global temperature).

Also the rural stations in US shows no temperature increase the last 115 years (since the start of the GISS data record; the city stations shows about a 2 degrees increase), and shouldn't all rural stations of the world be analyzed and compared with all city stations?


(BTW: How can there be som many barbeques at these stations? Well, I guess gavin or Real Climate can't answer that question!)

magnus said...

bigsitylib: "McIntyre runs around "demanding" all sorts of data from all sorts of people."

Sometimes I think that people rather wants to be cheated in a closed totalitarian society, if only the politically correct answers are "secured".

bigcitylib said...

Magnus,

It is ONLY rural stations that are used in calculating temperature trends. What makes you think otherwise?

Anonymous said...

I expect a higher level of forthcomingness. HAnsen has still not shared code and methods. It really did take some detective work to figure out the problem. Not just inspecting the code. And the problem had been there for years.

I agree that the automated data scraping from the website should have been handled differently (getting permission first, putting in a time delay). He got shut down half way through it (appropriately), but then was allowed to get what he wanted after explanation.

However, he still has not gotten the code and methods. I think that should be public. Look at the mistake that Lambert found in McItrick's work. Openness is the way to go. The way to find mistakes.

bigcitylib said...

TCO,

True but you are talking about perfect world stuff. In reality, data sharing is not anywhere near universal across the sciences. Look at what's happened to the Flores man fossils (and I think it was N. Bohn that published false data so other researchers wouldn't get to a theory of the atom before him). Also, the gathering of data often represents time, money, and effort on behalf of the researcher. It is not entirely irrational that they should want to exclusively benefit from that effort.

magnus said...
This comment has been removed by the author.
bigcitylib said...

That should be N(iels)Bohr.

magnus said...

TCO: I agree about the openness! Without that it isn't even science.

(OT: Michael Mann seams to me to be a total fraud when it comes to his math capabilities. Many times I've heard him he don't seems to have information about and realize the meaning of simple things, when he talks about his own stuff.)

magnus said...

bigsitylib: Here is a GISS diagram which shows no US rural temperature trend over the last 115 years (the site may not possible to reach all the time).

http://www.climateaudit.org/?p=1859

magnus said...

bigcitylib: "It is ONLY rural stations that are used in calculating temperature trends. What makes you think otherwise?"

Fact. Is it fun to return rethorical questions without substance? (**) Of course al stations are used, but some non-rural has been abolished due to that rules are breaken. But McIntyre and the people he works with has also shown that a huge amount of stations used for the GISS data breaks the rules for these stations (100 feet from buildings, parking lots, ...barbeques and fans ;-) etc...)


(**) On this page NASA sais they adjust included *non-rural* stations according to the long term trend, and that stations that can't be adjusted to a long term trend are abolished. So they abolish measurement that goes against an upward trend? That isn't science in my ears.

bigcitylib said...

And here is an account of which stations are used:

http://rabett.blogspot.com/2007/07/does-gisstemp-overcount-rural-stations.html#links

Ti-Guy said...

Let's marginalise the insane first. That includes any blogger who links to Atlas Shrugs.

magnus said...

Just an example of a possibly rural weather station providing data (You've maybe seen it on Climate Audit before).

magnus said...

I don't see any list of stations on that page bigcitylib.

Just a post shich sais the rural stations are used too much, not that they only are used! (...as well as a post that wants to emphasize the large temperature rise in the US. I rest my case...)

So that post was what made you think city stations isn't used?!? Wow!

And you have no problem that there is a huge difference of almost 3 degrees in temperature between used city station and rural stations respectively?

You too think, as the blogger you linked, that the rural station shall be used more?

-

Meta comment: By trial and error I 'v come to realize I can't get any warming mongerer to think rational or made up his mind on anything, and therefor my hypothesis is that environmentalists are retarded. (Some claims that they may be marxists, but I doubt that.) So excuse me for all this completely meaningless comments here.

magnus said...

ti-guy: Yeah Baby! I'm really glad you mentioned Atlas Shrugs!!

magnus said...

The about 2.5 degrees difference between rural and city stations is in the 100+ years trend. You'll see it when the Climate Audit site works again!

Ti-Guy said...

I'm really glad you mentioned Atlas Shrugs!!

And if she ends up behind bars, she'll be mentioned even more often.

Stay tuned...

bigcitylib said...

Magnus wrote:

"I don't see any list of stations on that page bigcitylib."

Because I linked to the wrong post:

http://rabett.blogspot.com/2007/07/theres-hot-time-in-marysville-or-how.html

And the money passage:

ONLY RURAL STATIONS CONTRIBUTE TO THE TREND IN GISSTEMP.

Magnus continued:

"By trial and error I 'v come to realize I can't get any warming mongerer to think rational..."

By trial and error seems to also be your approach to spelling.

Ti-Guy said...

Well, you have to give Magnus a pass on that. He's Swedish.

Magnus, how do you say "loony redneck wingnut" in Swedish?

Anonymous said...

Also, the gathering of data often represents time, money, and effort on behalf of the researcher. It is not entirely irrational that they should want to exclusively benefit from that effort.

Actually, it's typically MY money -- OUR money -- they are spending. As citizens, we have every right to insist that citizen-paid research be done in a proper and open manner.

I find it continuously astounding that people are allowing their preferred AGW answer to color their perspective on the basics of science.

You should be ashamed of yourself. But you aren't... because the truth is less important than winning.

--MrPete

Anonymous said...

If you've read The Star's story on this, it specifically says:

NASA officials have dismissed the changes as trivial. Even the Canadian who spotted the original flaw says the revisions are "not necessarily material to climate policy." [and] described his finding as a "a micro-change. But it was kind of fun."

Anonymous said...

bcl said "It is ONLY rural stations that are used in calculating temperature trends. What makes you think otherwise?"

and then to prove this statement quotes a blog post by Eli Rabett that says

"ONLY RURAL STATIONS CONTRIBUTE TO THE TREND IN GISSTEMP."

This clearly does not imply that only rural stations are used, only that only rural stations contribute to to the calculated warming trend.

In fact Rabett also says "The bottom line is that since the urban and periurban stations have their temperatures adjusted so that the trends match those of the "nearby (500 km)" rural stations, the long term trend is only determined by the rural stations.", which clearly shows that he knows that urban stations are used.