Clear Code has now reached an important milestone in its first big project:
... an all-Python reimplementation of GISTEMP, the NASA GISS surface temperature analysis.
And here's the chart that results from running this reimplementation:
As the lads at Clear Code note:
The results of running this release match GISTEMP results very closely indeed.
In fact, the annual global, northern hemisphere, and southern hemisphere anomaly results are identical, as are the southern hemisphere monthly anomalies. The global monthly anomalies differ 7 times, out of more than 1000, each time by one digit in the least-significant place.
...which should be surprising to no-one. By the way, the Clear Code programmers are looking for help as the project moves forward. There's a link to their mailing list here.
Not so much "wrapped up", more "at an important milestone". It's now all in Python. We have to work to make the Python _clear_.
ReplyDeleteDavid Jones:
ReplyDeleteI take it you are part of the Clear Climate Code project?
Thanks for doing this, I think it is a wonderful idea.
I have a few questions, if you dont mind.
I see you have now translated the code into Python and you're getting very similar results. This of course is only a validation of the translation, not of the original code - am I correct?
I suspect that once the Python code is made clear (a monstrous task, I have no doubt), it will then be possible to more easily audit whether the code itself is kosher - is that the next step in this project?
Do you or your colleagues have an opinion on AGW at the outset? (i should point out that I am a denialist and planetary traitor level 4)
Thanks again for your efforts.
I am part of the Clear Climate Code project.
ReplyDeleteYou are correct that the fact that ccc-gistemp performs identically to GISTEMP validates only the translation. We have two programs that do the same thing, that doesn't mean either program does The Right Thing (whatever that might be judged to be). However, we have found a few very minor bugs in the GISS version. I've read some of the papers and seen enough of the code to make this statement: «It is our opinion that the GISTEMP code performs substantially as documented in Hansen, J.E., and S. Lebedeff, 1987: Global trends of measured surface air temperature. J. Geophys. Res., 92, 13345-13372., the GISTEMP documentation, and other papers describing updates to the procedure.»
My opinion is that we can't really have a meaningful debate about the GISTEMP code until it's clear what it does. Hence Clear Climate Code. It's not Clear Climate Code's goal to verify that GISTEMP is "kosher", our goal (for this project) is to make it clear. Obviously I expect that to inform any assessment of whether it's any good or not.
Nick wrote "Sceptics are welcome", where he says "I am certain that anthropogenic global warming is real and a serious global crisis", and I would broadly agree with that. I cannot speak for the other participants. It's not something we talk about much. Sceptics are welcome, by the way.
David Jones:
ReplyDeleteThanks for your reply.
Thank you for this link. The project continues to move forward quite rapidly, and welcomes any competent Python coder to work on the code, or anyone else to download the code and look at it, to provide feedback on its obscurity.
ReplyDeleteWe achieved the all-Python milestone 0.2.0 by translating the GISS Fortran (and ksh, Python, and C) in a piece-by-piece way. For instance, the Python maintains the detailed file format of all the intermediate files. Another example: the original GISTEMP code divides its dataset into six separate files at one point, because in the 1980s the whole dataset would not have fitted into memory. Since 0.2.0 we are getting more adventurous, discarding intermediate files, combining stages, and so on, while being careful to maintain the algorithm. As a result, the code is becoming clearer at a very pleasing rate.
I hope that CCC-GISTEMP will inform any future debates about the GISTEMP algorithm and the resulting dataset. It certainly makes it very much easier for third parties to experiment with changes to the algorithm.
I should also say that although this project was indeed started nearly two years ago, it has only received anyone's full attention very intermittently. I wouldn't want anyone to think that it's an enormous endeavour. At a rough guesss there's probably about four person-months of work in it to date.
ReplyDeleteIt's very strictly pro bono - if I spend a week on this project, I don't get paid, at all, for that week.