from dot earth
October 5, 2009, 2:41 pm Climate Auditor Challenged to Do Climate Science
By Andrew C. Revkin
Bloggers skeptical of global warming’s causes* and commentators fighting restrictions on greenhouse gases have made much in recent days of a string of posts on Climateaudit.org, one of the most popular Web sites aiming to challenge the deep consensus among climatologists that humans are setting the stage for generations of disrupted climate and rising seas. In the posts, Stephen McIntyre, questions sets of tree-ring data used in, or excluded from, prominent studies concluding that recent warming is unusual even when compared with past warm periods in the last several millenniums (including the recent Kaufman et al. paper discussed here).
Mr. McIntyre has gained fame or notoriety, depending on whom you consult, for seeking weaknesses in NASA temperature data and efforts to assemble a climate record from indirect evidence like variations in tree rings. Last week the scientists who run Realclimate.org, several of whom are authors of papers dissected by Mr. McIntyre, fired back. The Capital Weather Gang blog has just posted its analysis of the fight. One author of an underlying analysis of tree rings Keith Briffa, responded on his Web site and at on Climateaudit.org.
What is novel about all of this is how the blog discussions have sidestepped the traditional process of peer review and publication, then review and publication of critiques, and counter-critiques, by which science normally does that herky-jerky thing called knowledge building. The result is quick fodder for those using the Instanet to reinforce intellectual silos of one kind or another.
I explored this shift in the discourse in some e-mail exchanges with Mr. McIntyre and some of his critics, including Thomas Crowley, a University of Edinburgh specialist in unraveling past climate patterns. Dr. Crowley and Mr. McIntyre went toe to toe from 2003 through 2005 over data and interpretations. I then forwarded to Mr. McIntyre what amounted to a challenge from Dr. Crowley:
Thomas Crowley (now in Edinburgh) has sent me a note essentially challenging you to develop your own time series [of past climate patterns] (kind of a “put up or shut up” challenge). Why not do some climate science and get it published in the literature rather than poking at studies online, having the blogosphere amplify or distort your findings in a kind of short circuit that may not help push forward understanding?
As [Dr. Crowley] puts it: “McIntyre is really tiresome - notice he never publishes an alternate reconstruction that he thinks is better, oh no, because that involves taking a risk of him being criticized. He just nitpicks others. I don’t know of anyone else in science who actually does such things but fails to do something constructive himself.”
Here’s Mr. McIntyre’s reply (to follow references to publications you’ll need to refer to the linked papers). In essence, he says he sees no use in trying his own temperature reconstruction given the questions about the various data sets one would need to utilize:
The idea that I’m afraid of “taking a risk” or “taking a risk of being criticized” is a very strange characterization of what I do. Merely venturing into this field by confronting the most prominent authors at my age and stage of life was a far riskier enterprise than Crowley gives credit for. And as for “taking a risk of being criticized”? Can you honestly think of anyone in this field who is subjected to more criticism than I am? Or someone who has more eyes on their work looking for some fatal error?
The underlying problem with trying to make reconstructions with finite confidence intervals from the present roster of proxies is the inconsistency of the “proxies,” a point noted in McIntyre and McKitrick (PNAS 2009) in connection with Mann et al 2008 (but applies to other studies as well) as follows:
Paleoclimate reconstructions are an application of multivariate calibration, which provides a theoretical basis for confidence interval calculation (e.g., refs. 2 and 3). Inconsistency among proxies sharply inflates confidence intervals (3). Applying the inconsistency test of ref. 3 to Mann et al. A.D. 1000 proxy data shows that finite confidence intervals cannot be defined before ~1800.
Until this problem is resolved, I don’t see what purpose is served by proposing another reconstruction.
Crowley interprets the inconsistency as evidence of past “regional” climate, but offers no support for this interpretation other than the inconsistency itself –- which could equally be due to the “proxies” not being temperature proxies. There are fundamental inconsistencies at the regional level as well, including key locations of California (bristlecones) and Siberia (Yamal), where other evidence is contradictory t.o Mann-Briffa approachs (e.g. Millar et al 2006 re California; Naurzbaev et al 2004 and Polar Urals re Siberia,) These were noted up in the N.A.S. panel report, but Briffa refused to include the references in I.P.C.C. AR4. Without such detailed regional reconciliations, it cannot be concluded that inconsistency is evidence of “regional” climate as opposed to inherent defects in the “proxies” themselves.
The fundamental requirement in this field is not the need for a fancier multivariate method to extract a “faint signal” from noise – such efforts are all too often plagued with unawareness of data mining and data snooping. These problems are all too common in this field (e.g. the repetitive use of the bristlecones and Yamal series). I think that I’ve made climate scientists far more aware of these and other statistical problems than previously, whether they are willing to acknowledge this in public or not, and that this is highly “constructive” for the field.
As I mentioned to you, at least some prominent scientists in the field accept (though not for public attribution) the validity of our criticisms of the Mann-Briffa style reconstruction and now view such efforts as a dead end until better quality data is developed. If this view is correct, and I believe it is, then criticizing oversold reconstructions is surely “constructive” as it forces people to face up to the need for such better data.
Estimates provided to me (again without the scientists being prepared to do so in public) were that the development of such data may take 10-20 years and may involve totally different proxies than the ones presently in use. If I were to speculate on what sort of proxies had a chance of succeeding, it would be ones that were based on isotope fractionation or other physical processes with a known monotonic relationship to temperature and away from things like tree ring widths and varve thicknesses. In “deep time,” ice core O18 and foraminifera Mg/Ca in ocean sediments are examples of proxies that provide consistent or at least relatively consistent information. The prominent oceanographer Lowell Stott asked to meet with me at AGU 2007 to discuss long tree ring chronologies for O18 sampling. I sent all the Almagre cores to Lowell Stott’s lab, where Max Berkelhammer is analyzing delO18 values.
Underlying my articles and commentary is the effort to frame reconstructions in a broader statistical framework (multivariate calibration) where there is available theory, a project that seems to be ignored both by applied statisticians and climate scientists. At a 2007 conference of the American Statistical Association to which Caspar Ammann (but not me) was invited, it was concluded:
While there is undoubtedly scope for statisticians to play a larger role in paleoclimate research, the large investment of time needed to become familiar with the scientific background is likely to deter most statisticians from entering this field. http://www.climateaudit.org/?p=2280
I’ve been working on this from time to time over the past few years and this too seems “highly constructive” to me and far more relevant to my interests and skills than adding to the population of poorly constrained “reconstructions,” as Crowley proposes.
In the meantime, studies using recycled proxies and problematic statistical methods continue to be widely publicized. Given my present familiarity with the methods and proxies used in the field, I believe that there is a useful role for timely analysis of the type that I do at Climate Audit. It would be even more constructive if the authors rose to the challenge of defending their studies.
Given the importance of climate change as an issue, it remains disappointing that prompt archiving of data remains an issue with many authors and that funding agencies and journals are not more effective in enforcing existing policies or establishing such policies if existing policies are insufficient. It would be desirable as well if journals publishing statistical paleoclimate articles followed econometric journal practices by requiring the archiving of working code as a condition of review. While progress has been slow, I think that my efforts on these fronts, both data and code, have been constructive. It is disappointing that Crowley likens the archiving of data to doing a tax return. It’s not that hard. Even in blog posts (e.g. the Briffa post in question), I frequently provide turnkey code enabling readers to download all relevant data from original sources and to see all statistical calculations and figures for themselves. This is the way that things are going to go – not Crowley’s way.
So should this all play out within the journals, or is there merit to arguments of those contending that the process of peer review is too often biased to favor the status quo and, when involving matters of statistics, sometimes not involving the right reviewers?
Another scientist at the heart of the temperature-reconstruction effort, Michael Mann of Pennsylvania State University, said that if Mr. McIntyre wants to be taken seriously he has to move more from blogging to publishing in the refereed literature.
“Skepticism is essential for the functioning of science,” Dr. Mann said. “It yields an erratic path towards eventual truth. But legitimate scientific skepticism is exercised through formal scientific circles, in particular the peer review process.” He added: “Those such as McIntyre who operate almost entirely outside of this system are not to be trusted.”
