The real risks of cherry picking scientific data [1] by Matt Ridley and his blog The Rational Optimist
To illustrate how far this problem reaches, a few years ago there was a scientific scandal with remarkable similarities, in respect of the non-publishing of negative data, to the Tamiflu scandal. A relentless, independent scientific auditor in Canada named Stephen McIntyre grew suspicious of a graph being promoted by governments to portray today’s global temperatures as warming far faster than any in the past 1,400 years — the famous “hockey stick” graph. When he dug into the data behind the graph, to the fury of its authors, especially Michael Mann, he found not only problems with the data and the analysis of it but a whole directory of results labelled “CENSORED”.
This proved to contain five calculations [2] of what the graph would have looked like without any tree-ring samples from bristlecone pine trees. None of the five graphs showed a hockey stick upturn in the late 20th century: “This shows about as vividly as one could imagine that the hockey stick is made out of bristlecone pine,” wrote Mr McIntyre drily. (The bristlecone pine was well known to have grown larger tree rings in recent years for non-climate reasons: goats tearing the bark, which regrew rapidly, and extra carbon dioxide making trees grow faster.)
Mr McIntyre later unearthed the same problem when the hockey stick graph was relaunched to overcome his critique, with Siberian larch trees instead of bristlecones. This time the lead author, Keith Briffa, of the University of East Anglia, had used [3] only a small sample of 12 larch trees for recent years, ignoring a much larger data set of the same age from the same region. If the analysis was repeated with all the larch trees there was no hockey-stick shape to the graph. Explanations for the omission were unconvincing.
Given that these were the most prominent and recognisable graphs used to show evidence of unprecedented climate change in recent decades, and to justify unusual energy policies that hit poor people especially hard, this case of cherry-picked publication was just as potentially shocking and costly as Tamiflugate. Omission of inconvenient data is a sin in government science as well as in the private sector.