Climate Change: How to spot a dodgy study

This has been an eventful year for natural calamities – drought in the Horn of Africa, floods yet again in Pakistan, and now also in Thailand.

NGOs, think-tanks and scientific organizations eager to share their insights or shed some light on what could be causing these events have tried to keep pace with nature by releasing an incessant supply of reports and studies.

Many reports are quick to link one-off events with climate change; others purport to provide regional evidence of how global warming is affecting a particular country; some try to identify which countries are most vulnerable to disasters.

Trying to sift through these reports to identify which are credible and authoritative can be tough. Here are some handy tips from top experts on how to spot the dodgy ones.

Question direct links to climate change

Reports linking any one-off event to climate change are really not worth pursuing. So why can’t scientists say climate change “caused” a certain extreme event?

The question is “nonsense”, said Daniel Huber and Jay Gulledge, researchers at the Pew Centre on Global Climate Change in their paper, Extreme Weather and Climate Change: Managing the Risk. “Climate is the average of many weather events over a span of years. By definition, therefore, an isolated event lacks useful information about climate trends.”

However, there have been some good scientific studies indicating that the occurrence of extreme climate events might have increased as a result of climate change. After the 2003 summer heatwave in Europe, researchers from the University of Oxford and the Hadley Centre for Climate Prediction and Research showed that human-induced rises in the concentration of greenhouse gases in the atmosphere had at least doubled the risk of a heatwave exceeding the magnitude of the one experienced that summer.

Be wary of regional and country-specific analysis of climate change impact

Much more research is needed into the regional impacts of climate change, said Robert Ward, head of policy and communications at the Grantham Research Institute on Climate Change and Environment at the London School of Economics.

Climate models, run on powerful computers which use decades of past and present climate data to simulate how climate will behave, or has in the past, are the most accurate and reliable source of information on potential future climate change. The models have been able to produce reliable projections on a global scale as opposed to regional ones.

However, “It is beyond the capacity of most current climate models to estimate regional impact,” Ward noted. So when a report says droughts will become more frequent in Kenya because of climate change, ask how the authors worked that out, and how many years of data went into their calculations.

But, climate models are becoming more powerful and sophisticated. The World Climate Research Programme has initiated a project that will help scientists using climate models to localise global climate data and provide information on how the climate system might behave in countries, cities, towns and even villages in Africa. This information will be available in the next assessment report by the Intergovernmental Panel on Climate Change (IPCC) in 2014.

Rely on the IPCC…

Their assessments of the state of knowledge on the subject are the “gold standard of climate change information,” said Gerald Meehl, a senior scientist at the US government-funded National Centre for Atmospheric Research and chair of the National Academy of Science. He would rather refer to them.

Hundreds of experts from across the world help prepare the IPCC assessments. The reports are issued at regular intervals and go through a rigorous review process. The panel also produces technical and other special reports, and people sometimes complain that the period between assessments is too long. The last one was released in 2007.

…but keep in mind its critics

Climate science is developing at a faster pace than IPCC reports are produced, said Sven Harmeling, co-chair of the adaptation working group of Climate Action Network International, a global coalition of NGOs. “Today we know much more about the dynamics of ice melting in Greenland, for example, than was contained in the last IPCC report.”

Read more:

• Soon every village will know what the weather may bring
• Climate change information overload?
• The impact of grey literature on climate projections
• Getting to the bottom of sea-level rise

Harmeling said the IPCC’s next assessment will have to strike a balance between being precautionary and less conservative. Previous assessments have come under criticism on both counts.

The 2007 report used a non-peer-reviewed study produced by an NGO – known as “grey literature” – to erroneously project that 80 percent of the Himalayan glacier area would very likely have melted by 2035.

In the absence of accurate data, the IPCC also did not take into account the full potential impact of the destabilisation of land-based ice sheets in the polar regions in the projections in its last report.

Meehl, who has been involved with IPCC reports since the first assessment in 1990, noted that previous reports were still “reliable” sources of information, as were special reports like the upcoming one on Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation, to be released on 18 November.

Peer-review is the safe route

Stick with peer-reviewed studies, said Richard Klein, who is leading preparation of the chapter on adaptation in the next assessment report. Klein is a senior research fellow at the Stockholm Environment Institute.

Ward said after the IPCC’s use of grey literature, the credibility of civil society organizations had also come into question, so introducing a peer-review process for reports produced by civil society could be helpful. A quality control process would lend more credibility to such studies, and reduce the chances of the NGO being accused of any conflict of interest. Organizations can conduct research but use the findings selectively to suit their campaign needs.

Check the authors

Use the internet to see if the authors of the study have been published or cited in any peer-reviewed journals.

Examine the methodology

It is common to find reports that cite sensational numbers which the media often highlight. “But find out how the authors arrived at those numbers – what methodology did they use,” said Ward.

Transparency is a critical issue for Klein. “We all share our methodologies and would like other organizations to do so as well, so we can vouch for the quality of their research.”

Source: Johannesburg, 9 November 2011 (IRIN).