ON CLIMATE CHANGE POLLS

Polls and Surveys Grab Media Headlines; But Beware Polling Pitfalls on Climate Change
John Wihbey, June 16, 2009 (Yale Forum on Climate Change and the Media)

SUMMARY
The many polls and varied results characterizing the public’s attitude on global climate change suggest misguided methods and misinterpreted results. The conclusion late in 2008 and early this year that the public is losing interest in the question is especially questionable given more recent and comprehensive polls.

Economy, Jobs Trump All Other Policy Priorities In 2009; Environment, Immigration, Health Care Slip Down the List (January 22, 2009), from the Pew Research Center, Increased Number Think Global Warming Is “Exaggerated”; Most believe global warming is happening, but urgency has stalled (March 11, 2009) , from Gallup, and Only 34% Now Blame Humans for Global Warming (April 15-16, 2009), from Rasmussen Reports, require reevaluation and in the light of a reconsideration of their methods and in the light of the findings of Climate Change in the American Mind (Sptember/October 2008), from Yale University and George Mason University, and a March 25-29, 2009, poll from The Mellman Group, Public Opinion Strategies and the Pew Environmental Group.

Under the best of circumstances, poll results are subject to the structure of the questions asked and the way answers are evaluated. Headlines describing polls often go awry.

Example: A March 11 Gallup poll claimed that 41% of Americans believe the seriousness of global warming is “exaggerated” in the media. Gallup described this as the “highest level of public skepticism about mainstream reporting” in over a decade. Despite more recent polls with different findings, mainstream media sources (Newsday, the San Francisco Chronicle, the Washington Examiner, USA Today) continue to report the Gallup results.

The way the question is asked and long term trends matter as much as the headlines. (click to enlarge)

Jon Krosnick, a Stanford polling expert and professor of communications, recommends a 2-step process for evaluating poll results: (1) Evaluate the clarity of the specific questions, and (2) check the poll’s methodology.

Careful analysis of the March 11 Gallup poll reveals its flaws.

Violating the professor’s first analytic step, Gallup’s question was flawed. It asked about “the media” without establishing whether the respondents’ replies referred to narrow media or mainstream media.

Violating the professor’s second analytic step, Gallup’s methodology was flawed. It used the jump from 35% to 41% between late 2008 and early 2009 in the public’s opinion that global climate change is exaggerated by the media, failing to account for long-term trends. The number was 38% in 2004 and 30% in 2006.

This finding from the same Gallup poll suggest a higher level of concern. (click to enlarge)

David Moore, a media critic and former pollster for Gallup, admitted that long-term trends, which make the jump measured by the March 11 poll between late 2008 and early 2009 of little significance, may be the only thing polls can accurately measure.

The nature of the question asked was also crucial in the Rasmussen poll. It got its result by limiting the choices offered for answers in a way Moore called “really flakey.”

Moore identified another example in the Rasmussen poll of how methodology can skew results. The 2008 poll began by asking about the environment. The 2009 poll began with a question about energy. By introducing upfront potential anxieties about energy, the 2009 poll could have “pushed” a result that diminished the importance of climate change.

Perhaps the greatest example of how polls on climate change be manipulated comes from How Citizens Integrate Information without Ideological Cues: Local Weather and Americans’ Beliefs about Global Warming, by academicians Patrick Egan and Megan Mullin. They studied the impact between the local weather conditions and opinions about climate change. Their conclusion: The less well-informed people are, the more the day-to-day fluctuations in weather misinform their opinions on global climate change over the time frame of decades and centuries.

click to enlarge

COMMENTARY
Poll results make great news stories. It is the journalistic version of holding a mirror up and giving the public the chance to admire itself. Poll results make the best news stories when they feed an undercurrent of conspiracy thinking and allow the public to think it is smarter than accepted wisdom.

Poll results are especially vulnerable to skewing. Nothing could make this more clear than a careful look at the March 11 Gallup poll. Not only was its primary question poorly constructed and its methodology weak, but the headline was extremely misleading. The headline suggests not that the media’s coverage of climate change is exaggerated but that the seriousness of climate change itself is exaggerated.

click to enlarge

While there was an increase in the number of people who thought the media was exaggerating the climate change story in the March 11 Gallup poll, there was little change in the number of people who believe climate change “will pose a serious threat” in their lifetimes.

Moore, the media critic and former Gallup pollster who looked more carefully at the poll’s results, said he really couldn’t write a news story on the its unimpressive findings. Nevertheless, Gallup had a poll to sell and found a headline. Moore calls this manufacturing opinion. Anybody who writes headlines knows how that works: “If it’s a slow news day,” goes a journalistic aphorism, “I’ll go out and bite a dog.”

click to enlarge

The headline that only 34% of the respondents believe climate change is caused by human activity was manufactured in the Rasmussen poll by asking whether “human activity” or “long term planetary trends” cause climate change. In fact, 80% of most poll respondents believe the answer is “both” but that was not an option in the Rasmussen poll.

A variety of factors may figure into public opinion, potentially skewing results as greatly as the weather can skew them. One pollster suggested magical thinking about President Obama’s powers to reverse climate change or unfounded fears about job losses associated with actions against climate change could alter opinions.

click to enlarge

In fact, many have questioned the use of the terms “global warming” and “global climate change.” NewEnergyNews consistently uses the latter in order to eliminate the irrational emphasis on the ideas of warming and weather but some pollsters believe either phrase is inappropriate and have suggested “our deteriorating atmosphere.”

That will get the climate change deniers going.

click to enlarge

QUOTES
- Jon Krosnick, polling expert/ professor of communication and political science, Stanford University: “Do not be a victim of the wording of the latest polls…Just because it happens, doesn’t mean it deserves coverage.”
- Frank Newport, Editor, the March 11 Gallup Poll: “[The urgency of climate change has] just not caught on…[The advocates] have failed.”
- David Moore, former Gallup pollster and media critic: “They may have kind of misled people by focusing on an issue that may not be fundamental…I would have a hard time writing a story [on the reality of the poll]…”
- Krosnick: “More than 80 percent of Americans think that nature and man are responsible…But when you ask just about man-made global warming, you get lower numbers.”

click to enlarge

- From the Eagan and Mullin paper on climate change polls and weather: “Global climate change has been called one of the most important public policy challenges of our time and one of the greatest threats to life on Earth as we know it. But it is a complex issue of low salience about which Americans have little direct experience in their day-to-day lives. As they try to make sense of this difficult issue, the public uses fluctuations in local temperature to reassess their beliefs about the existence of global warming. Across a variety of estimation strategies and model specifications, the effect of weather on beliefs is significant and substantively large.”