Campaign Desk has a thing about polls and their use and misuse.
On the heels of an excellent a plea to the media in a column published by The Hill, a newspaper that covers Capitol Hill:
It is abundantly clear that polls on the presidential contest conducted by respected survey-research firms are all over the place. Within a few days recently, two of the oldest names in the business had dramatically different results that would lead to conclusions about the race that are diametrically opposed. Harris said John Kerry was ahead by one point, while Gallup found Kerry trailing by eight. At about the same time, Pew had President Bush ahead by one while CBS pegged his lead at eight points.
Truth is, that has happened during many weeks of this campaign. Back in March, Gallup had Kerry ahead by six points. Less than a week later, a CBS/New York Times poll reported Bush ahead by eight. On almost exactly the same days in June, the AP reported definitively that Bush was ahead by one point while the Los Angeles Times was just as sure that it was Kerry who led by seven points.
… [C]learly, not all of these results can be definitively true at the same time.
The conclusion is inescapable: at minimum, the polls produce an uncertain portrait of the state of the race.
Mellman is singing our song here. Yet, those self-same polls are invariably touted as gospel by the news outlets that commissioned them. “Bush Opens Lead Despite Unease Voiced in Survey” trumpeted The New York Times last weekend, reporting on a poll commissioned by itself and CBS. Other polls, showing widely varying results, merited a brief mention far down in the newspaper’s story.
Anyone looking for a good bit of reporting on the plethora of polls need only to check out a story (registration required) in today’s Milwaukee Journal Sentinel by Craig Gilbert. Gilbert deftly walks his readers through a veritable flood of new polls in the Badger State. Then he calls in an expert witness to make sense of the numbers. That would be University of Wisconsin political scientist Charles Franklin who offers this assessment: “Averaging across three or four or five polls in a similar time period is a better estimate of what opinion in the state really is than counting on any one poll for everything.” Amen.
Pollster Mark Mellman (who clearly has an agenda in writing his column) goes a step further:
The truth is clear. Uncertainty looms large. Uncertainty may not sell papers or create the aura of clairvoyance, but it has the benefit of accurately reflecting reality. Reputable polls disagree. …
I suggest a simple syllogism. The truth is that the state of this year’s presidential contest is uncertain at best. Journalists report the truth. Therefore, journalists should be reporting that the state of the race is uncertain.
Sounds like a good idea to us.
–Susan Q. Stranahan
Susan Q. Stranahan wrote for CJR.