On the way to work this morning, we picked up one of New York’s free papers, Metro, and discovered the headline, “Poll Says Bush, Kerry Tied.” Once at the office, we clicked over to USA Today, only to see the cover story, headlined “Bush Clear Leader in Poll.” Readers of both publications could be forgiven for being confused. And their confusion would only continue if they read on: “The boost Bush received from the Republican convention has increased rather than dissipated,” USA Today exclaimed. No, declared the AP story in Metro, in reality Bush’s post-convention bounce had all but evaporated between September 8 and September 14.
The poll referenced by Metro, which showed a statistical tie, came from the Pew Research Center, which just three days earlier had released a different poll showing Bush with a 15-point lead among likely voters. (Meantime, a Harris poll taken in between the two Pew efforts showed a slim Kerry lead, and argued the “‘convention bounce’ has now disappeared.”) The poll featured in USA Today showing a huge Bush lead comes from Gallup.
There’s a history here; the same wild variance in September polls took place during the 2000 presidential campaign. In that year, Gallup showed Al Gore leading by 10 percentage points on September 20, yet losing by 13 percentage points on October 26 — a stunning 23-point shift in 36 days. (Meanwhile, polls commissioned by Fox and by Reuters/MSNBC on those same days showed statistical ties — which, of course, pretty well mirrored what happened on election day.)
USA Today regularly commissions polls from Gallup, though they didn’t commission this most recent one, erroneous media reports notwithstanding. The paper still gave the poll prominent play, however; according to Susan Page, who wrote the story, “we felt very comfortable comparing it to previous polls because it used the same methodology” as other polls that the paper had commissioned this year. Though the story does mention the contrasting Pew poll in the seventh paragraph, the lede, like the headline, presents the Gallup poll results as largely beyond question, asserting that the poll heralds “the first statistically significant edge either candidate has held this year.”
There were a few outlets that showed a willingness to recognize that competing and wildly divergent poll results had been released on the same day, like the Associated Press and CNN, which ran a story headlined “Latest Presidential Polls Vary Widely.”
So whom do you believe today — Pew, which portrays a dead heat, or Gallup, which shows Bush pulling away smartly? Our suggestion is to treat the polls as what they are — snapshots, not predictions, and snapshots taken by different cameras with different focus, shutter speeds, refraction and lens angles.
That’s the story that you don’t read or hear from the news organizations that commission the polls in question — and at this point, they all ought to be in question. It’s much simpler for newspapers to offer up raw numbers rather than to explain the intricacies of any given poll or trying to explain why one poll differs so wildly from another. Horse race numbers create easily digestible storylines, after all, and storylines, it is thought, keep people interested and sell papers.
Besides, relying on a poll to, in effect, do your reporting for you, instead of relying on said poll to put your reporting in perspective, is the easy way out. The new Gallup poll, with its disputed methodology, should not rate a banner headline in one of the nation’s most widely read newspapers; readers in search of an accurate take on the state of the race deserve better.
–Brian Montopoli
Brian Montopoli is a writer at CJR Daily.