WSJ’s Harwood Measures the Yardsticks

Over the summer Campaign Desk came across Vaughn Palmer, a columnist for the Vancouver Sun, who reacted in the most curious way when two contradicting polls were released measuring local government elections in British Columbia. Palmer not only reported the results, but also used the majority of his print space to explain why the two polls produced different results.

A similar situation arose in the U.S. presidential election last Friday when Gallup and the Pew Research Center simultaneously released polls with drastically different results. The Gallup poll measured Bush’s lead over Kerry among likely voters at landslide levels — 13 points, 55 percent to 42 percent. Yet, the Pew Research Poll found the race to be a virtual dead heat, with Bush leading Kerry 47 percent to 46 percent also amongst likely voters. This is the kind of thing that makes voters throw up their hands in exasperation and go out for a triple bacon cheeseburger with mayo, newly certain that no truisms are true.

Today, the Wall Street Journal’s John Harwood takes a page out of the Palmer playbook and uses his space not for pointless musings about what the latest polls “mean” for each candidate, but to educate his readers as to how two seemingly similar polls can produce such results so at odds with one another.

Harwood notes, as Newsday’s Jimmy Breslin did on September 16, that the proliferation of cell phones has left nearly 169 million phone numbers off pollsters’ random sampling lists, throwing off attempts to “ensure demographic and geographic balance in surveys.”

Harwood also notes that polls conducted during competitive elections routinely exhibit the polls’ “sampling error and methodological differences more often than shifting opinion.”

The real merit of Harwood’s piece comes in his lengthy explanation of how each polling outfit deals with party allegiance and likely voters. He explains that neither the Pew Research Center nor the CBS/New York Times poll weights the results to take into account likely voter turn-out. (In the 1996 and 2000 elections, Democrats beat out Republicans on voter turnout by 4 percentage points.)

“Unprecedented voter mobilization drives” that will gain velocity right up until Nov. 2 may also shoot traditional polling models all to hell, he notes. In the meantime, polling organizations can only stand by their methods — methods that differ from poll to poll. For example, the Wall Street Journal/NBC poll uses a single question to identify who will vote and who will not, while Gallup uses a method developed decades ago that is comprised of a series of questions.

As Ms. Grimley, your high school English teacher, used to say, “There is no right answer, students.” Or, as Peter Hart, a Democratic pollster, told Harwood in reference to a pollster’s ability to measure turnout, “This is art. This isn’t science. Nobody knows.”

In the end, of course, which voters do show up is all that matters — not what the Gallup or Pew polls said a month before or even the night before election day. So, as the race comes down to the final 43 days, it’s important to remember when digesting the latest poll that every “likely” voter is not necessarily as equal as the next.

Or, as Woody Allen first noted in another context, “Ninety percent of success is just showing up.”

Thomas Lang

Has America ever needed a media watchdog more than now? Help us by joining CJR today.

Thomas Lang was a writer at CJR Daily.