But in order both to be credible and to be perceived as credible, that undertaking has to come from a place of open inquiry, not from an expectation that the results will boost book sales, create a news cycle, or confirm a political narrative. The whole appeal of polling is its promise (sometimes oversold) to produce objective, almost scientific data that tells us something about the state of our politics. When that enterprise itself becomes an exercise in political gamesmanship, it may foster cynicism about the utility and quality of any effort to collect information. If any given piece of news is valuable only insofar as it advances a particular political view, then they are all in a sense equivalent, which is to say worthless.
That’s not to say that political perspectives don’t have a place in journalism (they do) or that our political views won’t shape the way we understand information (they will). But we still need to protect a place for facts and evidence in our political debate, and to do that we need to push back against rhetorical opportunism and statistical sloppiness on all sides. Journalists can do this by not cutting corners when reporting and compiling data—but also by making the case for why methodological rigor is valuable, and by providing readers with the tools to evaluate information themselves.
This, in the end, is the real value in Langer’s post—it explains why the Harris poll is flawed in a way that will hopefully prompt closer scrutiny of storyline it perpetuated. While this type of direct engagement between media outlets is somewhat unusual, Langer has a clear incentive here: as a pollster, he needs to uphold the integrity, credibility, and rigor of his profession. It’s a lesson for other members of the press to keep in mind.
Update: For another close look at the Harris poll featuring thoughts on how to design a more valid survey, see Mark Blumenthal’s “Mystery Pollster” column for National Journal.
Update 2, 3/30: James Taranto’s Wall Street Journal piece last Thursday also raises good questions about the sampling and survey design.
Ends today: If you'd like to help CJR and win a chance at one of
10 free print subscriptions, take a brief survey for us here.