Last week, Campaign Desk flagged a New York Times item that reported a surprising finding: for all the criticism about the government’s response to the Gulf oil spill, Barack Obama’s public approval ratings had remained basically steady.
But times have changed. On Wednesday evening, a joint NBC News/Wall Street Journal poll (PDF) was released, finding that fewer respondents approved of Obama’s performance (45 percent) than disapproved (48 percent). The numbers represented downward movement from late May, when 48 percent approved of the president’s performance, and even more slippage from early May, when 50 percent approved. What might explain the decline? Well, the oil started flowing from the Deepwater Horizon on April 22, and by a 50-42 margin, respondents in the new poll disapprove of the way Obama is handling the spill’s aftermath.
NBC’s news operation didn’t hesitate to broadcast the significance of its findings. Chuck Todd, appearing on Andrea Mitchell’s MSNBC show Wednesday afternoon to preview the results, declared that the spill is “the straw that is breaking the camel’s back as far as the president’s approval ratings.” Savannah Guthrie, appearing with Brian Williams on the NBC Nightly News, described how the president’s stock had weakened. “Why?” she asked. “The oil spill.” The headline on the online story was just as clear: “Poll: Spill drags the president’s rating down.”
Easy as putting two and two together, right? Well, no. In fact, while there’s no reason to doubt the quality of this particular poll, there is reason to continue holding off on the conclusion that the oil spill is the president’s undoing. And NBC, in its presentation of the data it paid for, failed to give its readers and viewers the full story in at least two significant ways.
To begin with, the NBC/WSJ poll, conducted by Bill McInturff and Peter Hart, is a good one. But there are other good polls out there too, and they don’t necessarily tell the same story. On Thursday, Pew released its latest numbers. The headline? “Obama’s Ratings Little Affected by Recent Turmoil.” Here’s the opening of the Pew write-up:
Since the beginning of this year, President Obama has signed a controversial health care measure, coped with a stubbornly high jobless rate, and struggled to manage the largest environmental disaster in the nation’s history. In that period, Obama’s overall job approval rating has moved from 49% to 48%.The point is not that Pew is right and the NBC poll is wrong, but that both data sets are legitimate—so journalists should include both, and be circumspect about sweeping conclusions. Any given poll contains uncertainty, so “until we see several of them moving in the same direction, it’s pretty hard to be sure that you’re picking up true change,” said Charles Franklin, a professor of political science at the University of Wisconsin and co-founder of the polling aggregation site Pollster.com.
Media institutions have an obvious incentive to play up the polls they pay for. But “a story written entirely from the point of view of either of those two polls would be misleading to readers,” Franklin said. A more accurate story would present the fuller range of data—which remains, at the moment, ambiguous.
To be fair to NBC’s journalists, the Pew poll hadn’t yet come out when they started reporting on their numbers. But they didn’t need it to offer more perspective. Franklin passed on via e-mail the chart below, which shows the trend for every pollster who has conducted surveys before and after the oil spill (click the image for a larger version as a PDF):
There’s a considerable amount of variation there. Taking a comprehensive view, “the trend lines do show some modest long-term decline,” according to Franklin. But while supposition that the spill might become a drag on the president is reasonable, “statistical tests show little evidence that the decline is specifically after the oil spill—rather, [we see] a continuation of a very slow decline since the first of the year.” What you’d need to see to make claims about the spill’s impact is not a downward trend after that point, but a worsening trend—and while it might show up eventually, “I just don’t think the evidence is there yet.”
Now, technical experts in the social sciences can and do differ on what constitutes sufficient evidence. But that leads to the second, even more frustrating problem with NBC’s coverage: it wasn’t its pollsters who were pushing the “spill drags down Obama” line. It was its journalists.