Last week, Campaign Desk flagged a New York Times item that reported a surprising finding: for all the criticism about the government’s response to the Gulf oil spill, Barack Obama’s public approval ratings had remained basically steady.
But times have changed. On Wednesday evening, a joint NBC News/Wall Street Journal poll (PDF) was released, finding that fewer respondents approved of Obama’s performance (45 percent) than disapproved (48 percent). The numbers represented downward movement from late May, when 48 percent approved of the president’s performance, and even more slippage from early May, when 50 percent approved. What might explain the decline? Well, the oil started flowing from the Deepwater Horizon on April 22, and by a 50-42 margin, respondents in the new poll disapprove of the way Obama is handling the spill’s aftermath.
NBC’s news operation didn’t hesitate to broadcast the significance of its findings. Chuck Todd, appearing on Andrea Mitchell’s MSNBC show Wednesday afternoon to preview the results, declared that the spill is “the straw that is breaking the camel’s back as far as the president’s approval ratings.” Savannah Guthrie, appearing with Brian Williams on the NBC Nightly News, described how the president’s stock had weakened. “Why?” she asked. “The oil spill.” The headline on the online story was just as clear: “Poll: Spill drags the president’s rating down.”
Easy as putting two and two together, right? Well, no. In fact, while there’s no reason to doubt the quality of this particular poll, there is reason to continue holding off on the conclusion that the oil spill is the president’s undoing. And NBC, in its presentation of the data it paid for, failed to give its readers and viewers the full story in at least two significant ways.
To begin with, the NBC/WSJ poll, conducted by Bill McInturff and Peter Hart, is a good one. But there are other good polls out there too, and they don’t necessarily tell the same story. On Thursday, Pew released its latest numbers. The headline? “Obama’s Ratings Little Affected by Recent Turmoil.” Here’s the opening of the Pew write-up:
Since the beginning of this year, President Obama has signed a controversial health care measure, coped with a stubbornly high jobless rate, and struggled to manage the largest environmental disaster in the nation’s history. In that period, Obama’s overall job approval rating has moved from 49% to 48%.
The point is not that Pew is right and the NBC poll is wrong, but that both data sets are legitimate—so journalists should include both, and be circumspect about sweeping conclusions. Any given poll contains uncertainty, so “until we see several of them moving in the same direction, it’s pretty hard to be sure that you’re picking up true change,” said Charles Franklin, a professor of political science at the University of Wisconsin and co-founder of the polling aggregation site Pollster.com.
Media institutions have an obvious incentive to play up the polls they pay for. But “a story written entirely from the point of view of either of those two polls would be misleading to readers,” Franklin said. A more accurate story would present the fuller range of data—which remains, at the moment, ambiguous.
To be fair to NBC’s journalists, the Pew poll hadn’t yet come out when they started reporting on their numbers. But they didn’t need it to offer more perspective. Franklin passed on via e-mail the chart below, which shows the trend for every pollster who has conducted surveys before and after the oil spill (click the image for a larger version as a PDF):
There’s a considerable amount of variation there. Taking a comprehensive view, “the trend lines do show some modest long-term decline,” according to Franklin. But while supposition that the spill might become a drag on the president is reasonable, “statistical tests show little evidence that the decline is specifically after the oil spill—rather, [we see] a continuation of a very slow decline since the first of the year.” What you’d need to see to make claims about the spill’s impact is not a downward trend after that point, but a worsening trend—and while it might show up eventually, “I just don’t think the evidence is there yet.”
Now, technical experts in the social sciences can and do differ on what constitutes sufficient evidence. But that leads to the second, even more frustrating problem with NBC’s coverage: it wasn’t its pollsters who were pushing the “spill drags down Obama” line. It was its journalists.
On Thursday, Todd devoted a segment of The Daily Rundown, his morning show on MSNBC, to talking about the poll with McInturff and fellow pollster Fred Yang. After a modest, low-key introduction (the poll was “the big story, frankly, driving the day”), Todd turned it over to McInturff, who said:
The most important point is how corrosive the economy is. After just a little bit of feeling better and April and May, once again, people dropped in economic confidence. Only 33 percent say they think the economy will get better. And the most powerful number is that 82 percent of Americans are dissatisfied with the country’s economy, half very strongly. No American president can stay in office and have those kind of economic numbers, and that kind of dissatisfaction, on top of—and you add the oil spill, which is a two-month, how-do-we-fix-this story, and not presume sooner or later you’re going to see a push-down on your job approval.
Yes, he said the words “oil spill.” He also said the words, “The most important point is how corrosive the economy is.” And, “the most powerful number is that 82 percent of Americans are dissatisfied with the country’s economy.” Pretty clear, right?
Apparently not. Todd, relentlessly on message, followed up by turning to Yang and asking: “Is it fair to say that the oil spill is the difference here?” That drew this ambiguous response:
I think it’s fair to say that. I think it’s not the oil spill per se—just two more months, as Bill said, of just bad news.
At the end of the segment, after sneaking in another remark about the “economic weight” dragging down the president, McInturff made a game effort to clear this up:
This is the Gulf spill in a second. The president’s been stopped—he’s had two months where he’s not been able to talk about jobs, jobs, jobs. And it’s pushed [aside] any ability the Democrats and the president have to try to reframe an economic message to try to get people feeling better. And I think that’s a lost opportunity.
There are reasons to doubt that with better “messaging” the president would be able to persuade voters that a lousy economy is not, in fact, a lousy economy. But the broader point, made clearly by McInturff, is that while the oil spill isn’t helping, the economy is the key factor shaping public opinion. It takes a perverse, willful adherence to your preconceived narrative to get this kind of input from the expert you are paying for his expertise and render it as “Poll: Spill drags the president’s ratings down.” (The Journal’s coverage was much better on this score, though it still didn’t take into account other polls.)
Applied properly, polls can tell us something about what the public thinks—discrediting bogus storylines and beating back speculation from pundits who “know” the public agrees with them. But when you hear someone claiming that a single poll has identified an “inflection point,” or the precise moment when a specific issue has “broken the camel’s back,” you’re hearing a claim for something polling can’t do. Time to tune that person out, and start paying attention to someone who’s interested in what the numbers can tell us.
P.S. Pollster.com contributor Brendan Nyhan makes a similar point at his blog.
Greg Marx is an associate editor at CJR. Follow him on Twitter @gregamarx.