My colleague Holly Yeager made a pretty good case the other day that the press is exaggerating the level of public concern over the deficit vis-à-vis other pressing problems, like lagging economic growth and unemployment. (For a fresher example of the kind of thing she was talking about, see this story from Thursday’s New York Times, which declared that “Efforts to win approval of [a new spending bill] were not helped by results from Tuesday’s election reflecting a continuing frustration with Congress and the strength of appeals to pare government spending.”) So it was a little surprising to see The Washington Post’s Ezra Klein, who’s recently come in for some kind treatment around here, mention in passing Thursday that “a new Gallup poll finds the national debt tied with terrorism as voters’ biggest worry.” (Elsewhere in the item he refers to the deficit, not the debt; they’re obviously related, both in substance and in voters’ minds.) Klein linked back to The Atlantic’s Derek Thompson who, like Annie Lowery of The Washington Independent, had flagged the poll under a headline declaring that the debt is now America’s “top fear.”
Those are some sharp journalists, and Gallup’s a reputable outfit. So, is there evidence after all that the public really is freaked out about the debt/deficit, more so than other concerns?
Not really. And while Chris Bowers has already done a good job explaining why, it’s worth lingering a moment on this case, to explain how too-hasty readings of polls can help reinforce misleading storylines.
To begin with, let’s look at the actual results. Here’s Gallup’s chart:
Gallup headlines these data as “Federal Debt, Terrorism Considered Top Threats to U.S.,” and writes, in its lede, “Terrorism and federal government debt tie as the most worrisome issues to Americans.” But that interpretation depends on a very particular reading that places undue emphasis on the distinction between “extremely serious” and “very serious.” Group the two together, and there’s a near-perfect four-way tie at the top. (Unemployment actually emerges as the “top fear,” though not by any meaningful margin.) This is, in fact, the approach Gallup takes in the “Bottom Line” section of its own write-up, which reads, “Of several potential threats to the future wellbeing of the United States, Americans seem most concerned about terrorism and several economic matters — the federal budget debt, healthcare costs, and unemployment.”
What else? Well, a cardinal rule of polling interpretation is not to make too much out of any individual poll, which, no matter how well it’s conducted, may be an outlier. And in this case, there are other polls that tell a different story. Yeager noted a Fox poll (PDF) conducted three weeks before Gallup’s in which “the economy and jobs” was the top priority of 47 percent of respondents, compared to 15 percent who chose “the deficit and government spending.” Bowers mentions that poll and also an NBC/Wall Street Journal survey that found similar, if slightly more balanced results. Those polls are still pretty current. If they were valid at the time, they don’t become junk data because a new item comes out that suggests a different storyline.
The equivocal nature of the Gallup data, and the presence of contrary information, should’ve been enough to make journalists exercise some caution before making any claims about it. And actually, a closer look reveals an explanation for the variation that doesn’t have anything to do with sampling error or shifting public opinion over the last couple weeks. It lies in the fact that if polls tell us anything about what people think, they tell us what people think about the question that was asked—not about related topics, choices, or courses of action. The Gallup survey asked folks to describe “how serious a threat to the future wellbeing of the United States” a whole host of issue areas are. The Fox poll, by contrast, asked “which ONE of the following items… is more important for the federal government to be working on right now?” Similarly, NBC/WSJ asked, “if you had to choose just one, which do you think should be the top priority?”
The differences, both in time horizon and in forcing respondents to pick a single priority, are pretty stark. And if you want to know what voters’ “biggest worry” (Klein’s words) or “most pressing issue” (Lowery’s) is—in the everyday sense of “what problem people want their elected representatives to spend their time fixing right now”—the Gallup poll really doesn’t undermine the earlier findings, that jobs and the economy are the priority, at all. (To be fair, Thompson preserves the language about “future well-being” in his post, though he doesn’t emphasize its potential significance.)
Poll data are grist for the link economy, easily available hard information ripe to be dissected, analyzed, shared, and explored. That’s all well and good. But, please, let’s see a little more time, and a little more attention, spent on what the latest numbers do—and don’t—mean.
Greg Marx is an associate editor at CJR. Follow him on Twitter @gregamarx.