TAMPA — “Perhaps if we all ignore PolitiFact, they’ll eventually go away,” went the lede of a Weekly Standard story distributed here on Tuesday. Not likely. Love them or hate them, PolitiFact and other journalistic factchecking entities are inescapable on this year’s campaign trail. Their ubiquity invites criticism—and, lately, questions about whether their work has any effect.
“What if it turns out that when the press calls a lie a lie, nobody cares?” asked Atlantic editor James Bennet, in a piece about the Romney campaign’s refusal to pull a campaign ad that was awarded “Four Pinocchios” by The Washington Post‘s “Fact Checker.” (“We’re not going to let our campaign be dictated by factcheckers,” said Romney pollster Neil Newhouse, who also declared that factcheckers have “jumped the shark.”) Others have noted that the factcheckers’ real-time dissections of the convention proceedings hasn’t actually stopped politicians from salting their speeches with lies, exaggerations, and half-truths.
Saying that the factcheckers are ineffective and therefore irrelevant doesn’t strike me as a particularly convincing argument; as CJR’s Brendan Nyhan noted today, the continued existence of false statements does not mean that the factcheckers—or other journalists who push back against misleading political rhetoric in traditional articles—are having no effect. More compelling are the criticisms of the factcheckers’ methodology. One recurring complaint involves the snappy names that the factcheckers like to give their truth scales—”Pinocchios,” or the “Truth-o-Meter”—and a general confusion over what these ratings mean and how they are determined. The PolitiFact Truth-o-Meter, for instance, has six settings, ranging from “True” to “Pants on Fire.”
As Nyhan has previously argued, this scheme “doesn’t work well for irresponsible and unsubstantiated claims,” such as Harry Reid’s unfounded suggestion that Romney hadn’t paid taxes in a decade, “that can’t be definitively falsified.” And even for claims that can be situated along a true/false axis, since the Truth-o-Meter is essentially a rhetorical construct, it’s sometimes hard to tell why a given item merits a “Half True” rather than a “Mostly True”—or, for that matter, a “Mostly False.” You can see why some critics think that the verdicts are rendered casually, and without much thought.
That’s not quite true. Wednesday afternoon, I sat in on a PolitiFact editorial meeting and watched as editor Bill Adair and two colleagues debated how to rate a statement made by South Carolina Governor Nikki Haley during her speech at the RNC the night before. Reporter Jon Greenberg had filed an item dissecting Haley’s statement—about Obama and the NLRB ostensibly suing the Boeing corporation—and suggested giving it a “Half True” rating. Adair and his staff were reviewing Greenberg’s piece to see if the rating was warranted.
Before any PolitiFact item goes up, three separate editors must agree on its Truth-o-Meter rating. They reach a consensus by consulting a checklist intended to systematically address the suitability of any given rating. First, they examine the item’s “ruling statement”—the assertion that is being factchecked. Adair starts the discussion: “So the ruling statement is that Nikki Haley ‘says President Obama and his National Labor Relations Board’ sued Boeing over its decision to open a plant in South Carolina.”
“Is that literally true?” Adair asks. “I would say no,” he answers. “They didn’t sue Boeing.” Adair’s right: the NLRB filed a complaint, not a lawsuit; Boeing was told to attend a hearing, not appear in court. A very brief discussion ensues, and all three editors agree that the statement isn’t literally true. But, in PolitiFact’s world, just because a statement isn’t literally true doesn’t necessarily mean it’s false. Adair moved on to a second question: “Is there another way to read the statement?”
“Yeah, I guess on how we interpret ‘sued,'” says Adair, noting that the word might be used as shorthand for a process that is similar to filing a lawsuit in court. And while Barack Obama himself didn’t file the complaint, Adair notes, it’s not necessarily unreasonable to hold Obama accountable for the actions of the NLRB. Again, a brief discussion follows, and the consensus seems to be that Haley’s statement might be read in a non-literal manner.
Next, they ask whether they’ve contacted the person who made the statement, presumably in order to defend or explain it further. There’s a sense in some criticism that PolitiFact blindsides people with its ratings, so it’s important to note that the outlet does always attempt to speak to the person who made the statement being rated. “Did we contact Haley?” Adair asks. They did, and Haley’s spokesman, Rob Godfrey, insisted it was fair to hold Obama directly accountable for the NLRB’s actions, because the president appointed the board’s members. Finally, they ask a fourth question: “What is our jurisprudence on similar items?”—or, how has PolitiFact ruled in situations like this in the past. They don’t spend much time going through this, but apparently, similar items have received “Half True” ratings before.
After finishing with the checklist, Adair throws the item open to discussion: “How do we feel about ‘Half True’?”
“I think that’s right,” another editor says. “My reason would be that Obama doesn’t have his fingerprints on this at all.” The other editor concurs. “’Half True’ it is,” says Adair.
Adair realizes that a lot of people think PolitiFact’s journalists just rate things helter-skelter. They don’t. Still, “Half True” means different things to different people, and once you start trying to gauge relative levels of truth and falsehood, you leave yourself open to criticism from people whose scales are calibrated differently than yours. Sure, the “Truth-o-Meter” rating is just a way to draw people into the article, which examines the disputed claim in much greater depth. But no matter how methodical Adair and his team are, the ratings they use are still imprecise—and imprecision is a problem when you exist to judge the precision of other people’s statements.
Admirable as their intentions are, there are legitimate criticisms that can be leveled at PolitiFact and its peers. It’s important that those criticisms are rooted in an awareness of how their processes actually work. Because despite the Weekly Standard‘s wishes, ignoring the factcheckers won’t make them go away.
Justin Peters is editor-at-large of the Columbia Journalism Review.