In the waning days of 2011, Politifact, the Pulitzer Prize-winning fact-checking site, brought the wrath of the liberal blog world upon itself when it dubiously—though not unexpectedly—chose as its “Lie of the Year” Democratic claims that House Republicans had voted to “end Medicare.” The uproar was just the latest wave of recrimination against the “fact-checking” enterprise, which has come in for scrutiny from outlets as diverse as Politico, The Weekly Standard, and Salon. But the rising tide of complaint seems not to trouble the fact-checkers—in addition to Politifact, the most prominent outlets include the Annenberg Public Policy Center’s Factcheck.org; The Washington Post’s “Fact Checker” blog, now written by Glenn Kessler, who penned a defense of the genre as part of his own year-end list of “Pinocchios”; and The Associated Press—who can tell themselves, and their readers, that the grievances reflect partisan or tribal sentiments.

But if the fact-checkers won’t listen to their critics, here’s hoping they’ll listen to one of their declared friends, the press critic Jay Rosen, who offered one of the most acute critiques of Politifact’s choice. The Republican proposal would have phased out an open-ended, single-payer, fee-for-service health care program for senior citizens in favor of a very different program that would, in effect, give future seniors vouchers that could be used to purchase insurance on the private market. If you know what the plan would do, whether or not it amounts to “ending Medicare” is an endlessly debatable point. The potential objection to the Democrats’ language is that voters who didn’t know what the GOP plan entailed could be misled—which might make that language problematic, but doesn’t make it a lie, as Rosen noted in a Dec. 22 post. “I don’t think Politifact chose a lie of the year in 2011,” he wrote. “Their sights were set on something different, and they erred by calling it what they called it.”

In fact, the sights of the broader fact-checking movement often seem to be set on something different than strict truth and falsehood. And by acknowledging that, the fact-checkers might grapple with some important questions about the project in which they’re engaged—and might see more clearly the box in which they’ve trapped themselves.

To get at those questions, it’s helpful to think about why “fact-checking” has emerged now. I’d argue that it’s a response to many journalists’ perception that they are ever more outgunned by the increasing volume and sophistication of professional political communication. The fact-check is a tool with which reporters can rescue themselves from oblivion. And the morally freighted language invoked by full-time fact-checkers—true and false, fact and lie—is a weapon, to be wielded by journalists with authority against other, presumably less trustworthy types who make political claims. (At the same time, the framing implicitly exalts a certain class of “fact-finding” journalists above workaday hacks, as a peeved Ben Smith noted in a November story for Politico.)

And two cheers for that. In the face of more and more skillful spin, it’s encouraging that journalists are thinking about ways to be more than, in Todd Gitlin’s poignant phrase, “connoisseurs of our own bamboozlement.” I’m entirely in agreement with The New Republic’s Alec MacGillis, and the CJR editorial he cites, that there are problems with assigning specific teams of reporters to call bullshit on political nonsense, rather than expecting all journalists to do so in the course of their work. Still, it’s good to see the impulse to push back—to assert that good reporting can be an independent source of knowledge about the world—finding institutional expression.

But while the language of fact-checking is powerful, it’s also limited—and the fact-checkers’ tendency to stretch that language beyond its limitations undermines the credibility of their project.

One of the problems is that the occasions on which that language is appropriate are less frequent than you might think. As the media critic Dan Kennedy wrote last month in a terrific item at The Huffington Post, “politicians don’t flat-out lie as frequently as we might suppose.” To be sure, they do lie sometimes—and, more frequently, they say stuff without bothering to learn whether it’s true. (When that happens, CJR often joins in the effort to make sure it’s called out.) But the volume, and the consequence, of that material just isn’t great enough to sustain what Kennedy aptly calls “a veritable fact-checking industry.”

In fact, many “fact-checking” pieces actually contain counterarguments—many of which are solid, some shoddy or tendentious, but few of which really fit in a “fact-check” frame. Mark Hemingway, writing in December in The Weekly Standard, took the AP in particular to task for this, and he’s absolutely correct. Back in 2009, I dinged the AP for, among other things, “fact-checking” Sarah Palin’s claim in her memoir that she was beckoned by purpose, rather than driven by ambition. (The bizarre “check” to this assertion of “fact” was that Palin’s book “has all the characteristics of a pre-campaign manifesto.”) Hemingway compiles some similarly ridiculous examples, the most egregious of which is probably a “fact-check” of Tim Pawlenty’s claim in an op-ed column that “Obamacare is unconstitutional.” Wrote the AP: “Obama’s health care overhaul might be unconstitutional in Pawlenty’s opinion, but it is not in fact unless the Supreme Court says so.” As Hemingway writes, this line implicitly embraces the idea of judicial supremacy, which is a contested theory in legal circles. Even more absurdly, it acknowledges that Pawlenty’s view is an opinion! (I would have preferred the term “argument,” which captures a slightly different concept.)

Legal arguments, which by definition involve changing and contested terrain, pose a particular problem for the “fact-check” frame, and the AP isn’t the only outlet to stumble. Earlier in December, Salon’s Glenn Greenwald persuasively pummeled Politifact’s Texas operation for assigning a “mostly false” ruling to Ron Paul’s claim that new legislative language governing the United States’s conflict with Al Qaeda and the Taliban makes American citizens “vulnerable to assassination.” In support of its claim, Politifact cited a pair of like-minded experts who said, essentially, “move along; nothing to see here.”

As Greenwald contends, Politifact’s handling of Paul’s (admittedly poorly argued) statement was obtuse. The legal environment around civil liberties is shifting in a way that gives the executive branch more power. The president now claims the authority to target for killing, without judicial review, an American citizen who is believed to be engaged in terrorist activity. That’s a situation that would have been unthinkable a little over a decade ago, and is still frightening to a lot of people. The new legislation’s language was intended to affirm, and arguably to broaden, the theories that support the government’s claim to that authority; Paul, alarmed by this development, was warning about where he feared it would lead. Politifact’s experts can certainly argue that his warning is overstated or unwarranted. But they can’t prove it “false,” mostly or otherwise.

So why have the fact-checkers ventured into uncheckable territory? Kennedy explains it as a simple matter of supply and demand—to keep content flowing to their sites, the operations need to expand their reach beyond clear untruths. There’s probably some truth to that.

But I think the situation also reflects exactly what critics like Hemingway and Greenwald allege: fact-checking, as practiced, is in part an effort to shape the public political discourse; the fact-checkers have set their sights on identifying not only which statements are true, but which are legitimate.

This impulse is reflected in the consensus among fact-checkers that the “end Medicare” attack was one of the worst offenses of the year. (In addition to Politifact’s designation, both Factcheck.org and the Post’s Kessler included it on a list of year-end lowlights.) The claim is, at most, semantically untrue. It was more plausibly unfair and, in some contexts, potentially deceptive. But some of the ways in which the Medicare attack was presented were indisputably uncivil, and arguably demagogic.

And that seems to be a big part of what bothers the fact-checkers. In their year-end write-ups, both Politifact and Factcheck.org point to an inflammatory ad produced by a liberal advocacy group that shows an elderly senior citizen being dislodged from her wheelchair and dumped over the side of a cliff. It was something of an odd choice, as an earlier Politifact item on the ad itself found its claim that the GOP plan would “privatize Medicare” to be “mostly true.” (A separate item found the ad’s assertion that the plan would leave the country “without Medicare” to be “false.”) But the ad’s outrageous imagery buttresses a larger point the fact-checkers want to make: the “end Medicare” line was a bit of Democratic demagoguery designed to scare seniors, which has no place in responsible political debate.

Similarly, Kessler began a broadside against a Democratic group’s anti-Mitt Romney attack ad by bemoaning the disappearance of “civil discourse in politics.” And while his post faulted the ad for alleged inaccuracy, notions of accuracy and civility can be wound together. Among other things, Kessler faults the ad for using footage of Romney saying, “Don’t try and stop the foreclosure process. Let it run its course and hit the bottom,” because Romney’s initial remarks had included a fuller explanation of his argument, which was omitted from the ad. Adding that extra context might have been more civil to Romney, but the ad accurately quoted what he said, and, more importantly, what he said is what he meant—more context might have added nuance, but it wouldn’t have changed meaning. As with Pawlenty and Paul, whose arguments were faulted because they hadn’t been validated by a particular arbiter, these ads represent political speech that is deemed by the fact-checkers to be in some way illegitimate.

Though I think the fact-checkers’ ideas about what constitutes legitimate discourse are, at times, dangerously narrow, I’m not as troubled by this impulse as are critics like Greenwald and Hemingway. For one thing, a lot of what makes it into these not-quite-fact-checks is useful information that provides important context about what politicians are saying. For another, the work of the press is (like the work of any profession) necessarily value-laden, and as values go, promoting civil political discourse isn’t a bad one.

But here’s where the fact-checkers find themselves in a box. They’ve reached for the clear language of truth and falsehood as a moral weapon, a way to invoke ideas of journalists as almost scientific fact-finders. And for some of the statements they scrutinize, those bright-line categories work fine.

A project that involves patrolling public discourse, though, will inevitably involve judgments not only about truth, but about what attacks are fair, what arguments are reasonable, what language is appropriate. And one of the maddening things about the fact-checkers is their unwillingness to acknowledge that many of these decisions—including just what constitutes “civil discourse”—are contestable and, at times, irresolvable.

Politifact and its peers can call the “end Medicare” line a “lie” all they want and never convince anyone who doesn’t see it that way, because there’s no empirical disagreement at issue, and there’s nothing in the journalistic toolkit that can confer ultimate authority about the questions that are in dispute. You can’t report your way to the conclusion that it’s okay to say, as leading Republican Mitt Romney has, that the GOP plan will “fundamentally transform Medicare,” and basically okay to say, as President Obama has, that the plan would “end Medicare as we know it,” but that it’s an egregious falsehood to simply say it would “end Medicare.” The argument about the legitimacy of that language is ultimately political, not journalistic, in nature. By insisting otherwise, and acting as if journalistic methods can resolve the argument, the fact-checkers weaken the morally freighted language that’s designed to give their work power—language that all journalists who are able to report their way to authority on a particular subject need to employ when it is justified.

If you'd like to get email from CJR writers and editors, add your email address to our newsletter roll and we'll be in touch.

Greg Marx is a CJR staff writer. Follow him on Twitter @gregamarx.