Factchecking made great strides during the 2012 campaign, but were those advances compromised by the pressure to maintain partisan balance?
Two respected Washington think tank scholars say yes. Thomas Mann of the Brookings Institution and Norman Ornstein of the American Enterprise Institute, who have recently argued that Republican extremism is to blame for many of the pathologies of Washington, told The Huffington Post’s Dan Froomkin last week that journalistic factchecking of the 2012 campaign may have been counterproductive:
Mann and Ornstein said that in practice, the fact-checkers may have made things worse rather than better.
“We had these little flurries of fact-checking—which I found not worthless, but not a substitute for coherent, serious reporting—and most of the time it just got stuck in the back of a news organization’s output and there was no cost to a candidate of ignoring it,” Mann said.
And then there was this terrible irony: “Fact checkers almost seemed obliged to show some balance in their fact checking.”
“There was some damn good stuff done, and stuff that really did hold Romney to account,” Ornstein said. But no fact-checker intent on “appearing to be utterly straightforward, independent, and without an axe to grind, is going to actually do the job of saying that we’re going to cover 20 fact checks on one side, to three on the other.”
So, Ornstein concluded: “If you looked at where the scales should have been, and where they were, they were weighted. And they weren’t weighted for ideological bias. They were weighted to avoid being charged with ideological bias.”
However, Mann and Ornstein’s factual premise about the factcheckers seems to be flawed. As Mark Hemingway pointed out in The Weekly Standard, PolitiFact actually has tended not to be balanced in the ratings they assign; neither has Glenn Kessler in his Washington Post factchecker column. Both have tended to give more negative ratings to Republicans (and have been accused of having a liberal bias for doing so!). Perhaps Mann and Ornstein believe that accurate factchecking would be even more asymmetric, but their statements suggest incorrectly that the ratings were artificially balanced.
The underlying problem here is the difficulty of definitively establishing which side is “worse” when it comes to taking liberties with the truth. While there are statistical methods we can use to compare polarization in Congressional voting between Republicans and Democrats, it is less clear how to determine whether one side is more irresponsible when it comes to factual accuracy. Some have tried to do so using ratings from the factcheckers. However, the process by which statements are selected to check is not random, so the evidence is only circumstantial—differences in average ratings between the parties don’t necessarily establish that one side is worse. In addition, while the factcheckers work quite hard to apply their ratings consistently, the categories are inherently subjective, which makes it difficult to rely on them as comparable indicators of accuracy. And even if one accepted both the factcheckers’ selection process and their ratings, it would still be reasonable to argue that statements in question and the issues they concern vary in their importance.
More fundamentally, it’s not clear what Froomkin or Mann and Ornstein would want factcheckers to do differently. Specialized factcheckers should not be expected to address the question of which side is generally less accurate—the question is simply beyond the scope of the format, which focuses on assessing the accuracy of discrete statements, not making sweeping generalizations about the state of American politics. Even if the factcheckers agreed with claims like those made by Mann and Ornstein (which they’ve generally disavowed), it’s not clear what they could or should do differently. Factcheckers are being asked to maintain a commitment to following the facts wherever they take them while also maintaining a reputation as non-partisan and fair. Of course, they should be fearless in following the facts in any given case, but focusing on asymmetry would destroy their credibility as honest brokers while adding little to the value of their work. Indeed, it has the power to create perverse incentives—should factcheckers stop factchecking whichever side is less bad at any given moment? Why?
Brendan,
Straw man alert!
Neither Mann nor Ornstein nor I suggested that the "factcheckers" had struck a perfect 50-50 balance. The point is that they seriously weighted the scales just to keep Democrats even vaguely in the running. In reality, the skew was almost incomprehensible, with one party basing nearly its entire campaign on lies, and the other fudging here and there.
Rather than risk being charged with ideological bias, the "factcheckers" went to often absurd lengths to appear evenhanded. The ultimate example, of course, came about a year ago. I don't understand how anyone could ever take Politifact seriously again after its decision to designate “Republicans voted to kill Medicare” as the 2011 “lie of the year” -- when it was absolutely true. See, for instance, my story here.
The trend continues, with "factcheckers" finding fault with true statements by Democrats to maintain their pox-on-both-your-houses superiority. See, for instance an example from this very morning.
I certainly agree that real factchecking should be integrated into news stories. Indeed, my point was that real factchecking -- exposing the GOP's vast credibility problems -- should have been the main news story of the 2012 campaign.
Dan
#1 Posted by Dan Froomkin, CJR on Tue 11 Dec 2012 at 04:33 PM
Hi Dan -
Thanks for the response, but I think the kinds of points you want to make don't fit well into either the fact-checking format or the non-partisan press more generally. In the conclusion I tried to suggest how reporters could account for asymmetry where and to the extent it exists, but I don't know how that could be the "main news story of the 2012 campaign." Remember, some journalists and operatives think (probably incorrectly) that Obama won on the strength of his misleading ads attacking Romney's record at Bain!
I won't defend the lie of the year selection because I've been critical of the terminology of "lie," the award itself, and the choice, but I don't think it's fair to judge PolitiFact's (massive) body of work on that basis.
Best-
Brendan
#2 Posted by Brendan Nyhan, CJR on Tue 11 Dec 2012 at 04:57 PM
Brendan,
You would really surrender the crucial role of reporting on the credibility of public figures to the partisan press? I'm not ready to go there yet, sorry.
This is not beyond the purview of an active, skeptical, non-partisan press, this is PRECISELY ITS JOB.
And I call a lie a lie. Why don't you?
Dan
#3 Posted by Dan Froomkin, CJR on Tue 11 Dec 2012 at 05:04 PM
Thank you, Dan Froomkin, for your comments. I agree with everything you said here. Brendan Nyhan asks what "fact-checkers" could do differently. As I've said in this space before, I believe they should stop using those bogus, subjective rating schemes like Pinocchios, and simply focus on identifying the evidence for and against the accuracy of claims made by politicians and other public figures. Politicians sometimes make statements that can legitimately be called "lies," but more often their dubious statements arise from their interpretation of facts and events which may or may not be supportable by evidence. For instance, Republicans claim raising marginal tax rates on wealthy people will stymie economic growth. While many call that a lie, it's their historically unsupported interpretation or prediction. Fact checkers should look to the evidence supporting or rebutting that claim, but it's not supportable to call that a lie. A lie, like a crime, requires a showing of wrongful intent. Democrats say converting Medicare into a voucher program will destroy Medicare. Fact checkers have called that a lie. But that's Democrats' prediction based on analysis of the evidence (I happen to agree with that prediction). As with these examples, many many political statements can't be categorized as lie or truth, or even shades in between. That's an intellectually dishonest exercise and fact checkers should stop doing it unless they come across a statement for which there is evidence that it's a deliberate falsehood (Romney's debate statement that he had no plans to cut taxes). BTW, Brendan, you never responded to my question about whether you now think you engaged in false equivalence in criticizing reporters and pundits for going after Romney on his 47% comments, which I predict will follow him to his grave.
#4 Posted by Harris Meyer, CJR on Tue 11 Dec 2012 at 07:04 PM
Fact-checkers are clearly valuable, but as Dan points out, their value is limited when there are no consequences for ignoring them. See, for example, Paul Ryan's convention speech, a litany of already-debunked lies. Ryan lied to the public and press, knowing that many of them knew he was lying, showing a disregard for the press and disrespect for the public that borders on pathological. For contrast, see Romney's Jeep lie, so obviously false that the media had no choice but to denounce it. How many undecided voters did Romney alienate with that one? We will never know, but far too late, the media finally got up the gumption to say "enough."
#5 Posted by Dave Saldana, CJR on Tue 11 Dec 2012 at 10:47 PM
Guess my little post over here stirred the pot. Sorry for dragging you in here Dan, not that I don't enjoy the visit.
Let's go into this:
"However, Mann and Ornstein’s factual premise about the factcheckers seems to be flawed. As Mark Hemingway pointed out in The Weekly Standard, PolitiFact actually has tended not to be balanced in the ratings they assign; neither has Glenn Kessler in his Washington Post factchecker column. Both have tended to give more negative ratings to Republicans (and have been accused of having a liberal bias for doing so!). Perhaps Mann and Ornstein believe that accurate factchecking would be even more asymmetric, but their statements suggest incorrectly that the ratings were artificially balanced."
Dude, the point was not that the ratings, pinnochios, or pants on fire ratings weren't balanced nor should be balanced.
The point was that modern republicans are completely gonzo because they've been so enabled by accountability free politics and press that they don't have brakes anymore when it comes to lying. About everything. Nearly all the f*cking time.
So why are the ratings close? No they aren't even - democrats get more truthful ratings but that's an implication of democrats being more truthful. (Because at the hint of a lie, a democrat will usually get the sh*t kicked out of them)
No it has to do with this:
http://www.desmogblog.com/more-evidence-republicans-are-more-factually-challenged-democrats
"I then carried out, with the help of a research assistant named Aviva Meyer, a similar analysis of 315 fact-checks by The Washington Post from 2007 through 2011. And the punchline is the same: Republicans fare worse than Democrats, especially when it comes to the worst ratings (4 Pinocchios, "pants on fire").
I find these fact-checker data particularly compelling, by the way, for the following reason: Neither PolitiFact nor Glenn Kessler (who writes the Post’s column), think of themselves as liberal partisans. To the contrary, I would argue that both go too far in trying to ding Democrats and liberals, just to make themselves appear balanced (and, presumably, to keep getting their calls returned by the other side of the aisle). Therefore, if their data shows Republicans fare worse, that really says something.
Indeed, I was so convinced of these fact-checkers' need to hug the center that I expected Kessler to try to course correct in the wake of my analysis showing that he rates Republicans worse than Democrats. I figured he would try to ding Democrats a little more, so as to even the score a bit and appear less biased.
It now appears that I was wrong in that assumption. Kessler--a thoroughly honest guy, there's no disputing that--has now extended the dataset across the first six months of 2012. And he finds, lo and behold, numbers just like mine. More specifically, Kessler gets the following:
In the past six months, we had 80 Fact Checker columns that rated Republican statements, for an average rating of 2.5 Pinocchios, compared to 56 that rated statements of Democrats, for an average rating of 2.11.
My overall finding for 2007-2011 was 2.45 (Republicans) to 2.12 (Democrats). In other words, I’m on precisely the same page with Kessler, data-wise.
And the data are all the more striking in that I believe Kessler inflates the Democrat total, intentionally or otherwise."
And you can read Mooney's examples for Democrat lie inflation. cont..
#6 Posted by Thimbles, CJR on Tue 11 Dec 2012 at 11:51 PM
But that isn't the entirety of Mann and Ornstein's point. I am of the opinion that factchecking has made a difference to the dialog in general because it is slowly introducing accountability for speaking debunked lies in the realm of the rational audience. We no longer have the press vs Gore dynamic in which republican lying just gets mainstreamed without thought, without criticism. People are able to push back and use the "balanced and non-partisan" factcheckers to do so.
But it's still limited in its effect.
When you confront the right who are mass marketing myths constantly popping out of hack think tanks and the mouths of talk radio shock jocks, when their own respected reps are making clueless and reckless statements, when their own devotees with a shred of rationality are leaving the movement en masse because of their paranoid style, what is their answer?
"What are you going to do about it?"
They have no shame, they have the funding to place little bullhorns on every network, and they own many big bullhorns. The factchecker can whisper "Liar pants on fire", but the bullhorn keeps on shouting.
"716 billion!" "Benghazi!" "Welfare without work!"
Political parties with integrity would stop producing advertising that repeated debunked lies. Media enterprises with integrity would stop airing them.
We have neither.
So what are we going to do about it? Fact checking is a start.
#7 Posted by Thimbles, CJR on Wed 12 Dec 2012 at 12:10 AM
And I guess it should be mentioned that Dan Frookmkin was pushing factchecking before it was cool.
"Knowledgeable beat reporters aren’t just stenographers, they are translators, educators, referees and analysts. If we’ve got people in our newsroom who really understand how a certain city or county works, or who are experts in certain policy areas, they should be sharing and showcasing their expertise in live discussions and blogs; should be answering reader questions and composing FAQs, should be on Facebook and Twitter, should be publishing and allowing readers to contribute to their beat notes, and should be writing and updating primers on key players and key issues. And much of the material they create for online should end up in the paper as well — quite possibly instead of the dry incremental news stories they currently produce. They should essentially become the anchor for a community of people who share an interest in that beat."
Become an anchor and ">call bullshit on people. Definitely call bullshit.
#8 Posted by Thimbles, CJR on Wed 12 Dec 2012 at 03:05 AM
I hate hate hate your cgi machine sometimes.
http://blog.niemanwatchdog.org/2006/11/on-calling-bullshit/
PS:
"Remember, some journalists and operatives think (probably incorrectly) that Obama won on the strength of his misleading ads attacking Romney's record at Bain!"
Yeah, I remember those discussions where your commenters took apart your arguments.
My personal favorite was the one where you took the fact checker's position on Harry Reid:
http://www.cjr.org/swing_states_project/another_fact-checking_fiasco.php
""We never said "lie." We said Reid provided no evidence and our reporting found no evidence."
Sorry Mr. Kessler. I assumed "Pants on Fire!" meant someone had told a lie. This pedantry stuff is complicated."
LOL!
#9 Posted by Thimbles, CJR on Wed 12 Dec 2012 at 03:51 AM
Harris Meyer gets it. A statement should only be called a lie if the intent behind the statement is to consciously and willfully deceive the audience. Establishing that a source is indeed lying can be a massively difficult problem, especially given modern understandings of how persons can, and do, deceive themselves so thoroughly and efficiently. When members of the right or the left spout their various forms of "Truth (sic) to power", they seem very often to genuinely believe the whole cloth of their narratives - despite having been presented with ample evidence that their narratives just don't jive with the known facts. There even seems to be evidence that facts contrary to a preferred narrative simply reinforce belief in the preferred narrative.
So it is the job of a journalist to maintain strict objectivity and high skepticism (NOT impartiality) in their professional work. A mere reporter - a non-professional, an amateur, perhaps a mouthpiece for emotion - can possibly be forgiven for failing to use care and qualifiers in their language. A journalist should be allowed no such tolerance.
In the end, it is not whether politicians lie, it is whether they understand and reasonably interpret current, objectively demonstrable knowledge. One hopes that the electorate wants politicians who can and do publicly demonstrate such capacities. And one hopes that journalists will point out when politicians' interpretations fall short of showing understanding and reasonable interpretation.
#10 Posted by Oldstone50, CJR on Wed 12 Dec 2012 at 11:43 AM
Since Nyhan asked what fact checkers could do differently, the other thing I would strongly recommend is that they evaluate the statements of public figures using common understandings of what those figures said, rather than engaging in Talmudic parsing of their statements. I wish I could think of an example offhand, but it was very common during the campaign for the fact checkers to engage in tortuously legalistic parsing of words and turn the public figure's statement into almost the opposite of what an ordinary person would have taken it to mean. This is not the Supreme Court, this should be an effort to get at common meanings and understandings.
#11 Posted by Harris Meyer, CJR on Wed 12 Dec 2012 at 12:01 PM
It's nice to see Brendan Nyhan stepping up his game with a good set of criticisms of the journalistic fact check industry.
I'll just quickly repeat some of the observations I've made over the years that Nyhan hasn't already mentioned:
Because of the selection bias problem, the subjective standards journalists use to rate political statements serve better as a measure of journalistic bias than of truth tendencies by party. Chris Mooney take note.
http://www.politifactbias.com/p/research.html
Second, journalists just aren't up to the job of getting significantly beyond "he said, she said" journalism. Journalists end up making calls on issues where experts disagree (see PolitiFact on the Obama "apology tour"). Does that make any sense at all?
I appreciate the suggestion from Harris Meyer that fact checkers take greater care in obtaining the correct understanding of a speaker's statement. It's something I try to emphasize at the fact check site I've started. But it's not as easy as Harris appears to suggest. One can't assume that a normal interpretation matches the speaker's intent. A thorough fact check needs to consider the range of possibilities and assess them for plausibility. Again, not the sort of thing the ordinary journalist is well-equipped to accomplish.
#12 Posted by Bryan White, CJR on Wed 12 Dec 2012 at 02:27 PM
Typical.
Conservative 1990's: "We don't like journalists. We're going to make up our own journalism!"
Conservative 2000's: "We don't like Wikipedia. We're going to make up our own Wikipedia!"
Conservative ∞: "We don't like facts, we're going to make up our own facts!"
Epistemic closure Bryan.
Look it up... On Wikipedia.
#13 Posted by Thimbles, CJR on Wed 12 Dec 2012 at 03:08 PM
So "Thimbles" won't discuss the evidence because of *my* epistemic closure?
Priceless.
Why do you extoll the value of PF and WaPoFC numbers despite the problem of selection bias? Is that scientific?
#14 Posted by Bryan White, CJR on Wed 12 Dec 2012 at 03:36 PM
Fact checks should be limited to blatant, deliberate, verifiable falsehoods, such as Joe Biden's statement during the VP debate that "I voted against" the Iraq war.
Biden voted for the war. He lied, full-stop, and no one called him out on it. Biden thought - correctly - that the so called fact checking press would never follow up.
Just imagine if Romney or Ryan had said that in a debate. It would have been front page news for days.
#15 Posted by JLD, CJR on Wed 12 Dec 2012 at 03:50 PM
"Why do you extoll the value of PF and WaPoFC numbers despite the problem of selection bias?"
Look, I don't give a crap about the value of those numbers absent the context that modern republicans and conservatives do not care if what they say is objectively true, so long as it reenforces their 'narrative truth'.
If you want to complain about a scale with an elephant and a donkey on it being tilted toward the elephant, you've got to confront the fact that the elephant just weighs more. The question you should be asking isn't 'why is the scale so tilted?' you should be asking 'why has the elephant evolved into this enormous thing in the room that lies all the f@cking time?'
And I'm sorry, but getting into that discussion with someone who lobbying to make the 'War on Women' the lie of the year seems a bit timey wastey. Not liking the label for something, in this case the wide spread and multifront legislative assault on women's reproductive autonomy, doesn't make it a lie.
So yeah, what was your question again?
#16 Posted by Thimbles, CJR on Wed 12 Dec 2012 at 04:36 PM
Well, it looks like the pot's getting stirred a bit more:
http://digbysblog.blogspot.com/2012/12/mann-and-ornstein-might-as-well-be.html
Went better than their shot on the Morning Joe.
http://www.youtube.com/watch?v=3U6rcJOUQLM
#17 Posted by Thimbles, CJR on Thu 13 Dec 2012 at 01:00 AM
"I don't give a crap about the value of those numbers absent the context that (...) it reenforces their 'narrative truth'."
Doesn't it matter whether the fact checks are accurate, then? The context you talk about depends on the objectivity of the fact checks, right?
"If you want to complain about a scale with an elephant and a donkey on it being tilted toward the elephant, you've got to confront the fact that the elephant just weighs more."
But how do we know the elephant weighs more if we don't know the scale is accurate? Do we just trust your judgment?
"And I'm sorry, but getting into that discussion with someone who lobbying to make the 'War on Women' the lie of the year seems a bit timey wastey."
Then perhaps you should marvel at my patience in having a discussion with someone who unaccountably confuses me with Dustin Siggins. I'm not Siggins.
My question is how you know the elephant weighs more before using the scale, leading you to confirm the scale's accuracy after the weighing based on the results you expected.
#18 Posted by Bryan White, CJR on Thu 13 Dec 2012 at 01:49 AM
"Doesn't it matter whether the fact checks are accurate, then?"
Umm, yah, duh?
"The context you talk about depends on the objectivity of the fact checks, right?"
No, it depends on the record of the movement - which breaks down into the record of the individuals within the movement. When the movement has a record of being anti-science, anti-history, anti-minority, anti-individualized sexual expression & autonomy, anti-impoverished, and - as seen by the promotion of Breitbart and his O'keefey videos and the foreign policy of the entire Bush administration - anti-truth, that is the context.
Like, how is this not simple?
"But how do we know the elephant weighs more if we don't know the scale is accurate?"
I guess you're not the type to trust your lying eyes.
"Then perhaps you should marvel at my patience in having a discussion with someone who unaccountably confuses me with Dustin Siggins. I'm not Siggins."
Then who is Siggins? Just someone on a website? Anybody's website we know? Is he somebody who's views you disavow? Tell us how much you reject the idea that 'The War on Women' was a lie.
Let's talk about the libertarian idea that women intrinsically hold domain over their property and that there is no more intimate a form of property than that over one's body. Therefore the government should not impose itself upon this property against the owner's desires nor interfere with the owner's access to the materials she required to manage her property as she sees fit.
#19 Posted by Thimbles, CJR on Thu 13 Dec 2012 at 05:31 AM
"My question is how you know the elephant weighs more before using the scale, leading you to confirm the scale's accuracy after the weighing based on the results you expected."
I'm not going to repeat the lying eyes argument, in spite of the way you've repeated the "how do you judge a scale with an elephant and a donkey on it?" question, but I will outsource my answer to Grunwald.
"This isn’t just cognitive dissonance. It’s irresponsible reporting. Mainstream media outlets don’t want to look partisan, so they ignore the BS hidden in plain sight, the hypocrisy and dishonesty that defines the modern Republican Party. I’m old enough to remember when Republicans insisted that anyone who said they wanted to cut Medicare was a demagogue, because I’m more than three weeks old."
How do I know the elephant weighs more before using the scale? Because I have eyes that see and ears that hear and I'm more than three weeks old.
I remember this shit. In fact, I'm cursed to remember this shit.
Thanks for asking.
#20 Posted by Thimbles, CJR on Thu 13 Dec 2012 at 05:43 AM
"Thimbles" on a roll:
"Umm, yah, duh?"
But that makes it look like you're arguing the numbers are unimportant when in actuality you're arguing their importance. That doesn't seem consistent.
"Like, how is this not simple?"
It's not simple because you're using a subjective scale to calibrate/confirm the (ahem) scientific scale. And at the same time apparently aghast at my anti-science attitude.
"Tell us how much you reject the idea that 'The War on Women' was a lie."
Tell us how much you now realize that's a red herring. You've pretty much done little else but ignore and evade my arguments in this response thread.
"I guess you're not the type to trust your lying eyes"
I like dealing in specifics. That's why I gave a specific example of PolitiFact finding disagreement among experts and choosing (with no explanation) the position of three liberal experts over that of the one conservative. I guess that's not a problem if you start with the presumption that the elephant is heavier and thus ought to weigh more every time you use the scale. And it would follow that if the donkey ever weighs more that the fact checkers are doing something wrong.
Science, or something.
"How do I know the elephant weighs more before using the scale? Because I have eyes that see and ears that hear and I'm more than three weeks old."
I have eyes that see and ears to hear and I'm more than three weeks old. I don't share your opinion (I'm agnostic on the issue). So we're supposed to trust your judgment. Fortunately you used no selection bias in reaching your conclusion. Right?
#21 Posted by Bryan White, CJR on Thu 13 Dec 2012 at 01:22 PM
Let's review real simple style:
Your statement: "Why do you extoll the value of PF and WaPoFC numbers despite the problem of selection bias?"
My answer: I don't, and you shouldn't. You can use numbers to establish a pattern and then use the evidence of pattern to make a hypothesis of the pattern's basis.
Your hypothesis is that fact checkers are biased against conservatives and you point to the pattern as evidence.
But my point is that the pattern is meaningless as evidence without the context in which the pattern has taken place.
And that context is the conservative environment, both media and political, in which lies are epidemic.
It would be sorta like concluding people living in Antartica are more sensitive to cold than people in Florida, which we can see because of the Antarctic population's pattern of wearing parkas more frequently than people in Florida, without examining the temperature of Antarctica or Florida. Just because you see a pattern doesn't mean you have evidence for your pet reason of why the pattern exists.
"Doesn't it matter whether the fact checks are accurate, then?"
Now we're not talking about numbers or patterns, we're talking about the content of individual fact checks. Does it matter whether fact checks are factual? Um, yah, don't it? (And shouldn't that be taken into account when establishing your pattern? Of course that might affect the validity of your pet hypothesis...)
"But that makes it look like you're arguing the numbers are unimportant"
Oh, we're back to arguing numbers/patterns, not the content of individual fact checks. See when you jump around like this, I'm not the one getting lost. You and your readers are. Think about your argument and try making it consistent.
"It's not simple because you're using a subjective scale to calibrate/confirm the (ahem) scientific scale."
And your using an incomplete analysis to form a subjective hypothesis to the meaning of the data gathered. Numbers alone don't make for objective findings, numbers and the rigor required to understand the context of those numbers do. And in the case of republican and conservative lying / reality departure, we can go back to as recent as the end of October 2012 and look at some poll analysis if you like.
My scale for republican unfact ain't as subjective as you'd like to think.
"And at the same time apparently aghast at my anti-science attitude."
Are you? You're an individual. Your movement may be poisoned, but that doesn't mean you are. Prove me wrong. Talk to me about the scientifically established evidence for climate change and evolution.
"Tell us how much you now realize that's a red herring."
It's not a red herring. There is recent, documented, historical evidence for the legislative push against women's reproductive autonomy and someone on somebody's website thinks that should be the new lie of the year. Is it worth my time discussing the integrity of fact checking with that person? No. Are you such a person? You tell me.
Or you could ignore and evade my argument some more. It's a free country.
#22 Posted by Thimbles, CJR on Thu 13 Dec 2012 at 03:27 PM
Here's a nice little article depicting the modern fact checking process, with all its warts and wrinkles, not to mention a guest spot from Brendan Nyhan himself:
http://www.niemanlab.org/2012/08/inside-the-star-chamber-how-politifact-forges-truth-in-the-world-of-make-believe/
Worth a look as far as the fault to the process and its limits go.
Like I said before, it's a start.
#23 Posted by Thimbles, CJR on Thu 13 Dec 2012 at 04:27 PM
More "Thimbles":
"(M)y point is that the pattern is meaningless as evidence without the context in which the pattern has taken place."
Selection bias renders the data meaningless regardless of the context. Your first post in the thread appears to disregard the fact. Feel free to suggest (specifically) how your point affects *my* research. It isn't clear that selection bias plays any part in the set of numbers I used, since I look at percentages rather than raw numbers and focus on a special subset of the data.
"Just because you see a pattern doesn't mean you have evidence for your pet reason of why the pattern exists."
Is my research based simply on my perception of a pattern? Make your criticism specific. Otherwise I might be inclined to reply with something like "Just because you think /just because I see a pattern it doesn't mean I have evidence of my pet reason for why the pattern exists/ doesn't mean that you have evidence that my evidence for the reason why the pattern exists consists of me seeing a pattern."
"Does it matter whether fact checks are factual? Um, yah, don't it? (And shouldn't that be taken into account when establishing your pattern? Of course that might affect the validity of your pet hypothesis...)"
Feel free to say how it would affect my research. So far as I can tell, it only matters to my project whether the people working for PolitiFact believe that their ratings are accurate. And maybe not even then. I'd love to see your argument, if you've got one.
"Oh, we're back to arguing numbers/patterns, not the content of individual fact checks. See when you jump around like this, I'm not the one getting lost."
The topic is the reliability of the truth-telling scale. The reliability of the scale is utterly dependent on the accuracy of the fact checks, as is the value of any pattern that in turn would measure factual accuracy. A slug should be able to make that jump with ease.
"And your using an incomplete analysis to form a subjective hypothesis to the meaning of the data gathered."
Looking past the "you, too!" fallacy, you just make it appear that you haven't even looked at my research. It's okay if you don't look at it. But if you don't look at it then you probably shouldn't try to evaluate it. If you think the analysis is incomplete then suggest what you think is missing (be specific). If you think the hypothesis is subjective then do something to counter the objective evidence in support. Illustrate your point with specifics. When I point to survey data showing the leftward lean in the general population of journalists, for example, you can show your evidence that PolitiFact hires conservative journalists to help counter that innate bias.
"And in the case of republican and conservative lying / reality departure, we can go back to as recent as the end of October 2012 and look at some poll analysis if you like."
I do love specifics. :-)
But let's deal with the main issue first before trying to net all the other topics you've introduced. Otherwise they may serve as distractions.
"someone on somebody's website thinks that should be the new lie of the year. Is it worth my time discussing the integrity of fact checking with that person? No. Are you such a person? You tell me."
Is it worth trying to discuss integrity of fact checking with someone when they make unsupported assumptions about what I've written? I'm amused that you want to return to this issue. Our site does not detail and does not endorse Siggins' argument about the "War on Women" being the "Lie of the Year." My co-author Jeff D. wrote that Siggins put together a good argument to make his point. I agree with him that it's a good argument and we encouraged people to read it (offsite). At the same time, I've always maintained the position that the "Lie of the Year" is a silly thing for a fact check site to
#24 Posted by Bryan White, CJR on Fri 14 Dec 2012 at 03:42 AM
(oops, over the character limit, but not by much)
a silly thing for a fact check site to engage in, and the principal value of Siggins' work, as emphasized in our review, was the fact that he got PolitiFact editor Bill Adair to admit that the choice is subjective. Given that the choice is subjective, do you think it makes sense to decide with whom to discuss issues using the litmus test of whether they think a particular candidate should win the subjective prize? Maybe I'll refuse to discuss things with you based on your preferences for the Emmy Awards. Heh.
"Or you could ignore and evade my argument some more."
And here I thought I'd been laboring to get you to coherently express your argument without allowing you to get sidetracked. The closest you've come to expressing your argument clearly, AFAICT, is in the first statement I quoted in this reply. I look forward to seeing your reply to my critique of your point.
#25 Posted by Bryan White, CJR on Fri 14 Dec 2012 at 03:47 AM
Like I suspected, this is getting timey wastey.
"Selection bias renders the data meaningless regardless of the context."
What data? As I mentioned, you are talking about two types of data:
1) the numbers of factcheck ratings - democrats vs republicans. You use this data to make the argument that fact checkers pick out (or maybe you prefer 'pick on') republicans more.
2) the content of individual factchecks.
By your words, you are either using data rendered "meaningless" by selection bias to establish selection bias...
or you are claiming that the content of an individual factcheck is rendered "meaningless" because of selection bias.
Which is it?
"Is my research based simply on my perception of a pattern?"
Nope, there is a pattern, it's no illusion of the senses. Is it evidence of what you believe?
I say no. Just like the pattern of parka wearing Antarcticans does not establish people from the Antarctic are different from people in Florida, the finding that factcheckers find more republican claims needing check does not establish that factcheckers pick on republicans more or worse.
It is more likely that republicans and democrats work in different political climates. One has a climate where lying is what you breathe day in and day out. The other isn't as bad.
And boy, do these different climates cause problems for factcheckers because the numbers that result from having to fact check one side way more often on way more serious lying really give the impression that these factcheckers are biased against one party.
And so we see them, me, Harris Meyer, Dan Froomkin, Chris Mooney, and many many others, see factcheckers fudge the factchecks on individual cases on liberal and conservative sides to get things a bit more balanced.
For example:
http://www.cjr.org/campaign_desk/what_the_fact-checkers_get_wro.php#comment-56187
But if you are just looking at numbers and not the factchecking content, if you are just looking at numbers and not the context of these factcheck patterns, you miss the real story.
Continually.
And it looks like this ain't the first time you've had this discussion with people.
So yeah, timey wastey. Final notes..
#26 Posted by Thimbles, CJR on Fri 14 Dec 2012 at 05:17 AM
"At the same time, I've always maintained the position that the "Lie of the Year" is a silly thing for a fact check site to engage in."
Good, something upon which we can agree (though I understand why Mr. Adair and others would want to pat themselves on the back for work that felt relevant, which is really what the "Lie of the year" is about.)
So when it comes to Siggins view of the conservative record against women's reproductive etc, you are not of the mind that this is somehow a lie or overblown hysteria used for political gain. You do acknowledge the fact that republican parties in many states have pushed bills drafted by ALEC and their little cousin Americans United for Life (AUL) which have put the boots to the privileges of a host of groups?
Just to be clear, we do share the same reality on this issue?
#27 Posted by Thimbles, CJR on Fri 14 Dec 2012 at 05:25 AM
the finding that factcheckers find more republican claims needing check does not establish that factcheckers pick on republicans more or worse.
It is more likely that republicans and democrats work in different political climates. One has a climate where lying is what you breathe day in and day out. The other isn't as bad.
Maybe. However, a conservative might point out that Rush Limbaugh does a lot of fact-checking. He calls himself "America's truth detector." The incorrect statements he points out were generally uttered by Democrats. So, a conservative might (wrongly) claim that:
the finding that Rush Limbaugh finds more democratic claims needing check does not establish that Limbaugh picks on democrats more or worse.
It is more likely that democrats and republicans work in different political climates. One has a climate where lying is what you breathe day in and day out. The other isn't as bad,
My point is that once you admit that the fact-checker may be politically biased, there's no objective way to decide whether his favoritism is due to bias or to real differences between the two parties.
#28 Posted by David in Cal, CJR on Fri 14 Dec 2012 at 11:10 AM
Yet more "Thimbles":
"What data?"
The data accounting for the patterns you were talking about in the context to which I replied. Context is a wonderful thing. I'm not talking about two kinds of data, contrary to your assertion. I'm talking about the data accounting for the pattern you mentioned.
"By your words, you are either using data rendered "meaningless" by selection bias to establish selection bias...
or you are claiming that the content of an individual factcheck is rendered "meaningless" because of selection bias."
lmao
Non sequitur, and a false choice. You either haven't read my research or you just plain don't get it. If you can pin down how selection bias affects my research then do it. I'm not looking at patterns that suggest truth-telling trends by party. You're the one doing that. I'm only looking at statements deemed false. The pattern we're talking about is the one you think suggests differential truth-telling by party. You're either confused or deliberately trying to confuse the issue.
"Nope, there is a pattern, it's no illusion of the senses."
You've misunderstood my question. I'm asking if you think the pattern is the total of my evidence, which is what you appear to suggest.
"But if you are just looking at numbers and not the factchecking content, if you are just looking at numbers and not the context of these factcheck patterns, you miss the real story."
Your unintended irony is pretty overwhelming. I've published more individual reviews of PolitiFact fact checks than probably any other living human, and you don't appear to have any familiarity at all with my research approach even after I've described it to you to some degree. I don't look at the sets of numbers that Mooney and Ostermeier look at. I put a special focus on the line of demarcation between "False" and "Pants on Fire." PolitiFact defines the difference between the two in subjective terms ("ridiculous"), and I challenge anybody to find PolitiFact executing the distinction in practice according to some set of objective criteria. Since I completed the first set of research, Adair has admitted that the "Pants on Fire" category was created as a sort of light-hearted joke. They apparently never got around to creating a serious distinction between the two bottom categories.
"And so we see them, me, Harris Meyer, Dan Froomkin, Chris Mooney, and many many others, see factcheckers fudge the factchecks on individual cases on liberal and conservative sides to get things a bit more balanced. "
The examples of errors in individual fact checks runs strongly against conservative subjects. I think liberals have a point that PF chooses stories to balance things out. That helps explain the numbers Eric Ostermeier collected (approximately equal number of fact checks for both parties despite the professed criterion of choosing stories on the basis of interest). That tendency ends up with PF checking Democrats on soft stories, at least at the national level. The record of the states is incredibly mixed. Which, of course, throws a spanner in the works for people like Mooney who want to use the data (of course the scale makes elephants look heavier because elephants are heavier!) to reinforce their elephantine preconceptions.
"And it looks like this ain't the first time you've had this discussion with people."
I was correct then, as now. Though the specific subject then was different so I don't know why you would think it relevant in the current context.
"So yeah, timey wastey. "
Indeed. Your distractions have proved a waste of time and you've yet to produce any specific criticism that applies to my research. So you're done?
#29 Posted by Bryan White, CJR on Fri 14 Dec 2012 at 02:41 PM
"However, a conservative might point out that Rush Limbaugh does a lot of fact-checking. He calls himself "America's truth detector....My point is that once you admit that the fact-checker may be politically biased, there's no objective way to decide whether his favoritism is due to bias or to real differences between the two parties.""
You have a point. In the case of the motivated checker, yes it's likely his selection will be biased. The questions you then have is:
a) does that render the content of his fact checks "meaningless"?
b) does his selectivity reflect the state of the subject measured ("truth in politics") or does it reflect a preference?
c) is that preference based on different ideals that are consistently applied to all individuals (say "the libertarian defined role of government") or is the same ideal measured one way for one individual (the D) and another for another (the R)?
Let's take a look at some examples:
a) Media Matters has a selection bias. Are their fact checks meaningless? No because they put the work required to document the claim accurately and refute the claim based on established work or gathered fact. There may be a selectivity to their choices but there's an objectivity to their work. The factchecks have value - though
b) in isolation, the frequency of their fact check choices may distort the reality of the political conversation. People may get the impression that talk radio and FOX only lie and race bait when, in fact, they have to make room for commercials (I kid).
People who use media matters, however, don't live in an isolated world and can accept when an organization that has lied in the past breaks a truthful story. It's requires extra verification, of course, but we are willing to accept the evidence of say Anthony's Weiner.
c) Reason and CATO have an agenda and a preference that's plain to see, but that's a preference that's fairly consistently applied.
(In fact, it was a CATO author that coined the term epistemic closure) And so that preference can be easily accounted for as far as assessing objective value (say in the benefits to be gained by
privatizingdeveloping personal accounts for social security).For organizations like FOX or Limbaugh, however, there isn't much in the way of ideas or values to define them. They are defined by their group which then defines the standards, they do not consistently apply their standards to define their group. And when their group is threatened by truth, they attack the truth or the teller.
Their central priority is the welfare of the group.
Which means that you cannot rely on them in isolation to get a true picture of the universe we live in. And, unfortunately, that is what many conservatives do. (Thus the closure) Cont..
#30 Posted by Thimbles, CJR on Fri 14 Dec 2012 at 02:48 PM
What we should expect from organizations that are purporting to express truth is a process of verification. Information consists of beliefs and 'facts' and the only thing that separates the two is the way reality affirms the fact. When reality stops affirming 'a fact', it stops being one.
It is through the rigorous testing of our beliefs that we establish facts 'to our best knowledge'. The universe is objective and fact checkers and truth tellers must strive to accurately reflect it.
The difference between modern conservatives and liberals is the effort expended to ensure our beliefs and words reflect reality. Most conservatives don't. Why? Perhaps it's the idea that in culture war, truth is a casualty. Perhaps it's the idea that received truth needs no verification, with enough faith and blood reality will adjust to the belief.
Whatever the reason, before you can say a group is biased towards another, you have to establish that 'the pattern of bias' is a preference and not a implication of current reality.
That has not been done with politifact. There's a pattern, but is it a bias? Look at the surrounding culture and you see that it isn't. In fact, politifact is too cautious to avoid charges of bias, which is why it fails to reflect reality as best we know it - and guys like Ornstein have to conclude “If you looked at where the scales should have been, and where they were, they were weighted. And they weren’t weighted for ideological bias. They were weighted to avoid being charged with ideological bias.”
That is the world in which we live today. Do what you will with that information.
#31 Posted by Thimbles, CJR on Fri 14 Dec 2012 at 03:05 PM
"Yet more "Thimbles":"
Almost missed that post while I was composing the response to Dave...
Funny, didn't see a part in there where you affirm the reality of what Siggins, according to the article on your site, purports to be a lie. Lie of the year even.
Are we in agreement on this portion of reality or do you hold different beliefs? Should be a pretty easy question.
#32 Posted by Thimbles, CJR on Fri 14 Dec 2012 at 03:24 PM
"Should be a pretty easy question."
It is.
Red herrings are often easy questions. If you want an excuse to cut off the conversation I guess you can just assume that I agree with Siggins, forget the fact that you made yourself look bad earlier by assuming I was him, and also forget the fact that where you haven't stuck with tactics of distraction your arguments don't amount to much. Your move.
With luck, your principled stand on not having conversations with people like me will make them forget about your performance.
#33 Posted by Bryan White, CJR on Fri 14 Dec 2012 at 03:52 PM
This whole outside the bubble CJR exercise is shown as suspect simply by comparing the reactions to this year's PolitiFact "Lie of the Year" compared to last year's. Last year, many conservative critics (I was one) offered sympathy to outraged Democrats. Democrats had belatedly discovered what we conservative critics knew already: The LOTY isn't necessarily false.
This year, we have a claim in a Romney ad that is in many respects perfectly true: Jeep's going to build Jeeps in China for the Chinese market, which erases the existing market of 30k + vehicles currently shipped from the U.S.A., The Obama administration sold Jeep to an Italian company (spare me the tale that it wasn't sold since Fiat paid nothing for it. I didn't pay for it, either, but I don't own a bit of it). Then, because Romney says he'll protect American jobs, PF infers that the ad's central message is that Americans will lose jobs because of the two true things in Romney's ad. Supposedly that means the ad is saying that Jeep will move all production to China, or something. Pants on Fire! And if some Americans really might lose jobs as a result, so what?
Does nobody see the parallel between this year's claim and last year's? Yet where are the liberal critics pointing it out? Oops! They're all busy clamoring for fact checkers to stop going easy on the Republicans.
Maybe this outside-the-bubble bubble really is just another bubble.
#34 Posted by Bryan White, CJR on Fri 14 Dec 2012 at 04:14 PM
" I guess you can just assume that I agree with Siggins"
\:/ Or you could just say whether you do or not. It's your mind, your website, your life.
Defend what you associate yourself with or don't asscociate yourself with it.
If you feel it has to be more complicated than that, then that's a problem with how you do business.
Cheers
#35 Posted by Thimbles, CJR on Fri 14 Dec 2012 at 05:44 PM
More Thimbley stuff:
"Or you could just say whether you do or not. It's your mind, your website, your life.
Defend what you associate yourself with or don't asscociate yourself with it.
If you feel it has to be more complicated than that, then that's a problem with how you do business."
Likewise, you could engage the conversation rather than keeping alive the running threat of holding the conversation hostage over an unrelated issue. It's perhaps understandable that you're eager to find some method of mitigating your earlier blunder of confusing me with Dustin Siggins.
I'm well able to hold up my end up the current argument regarding the meaning of PolitiFact's ratings. Your ability is in doubt, given the path of the above conversation. If you need an excuse to bow out, I'm sure anything will do. It's your business, as you say. Pick your excuse and get it over with. Either that or answer my criticisms of your argument.
#36 Posted by Bryan White, CJR on Sat 15 Dec 2012 at 03:35 AM
Sigh.
#37 Posted by Thimbles, CJR on Sat 15 Dec 2012 at 04:57 AM
@ "Thimbles"
Heh. A Monty Python video response from the guy who first insists that it's a waste of time to argue with a person who argues the alleged Republican "war on women" was a lie, finds out he made a false assumption about whether I made that argument, after which he tries to have an argument on the waste-of-time topic of the war on women.
Brilliant.
But doubtless you're just arguing nonsensically in your spare time.
#38 Posted by Bryan White, CJR on Sat 15 Dec 2012 at 11:23 PM
"after which he tries to have an argument on the waste-of-time topic of the war on women."
Still stuck for an answer on that question, Bry?
Must be tough.
*whistles*
PS.
Meanwhile, in ALEC news:
http://digbysblog.blogspot.com/2012/12/go-ask-alec.html
http://digbysblog.blogspot.com/2012/12/chris-hayes-on-democratic-retreat-on.html
This kind of thing is why I don't need to waste my time playing rhetorical ping pong with a guy who won't recognize the facts and rules which govern this reality.
To some people, this is all a big game to score points in and yell, "IN YOUR FACE, LIB!"
As I've put here before, this ain't a game to me. This 'game' costs people's lives. Considering that cost, it's only being respectful to ask that the people involved be honest and truthful 'players'.
And if you're not willing to be that for whatever reason, I don't owe you an argument. There are more important things in life.
#39 Posted by Thimbles, CJR on Sun 16 Dec 2012 at 12:00 AM
"Thimbles" keeps trying:
"Still stuck for an answer on that question, Bry?"
I guess. I mean, I gave the answer in the first draft of my last response since I told you I'd be happy to go after your red herring topic after dealing with the present one, given you current level of failure on the latter. But then I figured "Why do that prior to "Thimbles" admitting he's got no argument on the topic we started with?"
So, yeah, no pursuit of red herrings until you answer on the real topic on which you sought out a disagreement with me: Assessment of fact checkers by their ratings.
"As I've put here before, this ain't a game to me. This 'game' costs people's lives. Considering that cost, it's only being respectful to ask that the people involved be honest and truthful 'players'.
And if you're not willing to be that for whatever reason, I don't owe you an argument. There are more important things in life."
You've never owed me an argument. But it's basic to public debate to back up one's assertions. You attacked my first post in this thread in vague terms. When I sought out the basis of your complaint in specific terms you turned evasive. I got enough out of you, I think, to indicate that you criticized me without knowing what you were talking about. But once you got yourself into trouble you got serious about your evasions.
Lately you're just holding the conversation hostage over a tangentially related (at best) matter. And apparently that's your brand of honesty. That's a brand of honesty that we can do without in the public square. Back up your criticisms or go play your serious games somewhere else.
"Let's talk about the libertarian idea that women intrinsically hold domain over their property and that there is no more intimate a form of property than that over one's body."
But you won't have that argument with someone who disagrees with you, right?
So when you ask to have the argument, you're really saying you don't want to have the argument. Great honesty technique, there, Thimbles.
Again, if you want to bail, then bail. Nobody's stopping you. You can use any excuse you like, and it's quite true that you don't owe me an argument. Though I suppose I could reasonably copy you and demand your honest reasons (specifically) for criticizing my original post to this thread. You'll have to be willing to be non-hypocritical for that tactic to work, of course.
#40 Posted by Bryan White, CJR on Sun 16 Dec 2012 at 05:59 PM
"I gave the answer in the first draft of my last response"
Oh so 'yes' or 'no' made it into the first draft of your response (lol we have 'drafts' for comments now. Internets are serious business) but it got cut on edit on account of space?
You're a card, Bry.
Ps.
" Though I suppose I could reasonably copy you and demand your honest reasons (specifically) for criticizing my original post to this thread."
My reason was this, you posted your website above, I went to it with an open mind, saw your lie of the year link which was dumb, came back and told what I thought.
Because if you think politifact and other fact checkers are biased against conservatives - based what? Michelle Bachman's pants on fire ratings? Come on - you're living in a dream world.
The conservative movement is broken. You can either fix it, distract people from the break "politifact is biased!", or make the break worse "the war on women is the lie of the year!"
But you told me "hey, just because I have something dumb on my website doesn't mean I wrote it or agree with it. You got a lot of nerve assuming stuff about me from stuff I post on my website."
Which is true, I guess, so I gave you the chance to verify or dispute my assumption.
And you won't. Why? I don't know. Perhaps you believe it? Perhaps you see criticizing your own side as a sign of disloyalty. I don't care. The fact is there is something stupid and misleading on your website which you posted the link for us to see. You don't want to say you believe something stupid and misleading, but it's there, on your site.
The problem with modern conservatives isn't their ideas, it's that they lie in the desire to spread them and lie to hide their failures. You cannot learn from error if you do nothing but lie to yourself about error. Modern conservatives refuse to learn because they won't face facts.
There's your explanation. Pretty simple, huh? Why make it hard, I say.
#41 Posted by Thimbles, CJR on Mon 17 Dec 2012 at 12:02 PM
"Perhaps you see criticizing your own side as a sign of disloyalty. I don't care."
Rage, a broken link.
http://m.theatlantic.com/politics/archive/2012/11/how-conservative-media-lost-to-the-msm-and-failed-the-rank-and-file/264855/
So I might as well post another link. Why do conservative institutions lie all the time even to the detriment of their movement? Because their purpose was never to tell the truth about the world, it was to change the language of how we talk about the world:
http://www.alternet.org/story/147473/lakoff%3A_why_conservative_lies_spread_and_what_progressives_can_do_to_fight_them
"In the US, conservatives have set up an elaborate messaging system. It starts with an understanding of long-term framing and message experts who know how to use existing their long-term frame systems. Then there are think tanks, with experts who understand the high-level frame system and how it applies to the full range of issues. There are training institutes that teach tens of thousands of conservatives a year to think and talk using these framing systems and their language and argument forms. There are regular gatherings to consolidate messaging and policy around a contemporary issue that fits the conservative moral system. There are booking agencies that book conservative spokespeople on tv, talk radio, etc. There are lecture venues and booking agencies for conservative spokespeople. There are conservative media going on 24/7/365.
As a result, conservative language is heard constantly in many parts of the US. Conservative language automatically and unconsciously activates conservative frames and the high-level framing systems they are part of. As the language is heard over and over, the circuitry linking the language to conservative frames becomes stronger. Because the synapses in the neural circuits are stronger, they are easier to activate. As a result, conservative language tends to become the normal, preferred "mainstream" language for discussing current issues."
#42 Posted by Thimbles, CJR on Mon 17 Dec 2012 at 12:25 PM
"Thimbles" goes on: "My reason was this, you posted your website above, I went to it with an open mind, saw your lie of the year link which was dumb, came back and told what I thought.
I linked to the research page. So you go to the site, ignore the research page I linked and come back and criticize my research-related comments based on a review of somebody else's unrelated work that you assumed was mine? Incredible. It's especially nice that your version of history omits your erroneous assumptions about authorship.
I'm not interested in your distractions, and it would be a disservice to you to allow you to skate on your fallacious approach to public discourse. If you can come up with a reasonable criticism of the research then do it.l Or admit that you can't and then ask me what I think about the so-called "Republican War on Women."
At the bottom line, the research stands or falls on its own. It doesn't matter if I think the moon is made of green cheese, let alone what I think about the "war on women." If you think otherwise then you are reasoning fallaciously.
#43 Posted by Bryan White, CJR on Mon 17 Dec 2012 at 02:33 PM
"So you go to the site, ignore the research page I linked"
Oh? What's this? You assumed I ignored your little research without checking with me first OMG?!
No what I did was look over it, say to myself "WTF is this? A PoF rating? Six sided dice? Seriously?"
Seriously# It was junior college level stuff# Then I looked at the rest of your website and saw "Okay, these guys are jokers#"
But alright gumdrop# Let's do the chore of going through your research#
"Though neutral judgment ought to result in approximately equal proportions of unfair “Pants on Fire” ratings, Republicans drew that designation about 74% more often than Democrats#"
Why? We're not talking about politcally neutral, we're talking about truth# Why would a measure of truth approximate an equal proportion if one side is lying more often and more egriously?
"When PolitiFact first launched, it existed as a joint venture between the St# Petersburg Times newspaper and the Congressional Quarterly magazine, both owned at the time by the Poynter Institute in St# Petersburg, Florida# The numbers for 2007 and to a lesser extent for 2008 demonstrate tendencies not present in later years###Simply put, PolitiFact’s bias against Republicans as measured by its use of the “Pants on Fire” rating has intensified since the separation from CQ#"
So from 2008 on, the "PoF" rating went up# Yeah could be because of CQ's departure, though you've offered nothing but a corelation as proof and, as every junior college kid knows, corelation != causality#
What else happened in 2008?
Oh Yeah. The republicans went batshit, like they do over every democratic administration, like they did over a blackity black democratic president.
Carpet bomb lies. Ridiculous Kenyan Communist Islamic usurper who worshiped at a black liberation church, lies. Your numbers omit that context.
"When one party ends up with a PoF Bias number greater than 1.0 the number indicates a bias against that party in the application of the “Pants on Fire” rating."
How? That only works when two groups are telling equally ridiculous lies. Republicans lie much more ridiculous, much more often.
"Lacking an objective justification for the "Pants on Fire" ratings, the numbers indicate a clear and substantial bias against Republicans in the application of “Pants on Fire” ratings."
What it indicates is that politifact has to rate falsehoods based on their smell, some smell bad, others smell worse. There are some falsehoods which are hard to determine and honest people can be forgiven believing the false claim. There are others, like Sarah Palin's death panel claims, where the speakers should know better and there's a good chance they do. Chances are they are lying, not just false.
If the true information about a claim is easily obtainable and easily discernable, like your stupid war on women claims #which I'm going to assume are yours because hey, they are on your website#, and you persist on telling false information, you merit the PoF rating.
The subjective nature of the judgement is the measure of ease to obtain and discern the true informatio. It has nothing to do with party, except in the sense one party is lying all the time. If you omit that context, and I don't care how many times you roll a six sided die, it ain't going to make your findings more robust.
"At the bottom line, the research stands or falls on its own."
It falls. So now that we've dispensed with that, what do you think about the so-called "Republican War on Women." hackity hack. Do you believe in stupid things or just post them for fun?
#44 Posted by Thimbles, CJR on Mon 17 Dec 2012 at 10:57 PM
PS. computer's a jerk tonight. Keyboard's on the fritz and it kept inserting # tags. No, it ain't a twiitter joke/reference.
#45 Posted by Thimbles, CJR on Mon 17 Dec 2012 at 11:00 PM
"You assumed I ignored your little research without checking with me first OMG?!"
What did you think the question mark at the end was for? Just something you could conveniently lop off to change the context? I didn't assume anything about what you looked at. I asked a question based on appearances. If you read it originally then we should expect you to comment on the content, as you've tried to do now. And thanks. :-)
"It was junior college level stuff"
Good. I was aiming for a reading level between mass media level and an easy-to-understand research paper.
"we're not talking about politcally neutral, we're talking about truth"
Are we? Doesn't that presuppose an objective difference between false and ridiculously false? If so, what is it? How much of the junior college reading level stuff did you just skip over?
"Yeah could be because of CQ's departure, though you've offered nothing but a corelation as proof
All I asserted was a correlation. I asserted nothing about causation. Correlation is a good proof of correlation. Excellent, even.
"That only works when two groups are telling equally ridiculous lies. Republicans lie much more ridiculous, much more often."
That only works when there is an objective means of determining that Republicans "lie much more ridiculous." So what is that means? Do you just assume PolitiFact has such a means despite the lack of evidence in its work and in its definitions?
"The subjective nature of the judgement is the measure of ease to obtain and discern the true informatio(n)."
That despite the fact that PF doesn't mention that measure on its page of definitions and despite its absence in the great bulk of PF's PoF ratings? I love that story. Tell us another one.
"It falls. So now that we've dispensed with that ..."
Oh, there's no challenging your scintillating criticisms, eh?
My use of a simulated six-sided die, by the way, ought to serve as an acceptable means of randomization--a good feature in research like this. Though if you read the paper carefully you'd have noticed that I used the technique on an investigation peripheral to the main inquiry. So it's kind of sad that you think you can use it as a significant means of distracting from the quality of the research.
We can sum up your critique in brief: You're confident that there's an objective measure that establishes the difference between Pants on Fire and False ratings. So if we read through the ratings one by one, how many times will PolitiFact mention your key criterion? What percentage of cases will confirm your assertion? Ten percent? Or would it need to be higher than that?
#46 Posted by Bryan White, CJR on Tue 18 Dec 2012 at 01:39 AM
"Are we? Doesn't that presuppose an objective difference between false and ridiculously false?"
No. There is an objective definition between truth and false and there is a subjective definition between a false reading and a "Now come on. Really?" (to be referred to as my NCOR scale) reading.
As I mentioned, the measure of ease to obtain true information and discern it from false is dependent on the experience and ability of the individual. It is subjective.
But let's assume that we expect people to have a basic amount of experience and ability to be considered competent individuals - a form of common sense if you will. If it takes less than a basic amount of ability and experience to determine a falsehood, shouldn't that deserve a special category? Like the "Obama is a communist" falsehood. Is the fact that it takes a glance at how he's rubbed the shoulders of the banks and borrowed the market based health care solution from the Heritage foundation enough to qualify that falsehood as an NCOR?
I could see the beginnings of an objective standard there, but whenever you are quantifying the degree of false based on the knowledge required to recognize its falseness, that measure of individual knowledge makes the judgement inherently subjective.
But that doesn't translate into political bias. If one side is making claims which require less than basic abilities to refute, you can't blame them for rating those as ridiculous lies. The fault lies with the party, not the analyst.
"All I asserted was a correlation. I asserted nothing about causation. Correlation is a good proof of correlation. Excellent, even."
Is there a reason you brought it up then? If you didn't want your audience to construct a causative relationship, you would have said "This is the time the data changed PERIOD" at which point your audience could analyze the time period for evidence of potential causes.
That's not what your paper does. That is a no no.
"That despite the fact that PF doesn't mention that measure on its page of definitions and despite its absence in the great bulk of PF's PoF ratings? I love that story."
A fish has trouble describing the water it swims in. When it comes to the philosophy of truth in a world where the limits of human perception and knowledge hamper our ability to know what is true, politifact people maybe aren't that sharp. I'm not always either, but in a pinch I do alright.
The fact of the matter is I'm right. An individual's ease at doing a task shapes his perception of those who can't. The more ease, the more negative the perception, the more PoFs (NCORs).
"My use of a simulated six-sided die, by the way, ought to serve as an acceptable means of randomization-"
For what purpose? Randomizing a sample is what you do when you're, say, testing a quantity against a control and you want your measurement takers to be unaware of who is getting which. You should really go to a junior college and learn some of this stuff.
"We can sum up your critique in brief: You're confident that there's an objective measure that establishes the difference between Pants on Fire and False ratings."
Wrong. Go back, reread, and try again.
Then maybe we can get that answer on that topic of yours on your website.
#47 Posted by Thimbles, CJR on Tue 18 Dec 2012 at 04:51 AM