In 2007, Wired published an issue that focused on the emergence of “radical transparency” in business.
“Get Naked and Rule the World,” it declared. The subhead for one of its major features declared, “Fire the publicist. Go off message. Let all your employees blab and blog. In the new world of radical transparency, the path to business success is clear.”
There was even an article that used Dunder Mifflin, the fictional paper company featured on NBC’s The Office, as a case study in openness. “Because in this new era of radical transparency, the way you sell paper is by showing the world that you’re not above getting reamed,” it advised.
Well, what’s good for the boardroom is also good for the newsroom. And that’s why Markos Moulitsas of Daily Kos stepped up this week to get reamed. Here’s the opening sentence of a blog post he published on Tuesday, detailing the implications of an independent review of polls published on his Web site:
I have just published a report by three statistics wizards showing, quite convincingly, that the weekly Research 2000 State of the Nation poll we ran the past year and a half was likely bunk.
Writing at Salon, Dan Gillmor said:
If there’s a Mother of All Corrections, this comes pretty close. When the proprietor of a well-known and widely followed media organization brings something this awful to the attention of his audience, and in such a forceful and prominent way, he’s doing something fairly rare — and noteworthy.
Perhaps the best measure of an organization’s commitment to transparency is to see what it does when things go wrong. Does it clam up and dribble out useless, incomplete information? Or will it embrace the brutal side of being open, warts and all? It’s much easier to be transparent with information that seems neutral or positive; true transparency is all about what you do when things go awry. Will you hold the line or retreat into the shadows?
Moulitsas made the right decision, and now he’ll have to weather the inevitable storm that comes his way for prominently displaying research that turned out to be useless. One comment (or “letter,” as Salon deems them) on Gillmor’s post does a good job of pointing out the error of Moulitsas’s ways:
The REAL issue here, that the author is avoiding, is not that R2K provided faulty data. It’s that Kos didn’t do proper due dilligence on his poll provider, and passed on the faulty polls to his readers for over a year. That is unforgiveable. Perhaps Kos won’t be so smug in the future about the research that formal news organizations engage in. Fact checking is part of being a journalist. So is ensuring the quality of materials you outsource.
Fair criticism. It’s also useful in that it highlights how Moulitsas can prevent this from happening again: by being more rigorous in checking the data his polling partner provides. Sure, Moulitsas is a journalist, not a pollster—but he’s also a publisher, and is therefore responsible for what goes on his site. That’s why it’s good that he doesn’t mince words about what needs to happen with the Research 2000 data: “I ask that all poll tracking sites remove any Research 2000 polls commissioned by us from their databases. I hereby renounce any post we’ve written based exclusively on Research 2000 polling.”
It’s that Kos didn’t do proper due dilligence on his poll provider, and passed on the faulty polls to his readers for over a year.
It's easy to point fingers, as this commenter has done. R2K had several major media clients, both print and television. This was not a fly by night outfit; they've been around for some time and had developed a solid reputation.
Mark Blumenthal at pollster.com makes the more important point:" We must push for complete disclosure as a matter of routine and we need to develop better objective standards for what qualifies as a trustworthy poll."
FWIW, Blumenthal notes two prior troubling issues with R2K here:
http://www.pollster.com/blogs/daily_kos_we_were_defrauded_by.php
#1 Posted by cab91, CJR on Fri 2 Jul 2010 at 02:32 PM
@Craig,
I have a lot of respect for you, and I enjoy your blog and appreciate your groundbreaking commitment and work on corrections and accuracy in the media generally, but your tone here is disappointing. Your language about Moulitas is needlessly and highly inflammatory -- "enamored," "cloak of radical transparency." You diminished your point, and your credibility, when you take unwarranted, vicious shots like that. It isn't even clear what you meant. You are just over the top here in taking shots like that.
First of all, it is all but impossible for anyone but a highly trained statistician to detect sophisticated fraud in statistical data. In fact, has there every been a case when a journalist independently discovered polling fraud? To assert that Moulitsas didn't do due diligence is extremely unfair, and moreover shows your ignorance about statistics and mathematics in general.
You say:
. It’s also useful in that it highlights how Moulitsas can prevent this from happening again: by being more rigorous in checking the data his polling partner provides.
You get on your high-horse to tear into Moulitsas but -- exactly HOW is he, or any journalism enterprise, supposed to go about "being more rigorous" in checking the data? You obviously don't know what you are talking about here. It takes someone like Nate Silver, a brilliant statistician, to notice or analyze potentially fraudulent statistics to make a case for intention fabrication versus random variation.
Do you happen to know just exactly how, say, Washington Post or ABC go about being "rigorous in checking the data"? In fact, the journalists in these organization regularly run with just about any poll that comes out on any given day, including the completely discredited Strategic Vision and the suspected number-finagler Rasmussen. Please inform me exactly what the legacy media does to "be rigorous" that Moulitsas didn't do. And while you are looking in to this, please see if they have ever issued a correction when they used discredited Strategic Vision polling.
In fact, the reason that Moulitsas contracted with what he thought was a reputable polling outfit was because of the shoddy polling and analysis in the legacy media. It shouldn't be surprising that he is angry that he "got burned" if he did. At the outset of his polling initiative, he made the decision to be completely transparent with the polling data precisely because of the shoddy, opaque practices of big media, and their resistance to discussing the possibility of tampering with the data. You of all people should be supportive of transparency, yet you take a vicious shot at Moulitisas for it. What's up with that?
I can't understand why someone like you is taking such a gratuitous, vicious, and unwarranted hit like this. Your credibility is considerably diminished for doing it. It's as if you have some kind of stake in saving R2K and discrediting dKos. Is that the case? Or is it just peevishness at the "new media"?
#2 Posted by James, CJR on Fri 2 Jul 2010 at 08:37 PM
@James
Thanks for taking the time to comment on my column. I think there's been a big misunderstanding -- my column is wholly complimentary of Moulitsas. The words you've interpreted as me taking "vicious shots" at him are in no way, shape or form meant as shots. I'm sincere in my praise for his handling of this matter. I hope that helps clear things up.
#3 Posted by Craig Silverman, CJR on Sun 4 Jul 2010 at 09:20 PM
@Craig,
Then I apologize for misreading you. I guess the "radical" characterization kind of set me off, and then I don't understand why you chose that first quote. In fact, Marcos DID his due diligence, and I remember when he embarked on the contracting of the poll. In rereading, I see your "radical" referred to the transparency, and not to Marcos.
In fact, it's been a frustrating exercise to try to hold the legacy media accountable for the way they use polls. Political journos grab on to any poll results available without evaluating the credibility of the poll or the methodology of the poll, and they tend to make a lot of hay out of nothing. For example, there is no way -- and reputable pollsters will tell you -- that, for example an overnight poll on a political event is in any way a valid measure of opinion. Rasmussen does this all the time, and it is so obviously a ploy to shape the narrative rather than deliver information. Why is a pollster trying to shape a narrative like that?
And too, when a poll or a polling outfit is found to be wrong or even fraudulent, newspapers just flatly refuse to issue any correction or update, or even discuss it. They are completely indifferent to the fact that they are using fraudulent information, and drawing invalid conclusions and generalizations from it. It's maddening. There is just nobody out there that is trying to keep these polling outfits honest.
Again, apologies for misreading your post.
Cheers.
#4 Posted by James, CJR on Mon 5 Jul 2010 at 08:34 AM
Sorry for how belated this comment is.
James makes a key point. To the best of our knowledge none of the traditional media who have hired and/or featured R2K or Strategic Vision have done any checking either. The problem is generic.
One word of caution- a moderately competent faker with high school math skills could easily evade the sorts of tests we have used. And, so far as we know, one or more may well be doing so at this moment.
An entirely different sort of ongoing inspection system is needed, not ex post facto stats checks.
#5 Posted by Michael Weissman, CJR on Fri 9 Jul 2010 at 11:47 AM
You write: "In the end, Moulitsas is so enamored with his commitment to transparency that he manages to use it to take a shot at his critics. It’s amazing what a cloak of radical transparency can allow you to do. The problem is so many people are quick to disrobe when the going gets tough."
I'm not sure how you can read Marcos' comments as "taking a shot at his critics". Rather, he's saying, accurately, that exposing everything is worth absorbing the inevitable criticism of his critics. "Enamored" is one of those words that is often used in a derogatory or snarky way, and a previous commenter noted that (though he ultimately saw it differently).
And your last sentence, to my eye, makes no sense at all. The problem is not that people "disrobe when the going gets tough" -- it's that they do the exact opposite. They cover up. If your point is that people only embrace radical transparency when things get tough, then the sentence needs rewording and could use some factual support, as it runs counter to the point of your piece.
#6 Posted by Dave, CJR on Mon 30 Aug 2010 at 12:50 PM