A year and a half ago, George Mason University economics professor Daniel B. Klein wrote a column about his finding that liberals scored much worse on a test about basic economics than libertarians and conservatives. The Wall Street Journal trumpeted it with this sneering headline:
Are You Smarter Than a Fifth Grader?
Self-identified liberals and Democrats do badly on questions of basic economics.
The problems with the column were obvious, as left-leaning publications were quick to point out. It was based on responses to eight questions that were almost all tilted toward conservative ideas. In some instances, it counted the correct answer as the wrong answer or counted answers as false that are in fact unfalsifiable (meaning, can’t be proven false one way or another; e.g. “blue is the best color”). For instance:
5) Third World workers working for American companies overseas are being exploited (unenlightened answer: agree). 6) Free trade leads to unemployment (unenlightened answer: agree). 7) Minimum wage laws raise unemployment (unenlightened answer: disagree).
If there’s more than one Third World worker being exploited overseas by an American company, then “agree” is the correct answer to No. 5. Surely there are two people somewhere being exploited by American firms. Less narrowly: Whether somebody is being exploited is opinion that’s not falsifiable until everyone agrees on what exploitation is. While Wall Street Journal editorial page and Mercatus types might not agree that exploited is a fair word for, say, the twenty-nine Bangladeshis who died last year sewing clothes for the GAP and Target for 28 cents an hour (after management locked them inside to prevent theft), I would, and so would most everyone else.
And there are lots of “unenlightened” economists who disagree with the premise that the minimum wage raises unemployment and that so-called free trade doesn’t kill jobs.
But regardless of the merits of his first effort, Klein ought to get some kind of award for the remarkable intellectual honesty of his second one, which recounts how he got it wrong the first time. Klein revisited his earlier findings, going back to confirm them only to find out that what he’d been confirming was confirmation bias:
But one year later, in May 2011, Buturovic and I published a new scholarly article reporting on a new survey. It turned out that I needed to retract the conclusions I’d trumpeted in The Wall Street Journal. The new results invalidated our original result: under the right circumstances, conservatives and libertarians were as likely as anyone on the left to give wrong answers to economic questions. The proper inference from our work is not that one group is more enlightened, or less. It’s that “myside bias”—the tendency to judge a statement according to how conveniently it fits with one’s settled position—is pervasive among all of America’s political groups. The bias is seen in the data, and in my actions.
But you won’t find that retraction in The Wall Street Journal. It’s in The Atlantic this month. I thought that was odd, so I asked Klein if the Journal had declined to run it. He emailed back:
Back in May I approached the WSJ person I had worked with. They declined the idea of a follow-up.
Klein, for his part, says that he isn’t “inclined to fault them for that decision.”
The WSJ, dealing with the first survey study, had great difficulty dealing with 8 policy questions (plus the ideology question) and the results.
The idea of a new op-ed, dealing with 17 policy questions, and two sets of results, is very hard to imagine.
It’s not hard for me to imagine. Actually, it’s hard to imagine a publication not finding a way to run a piece by someone effectively retracting a column it has published. All seventeen questions don’t need to be in the piece, and 800 words is plenty of room to tell the story.
I asked the Journal for comment and will update if I hear back.
- 1
- 2
So the moral of the story is we should look with skepticism on research that paints entire groups of people based on political/ideological affiliation in a negative lights especially when it plays into our own confirmation bias. Quick, someone go tell Bob Altemeyer.
But that’s probably somehow “different”.
#1 Posted by Mike H, CJR on Wed 23 Nov 2011 at 12:13 PM
No Mike, I think the moral of this story is the WSJ is as dishonest and biased as the NY Post and other News Corp Meda outlets.
#2 Posted by Marlo, CJR on Wed 23 Nov 2011 at 12:19 PM
In what world did the economist think that his original survey was anything but link bait? Now he wants a second bite at the apple? I think everyone should ignore him.
#3 Posted by Alexander Muse, CJR on Sun 27 Nov 2011 at 05:28 PM
Please. The WSJ editorials and op-ed pieces are so consistently shrill, generally snarky, and intellectually adolescent, they are essentially self-parody. The original "survey" fits perfectly within this "brand," absurd on its face. Remember, these are the same people that bring you Fox News. The WSJ is now just a slightly upscale version of that outlet, namely Fox for people who read.
#4 Posted by Arthur Libby, CJR on Mon 28 Nov 2011 at 08:48 AM
Feels like we are living in an "misinformation age" thanks to our media outlets and their business models. Yuck.
#5 Posted by Kris Tuttle, CJR on Mon 28 Nov 2011 at 09:28 AM
It would help to know whether Professor Klein offered to write a follow-up column for the WSJ gratis, or whether he wanted to, you know, get paid for it. Because it would be hard to fault the WSJ for not considering Professor Klein's second thoughts as entitling him to another publication fee.
#6 Posted by RobC, CJR on Mon 28 Nov 2011 at 10:34 AM
Professor Klein graciously responded to an inquiry from me that he didn't receive any fee for the initial WSJ op-ed, and I presume he wouldn't have received any for a follow-up op-ed. So as to my previous comment, in the words of Emily Litella, "Never mind."
#7 Posted by RobC, CJR on Mon 28 Nov 2011 at 04:50 PM
It's great that Klein recanted, but anyone with even an ounce of intellectual honesty would have known that the questions (particularly those cited in this article) were bogus from the start. And the intellectual dishonesty continues...
"The proper inference from our work is not that one group is more enlightened, or less. It’s that “myside bias”—the tendency to judge a statement according to how conveniently it fits with one’s settled position—is pervasive among all of America’s political groups. The bias is seen in the data, and in my actions."
How can Klein that "myside bias" is "pervasive among ALL of America's political groups" based on one well-documented instance of his own, very conservative political bias? Based on his work, one cannot make this inference.
All we can conclude is that Klein's results were skewed based on poor, biased questions. Now, I'm not saying liberals may not do the exact same thing, but you cannot draw the "Oh, well...everyone does it!" conclusion or inference based on Klein's missteps.
Then again, economics operates out in la-la land anyway, so it's not surprising that such an obvious turd [the research] would make it into the pages of the Wall Street Journal.
#8 Posted by Justin, CJR on Tue 29 Nov 2011 at 06:37 AM