“The one misunderstanding I’ve seen on a lot of the social media posting about this is that I had a small sample and that it’s not random and that it’s uncontrolled,” Bohannon told me, when I reached him by phone on his way home from Canada, where he was administering the “Dance Your PhD” interpretive dance competition. “If my goal was to do a comprehensive story about the open-access business model of academic publishing, then this was a small sample.” The difference, Bohannon says, is he wasn’t trying to compare open-access to traditional journal models; he was trying to evaluate fee-based publications. “I didn’t do a small sample of those; I did all of them,” he said. “In all the hot air on this, I did all of them. Every single one.”
But some of the language of the piece (the words “open access” are used frequently, almost interchangeably with “fee based”) and a subsequent press release, reprinted on Eisen’s site, give a different impression. According to the press release, the article “exposes the dark side of open-access publishing”…revealing “a “Wild West” landscape that’s emerging in academic publishing, where journals and their editorial staffs aren’t necessarily who or what they claim to be.” Though Bohannon says he regrets not clarifying his fee-based intentions, he still thinks the sting reflects accurately on the open access model. “I have shown a very high frequency and incidence of fraud within this new and fastest growing section of academic publishing,” he says.
Much of the language used to criticize Bohannon’s piece is scientific jargon; this makes sense, because most of his critics are scientists. But critics of his work have held it to a higher standard of research than most works of investigative journalism—did Watergate require a control group? Perhaps because his original piece is so scientifically rigorous (and open access so hotly contested), he’s been held to a higher standard of proof. “I deliberately mixed styles between traditional narrative investigative journalism and science communication, because my readership is readers of science,” he said. “I wanted to spell out rigorously how I generated the paper, what the flaws were. I wanted to be more clear about the methodology of things than one typically is in journalism.”
Regardless of the criticism, more surprising is the fact that the piece has largely failed to have the rippling effects that Bohannon envisioned when he conceived of the project. Though one journal that accepted the paper was ultimately shut, public criticism hasn’t touched the others that accepted the piece in a significant way. Even the findings that reflect positively on open-access have been largely unacknowledged: Along with the journals that Bohannon debunked, the often criticized journal of the Public Library of Science, PLOS ONE, came off looking vindicated when they rejected the article in just two weeks.
If scientists can write off the results of such a sting with hedging about caveats, then Bohannon worries about the future of rigor in open-access publishing. “Shouldn’t you feel as surprised and worried as I felt when I saw these results?” Bohannon said in an interview this week. “I can only assume that people are either far more cynical than I am about academic publishing, or they just haven’t thought through this all clearly yet.”