The story says CNBC countered with its own database, of 445 stocks, a fraction of Cramer’s total mentions, which number as much as 7,000 a year, but a list that it believed are fully researched, bona fide picks, not mere mentions. Using that list, the story says, Cramer appears to beat the market by 1.8 percent over two months. But Alpert found flaws in the database.

It turns out that CNBC did its analysis incorrectly, and that the stocks beat the S&P by … 1.2% over two months. CNBC measured the stocks’ performance against the average performance of the S&P year-to-date, instead of against the performance of the S&P from the date of each stock pick. Also, it included more than 100 recently recommended stocks that weren’t held for the full one- or two-month holding period that CNBC claimed.

The ban, if that’s what it is, on Barron’s reporters remains largely in effect, though Steel, the CNBC spokesman, notes that Barron’s reporters have appeared on at least two occasions since the dust-up. (While the dispute was going on, it should be noted, Dow Jones was bought by News Corp., which owns a rival business-news network, casting uncertainty over a content-sharing agreement between Dow Jones and CNBC.)

CNBC still feels aggrieved by the story, although, relieved to have eliminated the leak angle, it did not publicly challenge it. The network and Cramer today insist that he does beat the market and that Barron’s’s data were erroneous and its methods designed to make him lose.

CNBC says (as it said in the Barron’s piece) that the list from was not authoritative and argues that the list on the Web site of is not valid, either. CNBC summed up its position in its August 7 letter to Barron’s:

Mad Money is produced entirely by CNBC. The “Mad Money Performance” section of is maintained by a contributor who has no ties with CNBC or Mad Money, and neither we nor Jim review his work.

Further, Cramer told me, Barron’s is unfair because it doesn’t eat its own cooking. It makes stock selections in its own pages, but starts counting the day before publication, allowing it to benefit from a “Barron’s bounce,” while denying that advantage to Mad Money.

In the end, I disagree with CNBC on almost all counts. Overall, the Barron’s piece is sound.

First, and foremost, CNBC is wrong to air a show that is centered on stock picking without tracking its own performance or even keeping a record, using whatever criteria it chooses, of the stocks it picks.

Alpert says as much in his piece. While CNBC officials maintain that the main goal of the show is educational and to give viewers insight into the thinking of successful trader, they acknowledge the problem and say they are working on it.

CNBC is also obviously wrong to dismiss the list posted by —a list endorsed by the Mad Money host of picks he made on the Mad Money show—as having nothing to do with Mad Money. Meanwhile, is closely tied to Cramer who is closely tied to CNBC. Readers and viewers should not be asked to sort through that linguine bowl of fine distinctions and overlapping interests.

Third, it borders on flimflammery for CNBC to put forward a list of sanctioned picks that eliminates 90 percent of the stocks Cramer talks about, including and especially mentions that are accompanied by cows and a “moo,” which occurs frequently in the lightning round.

And while viewers obviously know that words like “lightning round” and “sudden death” do not mean “meticulously researched,” in fact they do trade on those supposed non-picks, hence the next-day stock gyrations on all manner of Cramer mentions.

Finally, Cramer’s argument that Barron should measure him as it measures itself, while understandable from his point of view, is beside the point. Barron’s’s picks are another story. And since Mad Money picks are available to its viewers only after broadcast, that’s how they should be measured. In any case, Alpert’s story allowed CNBC its method (waiting five trading days after the pick) in measuring its list of 445 stocks.

Dean Starkman Dean Starkman runs The Audit, CJR's business section, and is the author of The Watchdog That Didn't Bark: The Financial Crisis and the Disappearance of Investigative Journalism (Columbia University Press, January 2014).

Follow Dean on Twitter: @deanstarkman.