Does this mean that finally, after fifteen years of mounting chaos in online metrics, a single standard will take hold? That something like the relative clarity of TV ratings will be achieved? Don’t bet on it. No trade group or task force can address the fundamental problem—if it is a problem—of counting online audiences: too much information.
The “banner ad” was standardized by the site HotWired in late 1994. The next step was obvious: HotWired began to report what share of people clicked on each banner, i.e. the “click-through rate,” giving advertisers a new way to think about the impact of their campaigns.
That origin story goes a long way toward explaining the informational mayhem that afflicts online media today. Every visit to, say, Salon or Nytimes.com yields a blizzard of things to measure and count—not just “click-throughs” but “usage intensity,” “engagement time,” “interaction rates,” and of course “page views” and “unique visitors,” to name a few. How deep into the site do visitors go? How long to do they stay? Match any numerator to any denominator to make a new metric.
The statistics accumulate not only at the sites you visit, but also in the servers of every advertiser or “content partner” whose material loads on the same Web page. Any of these servers can attach a “cookie” to your browser to recognize when you visit other sites in the same editorial or advertising networks. Data at each tier can be collected and analyzed (thus, measurement firms like Quantcast and Hitwise pull traffic figures from ISPs to come up with their own audience figures).
The Web has been hailed as the most measurable medium ever, and it lives up to the hype. The mistake was to assume that everyone measuring everything would produce clarity. On the contrary, clear media standards emerge where there’s a shortage of real data about audiences.
Nothing illustrates this better than Nielsen’s TV ratings system, which has enjoyed a sixty-year reign despite persistent doubts about its methodology. The company has responded to some critics over the years, for instance by increasing the number of Nielsen households and relying less on error-prone viewer “diaries.” It can’t do much about the most serious charge, that the panel is not a truly random sample and thus fails a basic statistical requirement.
But Nielsen’s numbers are better than nothing at all, and that’s what radio or TV broadcasting offers: no way to detect whether 5,000 people tuned in, or 5 million. With nothing to go on, accuracy matters less than consensus—having an agreed-upon count, however flawed, as long as it skews all networks equally.
Print publications have more hard data—a newspaper knows how many copies it distributes, though not how many people actually read them. So publishers rely on third-party auditors like the Audit Bureau of Circulations to certify the squishy “pass-along” multiples that magically transform a circulation of 192,000 at The Miami Herald, for instance, into a total “readership” of 534,000.
By comparison, computer networks are a paradise of audience surveillance. Why expect media outlets, agencies, and advertisers to abide by the gospel of one ratings firm, to only talk about one number, with so much lovely data pouring in from so many sources? “People use whatever numbers look good that month. It gives publishers some flexibility,” says Kate Downey, director of “audience analytics” at The Wall Street Journal, which subscribes to Nielsen, comScore, Omniture, and HitWise. “I think if everybody had the same numbers, we would hate that even more.”
There’s another reason for the lack of consensus about audiences on the Web: the numbers don’t matter as much to advertisers. As any Mad Men fan knows, Nielsen’s TV ratings are a kind of currency on Madison Avenue. An extra point or two of penetration translates into millions of dollars over a season. That’s why plot lines peak and the news gets trashier during “Sweeps Week,” when local ad rates are set.

This article makes me think that some open cookie standard would help. For one thing, the cookie would be regulated, which would be good for users. But it could be a source of clout for the social networks as well. A news viewer may access the same site from several computers, but one thing those computers have in common is that cookie they get from Twitter or Facebook or google.
Cookies.
#1 Posted by Http://mostmodernist.com, CJR on Wed 8 Sep 2010 at 11:58 AM
This article makes me think that some open cookie standard would help.
For one thing, the cookie would be regulated, which would be good for
users. But it could be a source of clout for the social networks as
well. A news viewer may access the same site from several computers,
but one thing those computers have in common is that cookie they get
from Twitter or Facebook or google.
Cookies.
http:twitter.com/mostmodernist
#2 Posted by mostmodernist, CJR on Wed 8 Sep 2010 at 12:05 PM
Ratings and measurement systems generally give me an upset stomach.
As the article implies, Nielsen's monopoly over television ratings has raised questions for years about how numbers are generated, tracked, manipulated, and published. I've spent more than a few nights looking at TV ratings, week-by-week, year-over-year or what-have-you, and my view is such that complacency with a crooked system is just as bad as being in promotion of said system. Set top box data? Some people have legitimate concerns about it, but it's a shame others refuse it just because the new technology would force market researchers to be a little better at their job.
#3 Posted by Aaron B., CJR on Wed 8 Sep 2010 at 12:26 PM
"But Nielsen’s numbers are better than nothing at all, and that’s what radio or TV broadcasting offers: no way to detect whether 5,000 people tuned in, or 5 million."
In the age of digital TV this makes little sense to me. If the cable and sat TV companies got together, they could tally an actual count of viewers. They could also indicate how many people actually watch commercials... (not many)... which is probably why they don't report this stuff. I imagine they are doing this kind of research anyway - for their own internal optimization purposes. But the idea that Nielsen is the only option out there seems wrong to me. There are plenty of ways to "detect," either by tallying actual numbers or using a statistically significant sample size. It just seems that there's no interest in doing it.... at least not for public consumption.
#4 Posted by ms, CJR on Wed 8 Sep 2010 at 02:25 PM
Another factor that's totally disregarded by the measurement services: the percentage of hits generated by non-humans (i.e. bots, crawlers, etc. etc.) According to TownNews.com, which hosts more than 1,000 newspaper websites, almost 70 percent of the traffic it tracked in January 2010 was generated by spiders, bots and other web crawling creatures. Not a pair of eyes among em.....
#5 Posted by Chucolo, CJR on Wed 8 Sep 2010 at 03:38 PM
Yes, some may count non-human (robot & spider) traffic. But many publishers using server- or client-side counting use (1) the IAB Robot/Spider list and (2) Dynamic Filtering to detect non-human traffic so they get a clean look at their real traffic. A CEO running a Digital Company has material motivation to understand this difference.
The Council for Research Excellence recently published a paper on Set Top Data and you should be able to find that at http://researchexcellence.com/committees/settopbox_committee.php
Nielsen, Arbitron and others are audited by the Media Ratings Council to provide the transparency and accountability spoken of. You are absolutely correct; the state of Digital Measurement is not what we need it to be today. And of more concern should be the challenge of keeping pace in a world of proliferating mobile apps.
#6 Posted by Dan Murphy, CJR on Tue 14 Sep 2010 at 07:52 AM
If editors are looking for useful data on which to base editorial decisions, I'd recommend our Newstogram platform. It goes beyond telling you what stories are popular to showing you the topics and entities that are trending across multiple stories.
#7 Posted by Neil Budde, CJR on Mon 20 Sep 2010 at 01:03 PM
I havden't read all the commenta, so this may have already been pointed out, but third party companies like Scarborough and Gallup determine the readership number, and ABC certifies the number of copies sold.
#8 Posted by peter Sullivan , CJR on Tue 5 Oct 2010 at 03:33 PM
Interesting report. I imagine it is useful for journalists and other media professionals working in large news-based media companies.
The point made towards the end of the report on the difference between big media and small media with regards to the relative obscurity of small media in most of the third party produced metrics needs to be properly explained.
Another tricky problem is reconciling the focus on journalism with the reality of the non-journalistic or quasi-journalistic media of the internet. That is, advertisers are interested in what consumers are interested in and consumers are not always interested in news.
Lastly, a minor point, for the niche media who service smaller audiences and who may integrate other features ('brand extensions') into their sites beyond content-based 'news' (forums, video, classifieds, etc.).
#9 Posted by Glen Fuller, CJR on Wed 26 Jan 2011 at 08:00 PM