Innovations

Can Tony Haile save journalism by changing the metric?

March 11, 2015
 

The company that wants to deliver journalism from a decade of digital bloodletting is perched like a physical metaphor in an office six stories above the Strand—a century-old monument to paper.

Housed in the airy, light-filled remnants of a yoga studio, Chartbeat, a Web analytics company, bears the usual startup trappings. There are free snacks, a fridge full of beer, and offices named after superhero lairs: The Hall of Justice, The Bat Cave. Such grandiosity is usually a sign of a company angling for a foothold, but in this case the ego is earned. In the six years since its creation, Chartbeat has become the arbiter of audience in the digital age. Roughly 80 percent of the top 100 publishers, as measured by traffic, use Chartbeat to track their online readership—places like Al Jazeera, The New York Times, Forbes, Gawker and Gannett. Yet the company’s mission has expanded exponentially beyond tracking Web traffic. Instead of simply monitoring journalism, Chartbeat wants to save it.

Chartbeat CEO Tony Haile says he conveys the company’s “singular purpose” to every employee who goes through orientation: “Twenty years from now, the journalist that wants to investigate the corrupt politician actually has the means to do so. As in: There is enough money to invest in that person to do that job.”

Chartbeat at first seems an unlikely candidate to keep quality journalism profitable, since its product helped spur the era of quick hits and clickbait. Launched in 2009, it was the first Web analytics service to show a website’s audience in real time, allowing an editor to see how many people are reading a story, right this minute. A separate display organizes the most popular pages by size in a colorful mosaic that re-arranges itself as readership grows or drops. Using Chartbeat, an editor can see with a glance which stories are popular and which are not, a service that’s proven lucrative for the company. Though launched during a time when most analytics services were free, Chartbeat has cultivated over 50,000 subscribers who pay monthly rates ranging from $9.95 to tens of thousands, depending on traffic.

chartbeat-labs.png

A story mosaic automatically re-arranges itself to show which stories are most popular at that moment.

It has also birthed a new age, in which journalists and editors can react rapidly to their audiences. During a 2012 interview with NBC, Gawker founder Nick Denton showed off the giant wall-mounted monitor that prominently displayed Chartbeat’s ranking of popular stories to his staff. “This is the most terrifying thing we have to show you, for traditional journalists,” he boasted. “It’s as if every single day, every single week is sweeps and the numbers tell the whole story.” This “big board,” now standard décor for many digital publications, translated the value of a piece of journalism into a simple round figure—the number of people who’d clicked on it.

The problem, of course, is that there are all kinds of ways to garner a click that have nothing to do with quality: slideshows, provocative headlines, uber-short pieces on timely topics. Some sites, like Gawker, directly incentivized the chase for clicks by paying writers based on traffic. But the onset of real-time data has increased the myriad pressures facing journalists, regardless of salary scheme. If a reporter’s efficacy hinges on her ability to deliver readers what they want when they want it, then Chartbeat seemed to provide a tantalizing way of assessing demand.

Sign up for CJR's daily email

But catering to clicks hasn’t encouraged editors to worry about the larger, more noble tasks that drove many to journalism in the first place, and that make it a valuable tool for the common good. “Chartbeat’s machines don’t track abstract notions like truth or accuracy,” read a cutting Gothamist post skewering the service for encouraging sensationalism by design. Lou Ferrara, the vice president and managing editor of the Associated Press, told me that he worries deeply about the effects analytics have on his reporters’ instincts. “Imagine if Woodward and Bernstein had access to this kind of data and they said, ‘I’m sorry, this break-in is not really trending.’”

If tools like Chartbeat have quickened journalism’s slow dive, Haile believes that’s due to a misuse of his product, catalyzed by the economics of the Web. There is a single villain: the click. The click is a poor measure of audience, applying the same value to an attentive reader and a drive-by visitor. They’re also easy to fake. At one point, corrupt marketers hired armies of humans to increase impressions with their mouse, but now the process has been outsourced to robots. These “fraudulent bots” are responsible for about 36 percent of online traffic, according to data cited by the Interactive Advertising Bureau. Yet, since the advent of digital advertising, CPMs, a measure of cost per thousand impressions, has been the industry standard for selling digital advertising, which sets the measure of Web traffic. Which means that, like it or not, analytics companies like Chartbeat have to measure impressions.

tony.jpg

Chartbeat CEO Tony Haile says he conveys the company’s “singular purpose” to every employee who goes through orientation: “Twenty years from now, the journalist that wants to investigate the corrupt politician actually has the means to do so.”(Photo courtesy of Chartbeat)

That’s what Chartbeat is out to change. Over the last year, Haile has increased his public stature to become one of the most visible proponents of a metric that he believes can realign advertising dollars to the content most worthy of them: attention. “Time is finite for each of us,” says Haile, which makes it a scarce and therefore valuable unit for advertisers. He and his colleagues at Chartbeat have been pushing the media establishment to adopt attention, or the amount of time spent on a landing page, as the universal measurement of Web traffic.

This switch, Haile believes, will increase the value of the kind of content people are actually reading–the kind of content that has taken a beating in recent years. “We’re in crisis because we chose the wrong metrics,” says Haile. “And that has kind of screwed everything.”

Haile, 37, is tall, soft-spoken, and an avid thinker. He chronicles his yearly book consumption in an oft-updated personal blog. In 2014, Haile read 54 volumes of fiction, history, and philosophy—a low number for him, which he blames on the demands of his growing business. Among the most meaningful reads, Haile lists Doris Kearns Goodwin’s The Bully Pulpit, a mammoth history of what Goodwin calls “the golden age of journalism” during Theodore Roosevelt’s term. On the pages of his blog, Haile’s writing can display a cutting intensity, but in person he is vigorously sincere. Lingering at his standing desk in the center of Chartbeat’s open office, Haile seems more affable engineer than CEO.

Haile emerged in New York’s tech scene by accident, having spent much of his twenties as a polar explorer (which explains his Twitter handle). Bouncing around the most remote places on earth, including the North Pole and the Canadian Arctic, with his expedition partner, Ben Saunders, Haile’s penchant for extreme adventure provided an outlet for his interest in technology. While crewing a round-the-world yacht race, Haile busied himself with communication tools. “Below deck, I was the geek, making sure the satellite could broadcast despite 90ft waves blocking line of sight,” he wrote on his blog. But by 2006, Haile was pushing 30, and had grown weary of the demands of fundraising for these expeditions. Recalling that time, he wrote on his blog: “My parents have started to have very real fears about my future and I can see the strain on their faces.”

This ‘big board,’ now standard décor for many digital publications, translated the value of a piece of journalism into a simple round figure.

A last-ditch effort to raise money for an expedition to the South Pole brought Haile to New York in 2007. New Yorkers would fund anything, he had heard. But when Haile’s six-page critique of a friend’s business plan made it into the hands of a startup founder, a job offer distracted him from his original quest. In 2009, he connected with Betaworks, the New York tech-incubator responsible for success stories like bit.ly and Instapaper, which had a product it didn’t know what to do with. One of Betaworks’ co-founders, Billy Chasen, best known for creating turntable.fm, a site that allows users to stream and share music, had created a mesmerizing display that analyzed Web traffic. “It was almost like an art project at the time,” recalls Haile. He was brought on to find a business that could be built around Chasen’s analytics product, which was called Chartbeat.

Four days after Haile was hired, Chasen went on an extended spiritual quest to India, leaving Haile to chart a course for a product with a market problem. “When we kicked off, it’s like you’ve got Google Analytics out there, which is free. And everyone has it,” Haile said. “And we’re an analytics service.” In order to charge for a service that Google was offering for free, Chartbeat had to offer its clients something different.

Google Analytics provided historical data, which could be sorted and categorized by data personnel in myriad ways to look at long-term trends. Chartbeat focused on “real-time” data—what’s happening on your site right now—presented simply, in a single dashboard.

While adhering to real-time data was a strategy to circumvent Google, Chartbeat entered into it at a precipitous moment. The rise of social media allowed stories to spike and trend in isolation from the mother ship; the homepage no longer dictated traffic. There was a sudden need for editors to understand who was reading what, so they could react immediately. Chartbeat could shift data away from the analysts and put it in front of the people who were generating the traffic in the first place—the journalists. “That was kind of our first real breakthrough,” says Haile.

breakdown.jpg
visitors.jpg

By measuring visitors’ viewing habits, Chartbeat can help organizations maximize their audiences.

But that meant that journalists and editors, whose roles had been comfortably insulated from data, had to reckon with it. Chartbeat dispatched Mona Chaudhuri, the company’s head of product, to assemble a handful of publisher-testers, from Web natives like Gawker to legacy publications like The Washington Post. “How do you make this experience enjoyable and delightful where previously there had just been a bunch of numbers?” asks Chaudhuri. They briefly considered removing numbers from the dashboard altogether, eventually settling on a simple display that set out the useful metrics in an accessible format, rather than an all-you-can-eat buffet. Next came the “big board,” a giant monitor display that would encourage writers and editors to engage with Chartbeat. “Once those went up, you could actually see editors getting out of their seats and looking at it, asking, ‘Why is that spiking?’” says Chaudhuri.

But all that data wasn’t driving reporters to make better decisions about their stories. Instead, it was making journalism disposable. In 2011, Haile was chatting with the Web editor of a mid-sized city newspaper he was courting as a client. When he asked about the interests of the paper’s readership, the editor told Haile that he didn’t care, “as long as I’m hitting my click numbers.” “That just depressed the shit out of me,” Haile recalls.

Those kinds of conversations were becoming increasingly frequent, so Haile decided to do something about it. In April, he began drafting a manifesto detailing a major shift in the company’s direction. He names all of his papers after battles in which the British defeated the French, and he called the document “Crecy,” after a surprise win for King Edward’s army during the Hundred Years’ War. “It was the first battle of medieval times where a small force was able to overtake a bigger force by superior technology,” explains Haile, a strategy mirrored in Chartbeat’s new purpose. Rather than remaining a passive participant in the measuring of metrics, Chartbeat was going to launch a digital revolution. The company would put its force behind making attention the most valuable currency for content. “The name seemed appropriate,” says Haile.

In 2012, Haile tasked Josh Schwartz, an MIT-educated data scientist hired to create Chartbeat’s analytical wing, to run a study to better understand audiences. Not the massive ones temporarily assembled and disbanded by a provocative headline or a viral story, but loyal audiences. After readers clicked onto a website, what factors predicted that they would return? As Schwartz looked at data compiled from Chartbeat’s clients, a simple pattern began to emerge. The more time a reader spent on a page, the more likely he or she was to return to that page in the future.

“It seems obvious, right?” says Schwartz, who has the spindling, anxious demeanor of an academic. “But how much time people spend on articles—this is just a metric that people hadn’t really measured or taken seriously.”

Part of why no one had been paying attention to time is that attention is particularly difficult to quantify on the Web. Registering a click is easy; a server notes when a user visits a page and stores that number. Time measurements, however, are based on fickle users. “The way people have historically measured time is when a person opens a page, you start a timer and when [he closes] a page, you stop the timer,” says Schwartz. But people don’t read everything they open. More often, a reader opens a page and forgets about it in an array of tabs. Measuring time spent on a page requires determining if there’s a human on the other side of the screen. And the problem, Schwartz says, is that “the vast majority of the time, there’s no human.”

To gauge what attention looks like, Chartbeat began a series of lab studies. The company asked roughly 5,000 participants to read an article and broke down second-by-second what this behavior looked like. Readers, Chartbeat discovered, were easy to spot: They were almost constantly interacting with their computers—scrolling or flicking a mouse at least once every five seconds. (Readers on mobiles and tablets interacted even more often.)

Since then, Chartbeat has produced an impressive array of data to highlight problems with our current methods of ascribing value to online content. While Facebook and Twitter shares can drive traffic to a site, most people don’t read the articles before (or after) they’re shared, making these shares useless to advertisers. In the summer of 2013, Chartbeat partnered with Slate’s then-technology reporter, Farhad Manjoo, to look at the Web’s retention rates. The result, “You Won’t Finish This Article,” summarizes a depressing fact: rarely is anyone reading anything at all. “The more I type, the more of you tune out,” wrote Manjoo. “And it’s not just me. It’s not just Slate. It’s everywhere online.” Last March, in a bold Time article titled, “What You Think You Know About the Web is Wrong,” Haile laid out his argument for a time-based metric. Valuing time spent on site would transform the internet into a place “where quality makes money and great design is rewarded.”

This September, Chartbeat finished a six-month audit by the Media Ratings Council, the company that verifies measurements of audience for advertisers. This certification allows a publisher to sell advertising based on Chartbeat’s measurements of time-on-site, making Chartbeat the first company licensed to measure attention. A few weeks after certification, Chartbeat released its methodology for tracking on its site, encouraging other analytics companies to follow suit. By the time Haile addressed the Online News Association this fall, the subject of his speech, which focused on building what Haile has dubbed the “Attention Web,” was familiar to the journalists in attendance.

The only metric that matters, what we in this room are obsessed with, is attention.

But the real question is whether shifting the metric will shift advertising value towards better content. Haile believes that the stories people read are of higher quality than those they only click on, an assumption he can back up with some data. Chartbeat researched keywords in 2013 headlines that received the least amount of attention per click, including “top,” “best,” “biggest,” and “richest.” Words receiving the most attention included “Syria,” “Egypt,” “Obamacare,” “Trayvon,” “Snowden,” and “D.C. 2013.” If those more deeply read articles weren’t better quality, they at least focused on more serious topics.

Last week, for example, digital publications churned out story after story on “The Dress,” a photograph of a cocktail dress that appeared to be either white and gold or blue and black, depending on the viewer. Judging by the number of clicks it received, The Dress seemed to dominate our attention. Across the Web, stories about it received more than five times as many clicks as stories covering the ruling in the federal case on net neutrality. But when measured by attention, “things get a little more interesting/hopeful for the human race,” a Chartbeat staffer told the Daily Dot. Dress stories garnered only 2.5 times the amount of reader time as the net neutrality story. By attention, The Dress had still proven itself a more valuable news item than the net neutrality ruling—but by a narrower margin.

Advertisers seem poised to adopt the attention metric, a move that would signal an industry shift for journalists. Digital Content Next, the only trade association for digital content companies*, named its conference this January, “The Attention Economy.” During Social Media Week this February, Meredith Kopit Levien, The New York Times’ executive vice president of advertising, told a packed auditorium that impressions and clicks are “faulty metrics.” “The only metric that matters, what we in this room are obsessed with, is attention,” she said.

probability.jpg

Slowly, led by the first group of digital natives like Upworthy and Medium, which measure traffic by time-on-site, and the Financial Times, which began selling ads based on attention last year, traditional journalists have been adjusting to what a change in Web metrics might mean. “Chartbeat has helped us be much more aware of dwell time,” Giles Wilson, features editor for the BBC, told me. While this awareness hasn’t shifted his editorial standards, it has helped him be a better editor. Wilson uses a Chartbeat tool that pinpoints the parts of a story where readers stop reading. It’s essentially a crowdsourced editor, allowing Wilson to untangle a sentence or clarify a point that’s driving readers off his site.

Even Nick Denton, a self-proclaimed champion of the pageview, announced in January that Gawker, which had already abandoned its practice of paying writers based on traffic, would recalibrate its editorial mission further to encourage quality. “That’s the theory: do good stories and the traffic will come anyway,” Denton wrote in an announcement on the site. “Unique growth is a by-product of good journalism, but a dangerous target to follow blindly.” To demonstrate the commitment, Denton replaced the “Big Board,” which had increasingly become a home “for stories we weren’t proud of,” with an editor-curated display showcasing the staff’s best work.

But every change comes with risks. Haile, for his part, believes that measuring attention won’t just be a more accurate measure for advertisers, but a more nuanced one. It will naturally assign value to a range of content, in line with our multifaceted interests. “I could be [spending time] on BuzzFeed, where I’m seeing these amazing cat things, as well as The New York Times,” says Haile. “But what I’d like to say is, they better be damned good cats.”

“People [are] jumping around and trying to find the God metric for content,” BuzzFeed-founder Jonah Peretti said in an interview last June with Medium. There is no God metric, Peretti concluded. Yet, attention is a particularly heady measure because it places power back in readers’ hands. If, as Haile believes, what garners our attention is worthy of it, then his Web is a better, more thoughtful place. But if our attention is as easily captured as our clicks were, we’ll be back in the same situation—with one difference. We won’t have editors, advertisers, or Gawker to blame. We’ll only have ourselves.

*This originally stated Digital Content Next was a trade association for digital advertising. It is in fact for digital content companies.

Alexis Sobel Fitts is a senior writer at CJR. Follow her on Twitter at @fittsofalexis.