AOL and Its Algorithm

The company is hiring hundreds of journalists. What will they produce?
November 23, 2010

“Are you a passionate and entrepreneurial online journalist? Want to be part of a dynamic and innovative team of journalists, engineers, designers, and business pros who are creating a bold new solution for our industry? Do you think that traditional news media just don’t get it anymore?…”

AOL may have a job for you. Really. In fact, in June AOL pledged to hire as many as 500 journalists over the next year as part of a push to propel the struggling company from its past as an Internet service provider into a future as a content and advertising company. This is on top of what the company says is 500 full-time journalists already on staff. Lately, the company has been posting hundreds of openings, primarily but not entirely in its fast-expanding Patch.com community news network. Full-time jobs. With benefits and what many might call respectable salaries, for doing what it calls quality work.

But, what kinds of jobs? What kind of journalism? What kind of quality? How does this “bold new solution,” which forges a closer union between journalism and technology, work?

It’s too early to know the answers to those questions. Still, the developing model at AOL—and, on a smaller scale so far, at its rival Yahoo—which recently hired a half-dozen veteran journalists for its new breaking news site called “The Upshot”—represents an aggressive hybrid of editorial skill and computer algorithms, a combination with the muscle to influence the future of journalism.

Some journalists blanch at the mathematical ring of “algorithm.” They see it as a road to computer-assisted pandering. Proponents argue that an algorithm is simply a software program that can make journalists more successful by telling them what people are interested in. Success, they say, translates to the kind of stories that entice users to click and companies to advertise.

The algorithm developed by AOL, called Demand/ROI, does two main jobs. It scours databases and social networks to discern user interests—through search and other behavioral data. And it monitors how readers are responding to a story or aspect of a website in real-time. In essence, it’s a kind of “most e-mailed list” on steroids—not just reporting on a story’s success, but predicting the degree of success and even how much revenue it might thus generate. This information, obviously, is potentially useful to those assigning and producing content and to those advertising alongside it.

Sign up for CJR's daily email

Currently in Beta mode, Demand/ROI is poised to roll out soon for use by AOL News and the rest of AOL’s fifty-plus niche sites, or brands, according to David Mason, AOL’s senior vice president, AOL Content Platform. Demand/ROI is so adept at assessing content “opportunities,” Mason says, that eventually the algorithm may be used to post some basic assignments—automatically—in areas such as Seed.com, AOL’s new professional-amateur site, which offers low-pay assignments for freelance writers, photographers, and, soon, videographers.

Tim Armstrong, the company’s chairman and chief executive, stresses that the algorithm is a tool to make journalists smarter and their output more relevant, not to be the sole dictator of content. “Technology is not a weapon against journalism, it’s a weapon for journalism,” he says. A tall, lanky man of thirty-nine, Armstrong took the helm at AOL’s sprawling corporate headquarters in lower Manhattan in March 2009. He came from Google, where he oversaw advertising sales, marketing, and operations as president of the company’s Americas Operations.

Armstrong envisions AOL producing good journalism guided by humans and enhanced by machines, a form at once familiar and alien, especially to members of—and refugees from—legacy media. Yet AOL is luring veterans of those newsrooms, along with graduates of top journalism schools. It hired close to 900 people over the summer, though editorial hiring for the company’s chief journalistic brands beyond Patch—such as AOL News, DailyFinance, PoliticsDaily, WalletPop, Engadget, and FanHouse—has slowed somewhat, according to people familiar with AOL. Before that, the company already had signed on dozens of experienced journalists, many with distinguished backgrounds at major news organizations, from USA Today and the San Francisco Examiner to The Associated Press and The New York Times. AOL’s corporate site mentions nine employees who have been involved in writing or editing Pulitzer Prize-winning stories.

AOL declines to disclose salary information, but people familiar with the company estimate that some top editors on AOL’s leading journalistic sites earn six figures, while some staff writers make $70,000 or more, depending on their experience or, in the case of columnists, their following. In addition, AOL claims that more than 40,000 “content creators” work across its properties, some on contract and others on a per-assignment basis. Most of these are paid on a much lower level, often under $100 for an article.

The notion that a company like AOL would claim to be investing in “quality” journalism has created a modicum of hope. At the same time, pessimists worry that the acquisition of such news veterans provides a veneer of credibility for a venture that might wind up essentially akin to “content farms” like Associated Content (recently purchased by Yahoo), Examiner.com, and Demand Media, where armies of low-paid freelancers churn out material in vast quantity and of varying quality on topics driven primarily by algorithms, producing only content that is predicted to attract the most users, and thus, advertisers.

That’s not going to happen at AOL, according to Armstrong. “We keep a Chinese wall between our ads and our content,” he says, but adds: “Even though there’s a Chinese wall, both sides could be looking at that Demand algorithm,” and what each does with that information could be entirely different. Armstrong, who was a co-founder and former chairman of Associated Content, argues that journalistic concerns about AOL’s algorithm are understandable but unfounded. The algorithms, he says, are “just helpful data.”

He has allies in this view, including Jay Rosen, the press critic and professor of journalism at New York University. Rosen says he understands journalists’ fears about them but he argues that if they are not abused, the use of algorithms—“learning what people are clicking on, searching for, and interested in now, today, and tomorrow” can be a good thing.

“Who wouldn’t want to know? It’s important data and if you treat it as anything but important data, you are making a mistake…. Journalists missed the boat on data a long time ago and that’s one of the reasons why they’re in the hole they’re in,” he says. “Think about it: in what other industry in America could you sit there as the most valuable employees in the business and be ignorant about the data?”

Like Dubai, which is rushing to transform itself into a glittering playground of the Middle East before the oil under its sands runs out, AOL is scrambling to develop a new business model before its dial-up screech falls silent and its home page no longer reliably funnels dial-up subscribers to its various sites. Divorced from a rocky, ten-year marriage with Time Warner at the end of 2009, the company is in a race to reinvent itself.

The dial-up business remains lucrative, but is retreating before the march of broadband. Many rural areas of the U.S. don’t have broadband service. But in its most recent annual 10-K filing with the Securities and Exchange Commission, AOL acknowledged that this number is rapidly declining: dial-up households dropped from 44 million in 2004 to 10 million in 2009, while broadband penetration in U.S. households rose from 28 percent to 69 percent. The number of AOL’s dial-up customers, unsurprisingly, is down to about 5 million, from a peak of 26.7 million in 2002. According to its 10-K filing, the company derived $1.4 billion in revenue from dial-up in 2009, down from $1.9 billion in 2008. Advertising revenues also declined, as they have through the first half of 2010.

Armstrong is betting on user-data-guided content to extricate the pioneering twenty-five-year-old tech firm from its “You’ve Got Mail” dial-up roots and reposition it as a “You’ve Got My Attention” media company. To do that, he believes that content must reach a level of quantity—and quality—that will appeal not just to users, but to national advertisers. As part of that strategy, AOL is also making acquisitions. In late September it bought the popular TechCrunch blog, along with an online video-instruction company called 5minMedia.

AOL’s content flows out through multiple streams, a wide range of niche sites, or brands. Many of these sites—such as Games.com, Love.com, and the country music-oriented The Boot—offer little or no journalism content. Many others feature service journalism, such as KitchenDaily and StyleList, “real style for women who love fashion, beauty, and celebrity.” Experienced journalists are sprinkled throughout these service sites, primarily as editors.

The AOL sites most deeply rooted in journalism, and where many of the journalists hired so far are working, are in the news and information group: broad-based AOL News, which tries to provide national and foreign news and analysis edited by former New York Times web journalist Michael Nizza; PoliticsDaily, run by former Times journalist Melinda Henneberger; DailyFinance, run by former BusinessWeek reporter Amey Stone; Engadget, a popular tech site; WalletPop, on personal finance; and sports-oriented FanHouse, which is crammed with former newspaper sportswriter stars.

Near the other end of the AOL editorial spectrum is Seed.com, run by Saul Hansell, a veteran New York Times technology writer and the programming director of AOL Content Platform. On Seed, professional and amateur freelancers choose from a variety of primarily service-oriented assignments generated by editors with assistance from AOL’s algorithm. Contributors bid for assignments along the lines of “Best Public Restrooms in Park City, UT”—competing for assignments that pay as little as $50 for 1,000 words. If a submission is selected for publication, editors then shape the winner’s material. The plan is for Seed to generate large amounts of original content—including articles, photos, and videos—for use across AOL brands.

This is the closest AOL currently tilts toward the so-called “content farm” model. Content platform chief David Mason contends that, over time, a stable of trusted contributors will emerge from the fray and some will receive direct assignments. He said payment for projects on Seed will eventually run the gamut from $10 to more than $1,000, depending on such variables as the required level of expertise and the complexity of the assignment.

AOL’s Manhattan headquarters occupies three sprawling floors in the old Wanamaker department store building on lower Broadway. Row upon row of gray cubicles are punctuated by large flat-screen color monitors on the walls. The floors are so cavernous and similar in appearance that color-coded location charts are provided on counters near the elevator banks. Hushed but busy engineers, programmers, designers, and editors work side by side. Brainstorming takes place in quirky seating areas defined by orange and white shower curtains, and most people have at least two computer screens on their desks.

One of them is Cheryl Brown, editorial director of AOL’s KitchenDaily, a new recipe-oriented food site for home cooks, and its older Slashfood news blog. Brown spent ten years as an editor at Condé Nast’s recently shuttered Gourmet magazine, until 2005, and then served as managing editor of Disney’s now-defunct parenting magazine Wondertime before joining AOL in October 2009, just four months before the launch of KitchenDaily.

After the demise of Wondertime, “I decided this was the time to hitch up the wagon and learn some new skills if I were going to stay in this business,” says Brown. She manages two full-time and two part-time editorial staffers, along with a long list of regular contributors, many of them ex-Gourmet writers, and nearly a dozen “partnerships” that provide columns and recipes from people like author and New York Times columnist Mark Bittman and outlets like The Culinary Institute of America. “The pace is different,” she says. “It’s almost like putting out a whole magazine every day, with less staff and fewer resources. So, it’s almost blinding whiplash for a little while. But then you kind of get into the groove and figure out how to make it work.” A few graying heads can be spotted around the office, but just a few. “It’s really young,” notes Brown, who is forty-one.

At this point, she said, some 90 percent of the content on KitchenDaily (not counting recipes) is original, plus some licensed reprints. KitchenDaily also ventures beyond the stove. It sent an editor to the White House in June for an exclusive story on harvesting honey from the presidential beehives, including photos, video, an article, and links to honey recipes. The goal is increased engagement with users, measured in clicks, comments, and participation, as in inviting cooks to share food photos through Flickr.

More news-oriented than KitchenDaily, the Slashfood blog publishes such stories as “Wine Vending Machines Debut in Pa.,” “Ready Pac Baby Spinach Recall,” and features such as “Celebrating Cow Appreciation Day,” along with modules on chefs, restaurants, reviews, and items on new beers and other products. KitchenDaily has about fifty regular contributors and SlashFood has eighteen, but Brown can access more original content from different regions of the country for both sites through local Patch editors and assignments posted on Seed.

Brown admits she’s become an algorithm addict. She puts up what she calls her “heat map” on a screen, a program that instantly tells her exactly what’s hot and what’s not on her two sites by tracking where people are clicking.

And she can do something about what she sees. If a feature on asparagus is not pulling in the expected eyeballs, its headline and deck copy can be changed to increase appeal. If it’s determined that a cooking video is more popular at a certain hour, it can be shifted. While it doesn’t dictate what they do on the sites, Brown said, the algorithm “helps us focus the content.”

AOL’s biggest current push into journalism is in Patch, a spreading network of hyper-local community news sites, in which it is investing $50 million this year. AOL has been posting hundreds of openings for Patch editors across the country, along the lines of: “Journalists wanted. Small town news. Big time job.”

Armstrong was one of the initial investors in Patch, which AOL purchased in June 2009 (Armstrong says he recused himself from the negotiations and took back only his initial investment). Focusing on neighborhoods of 15,000 to 50,000 people, Patch news operations emphasize original reporting, whether about the local high school graduation or the city council fight over taxes. There were 100 Patches in nine states by mid-August; at least 400 additional sites in more than a dozen more states are projected by the end of 2010, particularly in areas where local newspapers have pulled up stakes.

Most Patch launches tend to in be middle-class to affluent bedroom communities, where demographics are attractive to local and national advertisers. Perhaps cognizant of that class divide, AOL started the Patch.org Foundation in March 2010 to partner with local organizations in inner-city neighborhoods to fund Patch sites in underserved areas. In September, AOL announced the launch of PatchU, a program in which journalism students at a number of colleges and universities can intern at local Patch sites to get course credit and practical experience. The program, which partners with thirteen major journalism schools, offers hands-on training for students and provides a degree of journalistic credibility to Patch—and a source of free content.

Still, the landscape is littered with failed or gasping hyper-local sites, from independent start-ups like the defunct Backfence and the struggling NewJerseyNewsroom to legacy experiments such as The Washington Post’s defunct LoudounExtra and The New York Times’s The Local, which recently shut its three New Jersey sites and pointed readers to Baristanet. Even AOL, in the late 1990s, tested the hyper-local waters with Digital City, a partnership with Tribune Co., and found them too chilly. It was a different model—using reporters employed by Tribune newspapers—and a different time—fewer people had computers, there were no smart phones, and everybody used dial-up. Lack of scale was a problem.

And so was lack of engagement. People had been trained to expect local news to arrive only on a certain day via weekly newspapers, recalls Owen Youngman, then in charge of Tribune’s Digital City effort and now the Knight Professor of Digital Media Strategy at Northwestern University’s Medill School of Journalism. Youngman said success will still be a challenge, but if a local online news service can demonstrate comprehensive and continuing coverage of a big story, akin to news radio, he said, it’s got an opportunity to grow.

According to people familiar with AOL, local full-time Patch editors, who range from fresh journalism school graduates to twenty-year-plus veterans, make about $35,000 to $50,000. They are the 24/7-foot soldiers and they work hard at cultivating their Patches. Every one gets a Blackberry, laptop, digital still/video camera, and a police scanner to keep them up at night. None of them has an office. They are encouraged to work out of local coffee houses or other public venues where they are supposed to be in touch with their neighbors—and the local news.

Satta Sarmah is the editor of the Patch in suburban Rye, New York (rye.patch.com). Sarmah, twenty-five, interned at CBS News London and the Columbia Journalism Review, and then worked at the Orlando Sentinel, CNN, and Everyblock.com before joining Patch in November 2009. She got a scoop in June when coyotes attacked and slightly injured two little girls in separate incidents. Rye Patch broke the story of the second attack.

Typically, Sarmah said, she posts her first story between 6 and 8 a.m. and on some days may finish work after a city council meeting ends at midnight. She writes as well as assigns and edits her dozen-odd freelance contributors and manages a weekly budget. She is supervised by the Hudson Valley regional editors, Kathleen Ryan O’Connor and William Demarest. Patch supervisors are said to earn around $80,000, depending on experience.

On September 11, the Rye Patch news site led with a comprehensive rundown on the fall activities of local youth football, field hockey, soccer, volleyball, tennis, and cross-country sports teams. Also featured were stories about: local regulations governing the installation of residential walls and fences, and the intricacies of the “bagel tax” for local coffee shops. Then there were the usual events and announcements.

Patch writing tends to be competent if no-frills. As Ken Doctor pointed out on his blog Newsonomics, a Patch story may give the facts, but often with only a single source and sometimes without much context. He cited a story this summer about a police shootout that resulted in the death of an armed gunman at a 7-11 in San Ramon, California, in which Patch offered nothing but the bare facts. A story about the incident in the local Contra Costa Times newspaper, meanwhile, with two bylines, gave much more depth and background. Yet due to adept use of search-engine optimization, the Patch story topped the newspaper story on both Google web and news searches. And, Doctor added, there were nine comments on the Patch story and none on its Contra Costa counterpart.

It is on the bigger stage—breaking national and global news, at its AOL News site—where AOL faces an immediate challenge. Journalists from “legacy” organizations including The New York Times, the Chicago Tribune, USA Today, and others are staff members and contributors there. The question is, how to consistently produce quality, original national and foreign news with a full-time staff of about twenty people, heavily skewed toward editors, plus about two-dozen part-time staffers and dozens of freelancers?

By mid-summer, somewhere between one-third and one-half of all the content on the site was original, according to former AOL editor Michael Nizza (Nizza departed for News Corp. in October). The balance was a mix of wire stories, content from partners, such as Space.com, and an often-awkward hybrid of rewritten outside stories with staff reporting added. Stories from AOL News full-time staffers consistently exhibit original in-depth reporting and analysis. But work from the dozens of freelance “contributors” is uneven.

For example, on July 6 a U.S.-based contributor, trying to report on Raoul Moat, a gunman loose in northern England, inadvertently quoted from The News Grind, a satirical British news site. The unwitting—but doubtlessly under pressure—contributor included this quote in his AOL News story: “I can scarcely wait for the climax,” confirmed Elsie White, 77, as she raced back to her house after picking up some toffees and copies of today’s paper from a local news agent featuring the blood-soaked face of a police officer allegedly shot by Moat. “We haven’t had a live event like this to enjoy for quite some time and there’s only old ‘Doctors’ episodes on at this time of day.”

The News Grind and The New York Observer gleefully noted the misstep and a correction swiftly followed. But the incident highlighted the potential danger when reporting is rapidly cobbled together from outside sources. Nizza said AOL News has since formalized a ban on posts based on single sources beyond proven news operations.

Many of these hybrid pieces, sometimes attributing to as many as five different news organizations, are clunky. They’re also vulnerable to errors, as journalists scramble to rearrange quotes and paragraphs during rewriting. This happened, for example, with a June piece on Starbucks offering free Wi-Fi, in which a quote taken from a New York Times story was attributed to the wrong person.

It’s not what Armstrong wants AOL News to be, he said. “I have a hard time seeing an economic long-term value in journalists scraping other journalists and adding 5 percent more to the story. I am not a fan of that. I think it’s not an economic viability and I don’t think it delivers great consumer value.”

But he also acknowledged the challenge AOL News faces. “Patch is very clear journalism. Something like PoliticsDaily is really clear journalism. Engadget and FanHouse, really clear original journalism, and strong,” he says. “Real time breaking news? How do you do that? You can be either the originator or the partner, but probably being in-between is not a good place to be.”

AOL is still working on that conundrum, he said, as well as how best to use the algorithm and its indicators of reader interest and response at AOL News.

This new union of journalism and algorithm is a tricky area that is still evolving. According to Ken Doctor, “No one’s done it right yet. The blend of real journalistic know-how, talent, and experience and the technologies of the day to aid that, and to distribute the work itself—it’s the blend of the two that nobody’s gotten right yet.” Still, he applauded AOL’s effort in trying.

No one, including Tim Armstrong, knows if AOL’s grand plan will work. “Our overarching business question is: Is journalism undervalued?” He continues: “As Warren Buffett says, be greedy when people are fearful, and fearful when people are greedy. We’re being greedy when people are fearful about journalism.”

Of course, he concedes, there may be good reasons to be fearful. “But I have to believe that journalism in the future will be just as important as journalism in the past.” 

Lisa Anderson is a 2009-2010 Encore Fellow at CJR. She was the the New York bureau chief and a national correspondent for the Chicago Tribune until December 2008.