Finding new ways to follow the story

December 1, 2016

What are the purposes of innovation in journalism? Since the birth of radio almost a century ago, one has been to deliver news to audiences as they stampede from one new communications medium to another. Radio, television, cable television, the World Wide Web, and smartphones successively disrupted the formats in which journalism producers reached the public. And since it is usually necessary to reach large audiences to pay for the costs of professional journalism, each of these disruptive waves has brought anxiety about—and innovation to solve—the problem of making money.

In today’s profession, there are many inspiring stories of entrepreneurial adaptation led by journalists who have broken away from legacy newsrooms (Politico, Recode, Serial) as well as impressive stories of startups that have reached scale, even if their sustainability is as yet undemonstrated (Vice, Vox). But in too many newsrooms, editors, producers, and reporters feel whipsawed by business leaders who are desperate to connect audience with revenue, but aren’t sure how journalism can help. In newsrooms pressured by speed and focused on counting clicks, as an ethnographic study by Caitlin Petre, for the Tow Center on Digital Journalism, showed, the dominance of metrics can leave reporters in a keyed-up state of anxiety. The power of Chartbeat, a real-time tool that measures traffic and audience engagement, to help editors chase trending stories, “is the feeling the number produces,” as a Chartbeat employee said. At best, it’s the feeling after a venti double latte; at worst, it is the blank stare of a hamster on a wheel.


These sorts of investigations are essential if professional journalism is to reaffirm and defend its First Amendment role in an era when code is power.


Newsrooms require new ways to think about—and embrace—technological innovation. One place to begin is with the public mission of journalism. After so many years of dislocation and confusion, it is past time for journalists to separate professional innovation from business and audience innovation. This is not an either/or choice, but it is imperative if the profession is to revive its own credibility and appeal.

Of course, journalists have an interest in participating and even leading as their owners and nonprofit fundraisers attempt to deploy emerging technologies to grow audiences and revenues, while maintaining the news file’s integrity and independence. Yet distinctively, journalists also have an interest in exciting and persuading the public about the value of deep reporting by modeling how it can be done in dazzling and impactful new ways. And as a practical matter, if reporters are to carry out their aspiration to hold powerful institutions accountable to the public in democratic societies, their techniques must respond to the new ways governments and corporations use information to exercise power in this era of algorithmic decision-making and Big Data.

Sign up for CJR's daily email

One of the most promising areas for innovation in investigative reporting is the interrogation and reverse-engineering of algorithms, those if-this-then-that mathematical formulas that determine what comes across your Facebook feed; what price you may pay for services or insurance, depending on your zip code; and where your local government may plan to direct the most police, based on a computer-generated forecast of crime known as “predictive policing.”

Algorithms often involve the exercise of power and they may be discriminatory, by design or unintended consequence. They are rarely transparent. The formulas created by corporations are deliberately kept secret, defended as intellectual property. Contractors may write algorithms used by governments and protect them similarly. In the analog era, if a journalist wanted to question the possibly discriminatory effects of a new program like predictive policing, he or she could interview the city official in charge or FOIA the details. Algorithms present a black box that requires sophistication and experimentation to illuminate.

A pioneer of “Algorithmic Accountability,” as this emerging field is sometimes called, is Julia Angwin, author of Dragnet Nation and now a reporter at ProPublica. With Jeff Larson, the nonprofit’s data editor, Surya Mattu, and Lauren Kirchner, she published an investigation in May that showed how an algorithm used across the country to predict future criminal behavior discriminated against African Americans in bail and sentencing decisions. Angwin and Mattu recently reported as well on how Amazon appears to use its “market power and proprietary algorithm to advantage itself at the expense of sellers and many customers,” as their story put it.

These sorts of investigations show journalists applying the math, statistics, and data science knowledge necessary to investigate contemporary hard targets. It may not always be glamorous, or drive traffic, but it is essential if professional journalism is to reaffirm and defend its First Amendment role in an era when code is power and coders may be hubristic, motivated by profit, or unprepared by their own training or professional tradition to identify and evaluate the ethical and discriminatory effects of automated decision-making.

A second field of technology innovation that has already produced breakthrough investigative reporting deploys sensors. Devices that can monitor air and water quality or detect cracks in pipelines are cheaper, faster, and more portable than even a decade ago. The spread of broadband and other connectivity means that dispersed sensors can be rigged to feed data to a hub or dashboard in real time, so that citizens may monitor it, not just companies.


If automation can replicate some routine news reporting accurately and discerningly, it may cost more reporters their jobs, but it may also free up reporters to go deeper.


Investigative reporters in India and China use air pollution sensors to hold their governments accountable for air pollution. In the United States, environmental reporting that used to require months and costly collaborations with laboratories can be carried out in creative new ways. More than a decade ago, the Houston Chronicle’s Dina Cappiello pioneered the use of air pollution sensors distributed across neighborhoods in an investigation of whether EPA standards were being met in disparate areas of the city. Blake Morrison at USA Today led an investigation into water pollution caused by old lead factories where the reporters carried their own sensors into the field. The Florida Sun Sentinel conducted an ingenious investigation into speeding by off-duty policemen that resulted in fatal crashes by acquiring and analyzing data from highway tollgate sensors. That series won the Pulitzer Prize for public service in 2013.

The most promising field of innovation that is still emerging, but has yet to produce such impressive models, involves machine learning and artificial intelligence. The automation of news—like artificial intelligence in general—has been slow to live up to futuristic forecasts but may finally be approaching its promise in applications like self-driving cars. If automation can replicate some routine news reporting accurately and discerningly, it may cost more reporters their jobs, but it may also free up reporters to go deeper.

Campaign finance is one promising field. Meredith Broussard, an assistant professor of journalism at New York University and Tow fellow, led a team during this year’s presidential election that tested an artificial intelligence tool that can crawl over public campaign finance data and recognize potential stories as an aid to human reporters. Imagine computers programmed to improve their discernment by repetitious action crawling over the records of police discipline cases, conflict of interest disclosures by politicians, or the minutes of local zoning board meetings. They might be developed to identify anomalies or story leads faster and more comprehensively than human reporters could hope to do. In time, they might even provide a way for reporters, citizens, and nonprofits to replicate at sustainable cost some of the watchdog function that local professional newspaper reporters used to provide, before the vortex of the digital revolution and the Great Recession swallowed so many local reporting jobs.

The idea that journalism might become a disinterested, independent, public-minded profession that keeps watch on rich, powerful individuals and institutions arose before the spread of radio. It has survived and adapted through one technological and media business disruption after another, and through the rise and decline of major industries.

“Where’s the goddamn story?” the Ben Bradlee character says to Bob Woodward in All the President’s Men.

“The money’s the key to whatever this is,” Woodward answers, referring to the scandal that became Watergate.

The parameters of public interest journalism have changed so little across a century because the country’s patterns of public corruption, racism, and abuse of power haven’t changed very much, either. Venture capitalists ask of a new technology: Can it scale, disrupt, and create opportunities for profit? For journalists there is another question: Can it help us follow the money?

Steve Coll has been Dean of Columbia Journalism School since 2013. A former president of the New America Foundation, he is also a staff writer at The New Yorker, where he writes on politics, national security, and the media. He is the author of seven nonfiction books, a two-time winner of the Pulitzer Prize, and a former reporter, foreign correspondent, and senior editor at The Washington Post.