Former New York City Mayor Ed Koch died in the wee hours of Friday morning, and The New York Times had its obituary ready to post–sort of. Poynter noted that Koch’s Times obit was updated multiple times to add paragraphs on his failure to address the AIDS crisis as well as to tweak the language surrounding his silence about his sexuality.
These changes to Koch’s obit might have been slipped into the piece under the radar if not for NewsDiffs, a site created last June that tracks and archives changes to stories in the Times and a few other prominent mainstream outlets. The site attracted coverage when it first went live, but it continues to serve as a unique source for media analysis in an era when journalists can revise copy with a click.
Journalist and former Timesian Jennifer 8. Lee had this watchdog sensibility in mind when she and Eric Price, her then-boyfriend’s younger brother, created the site, which also trawls CNN, Politico, and the BBC. When these sites update articles, NewsDiffs displays and highlights what changed with a prominent link back to the original story page.
They created NewsDiffs at a hackathon at the Massachusetts Institute of Technology, where Price is finishing up a graduate degree. Lee was crashing on his couch the night before the MIT-Knight Civic Media Conference, of which the hackathon was a part. The idea was to stick a bunch of digital geeks in a room overnight, with Indian food, and see who could emerge with the most innovative code to solve an issue related to digital journalism.
“He’s an amazing programmer and so we sat there, and I didn’t really know him at that point,” said Lee. The whole creation process, as subsequent coverage reported, took “all of 38 hours’ work, including sleep.” NewsDiffs, written in Python, is open source–it’s posted on Github–and versions have been built in Spain, Argentina, India, and Germany. A South African version is in the works, Lee said.
The idea to scrape news websites and compare versions evolved in the midst of the hackathon, Lee said. Someone suggested building a way to archive The New York Times‘s front page, but Lee thought it would be more useful to track updates.
NewsDiffs’s launch was heavily covered, including by the Public Editor at the Times, as filling a media niche, but it’s remained a side passion project, hosted on MIT servers and tweaked over holiday weekends. The most recent update, executed during Thanksgiving, made the system 20 times faster; the backend work mitigated the fact that the accumulation of content slowed the site. But any further updates are contingent on busy people having time to do them.
Price, enmeshed in a job search, said he hoped to one day build a way to differentiate meaningful changes NewsDiffs picks up from inconsequential punctuation fixes and the like.
“There aren’t people looking through it trying to find interesting changes, because there are too many changes a day to do that reasonably,” he said. Indeed, the Koch obituary is surrounded by both records of stories with minor language updates as well as changes like the insertion of a pithy quote by Hillary Clinton, and there’s no way to tell if tracked changes will be significant without browsing through the exhaustive list.
And that’s a shame, because when the site does scrape significant changes that aren’t noted as corrections, it’s possible to catch the Times violating its own standards. One update to Koch’s obituary changed a mention of the late mayor as a possible “closeted homosexual” to “gay man.” CJR Minority Reports columnist Jennifer Vanasco has pointed out that it’s the Times‘s own policy to avoid the word “homosexual” because, as the Gay and Lesbian Alliance Against Defamation wrote, it’s “a word whose clinical history and pejorative connotations are routinely exploited by anti-gay extremists to suggest that lesbians and gay men are somehow diseased or psychologically/emotionally disordered.” (Vanasco has also written about the trickiness of “outing” a semicloseted figure in an obituary.)
And it’s no longer necessary to waste words debating whether the Times and sister publications should publicize its ongoing editorial process, because NewsDiffs is doing it for them–and snagging pageviews in the process. Lee said that the Koch obit updates on NewsDiffs received about 1,000 views, an “unusual” number of hits for the site.
Kira Goldenberg was an associate editor at CJR from 2012-2015. Follow her on Twitter at @kiragoldenberg.