The COVID Tracking Project is (nearly) gone. Can we see clearly now?

One evening in early March of last year, Alexis C. Madrigal and Robinson Meyer, colleagues at The Atlantic, set out to answer a simple question: how many people had been tested for the coronavirus in the US so far? The answer, it turned out, was actually quite complicated: in the absence of data from the Centers for Disease Control and Prevention, it was hard to tell whether low reported case rates to that point reflected low incidence or low testing. Madrigal and Meyer sent a form email to health officials in every state; they soon found out that the answer was the latter, and that the federal government did not have a handle on the numbers. As Emily Sohn reported for CJR, Jeff Hammerbacher—a data scientist who had been working to track the same information, and who knew Madrigal from college—saw their work and reached out. They teamed up, and soon, the COVID Tracking Project was born. It was meant, initially, as a short-term gap-filler. “Every day,” Erin Kissane, its managing editor, told Sohn in late March, “we hope the CDC will put us out of business.”

But the days went by, and the CDC did not, leaving the Tracking Project’s collective of journalists and tech folk to serve, in their own words, as “a de facto source of pandemic data for the United States.” The Atlantic agreed to host the project; its team grew to include hundreds of volunteers, and the project’s founders solicited philanthropic donations to pay some of them. “It just got really complex,” Madrigal told CNN’s Brian Stelter recently. “At first, we just thought you could pull numbers off dashboards and that was gonna be the main process. As it turned out, we actually needed to do deep research. We made hundreds and hundreds of contacts with state officials… We added race and ethnicity data, we added outcomes like hospitalizations and deaths, we added tracking of long-term care facilities. So all of those things came together to just require tons of labor; I mean, tens of thousands of hours of people’s time.” State and federal officials cribbed the project’s data. Other journalists became reliant on it.

ICYMI: Adbusters and the roots of the contemporary left

Now, finally, the project is winding down. It stopped collecting data ten days ago; it plans to spend the next couple of months on analysis work and archiving before shuttering completely. The analysis will mainly focus on data streams—around race and ethnicity, for instance—that are still insufficient, but the project’s founders are confident that federal officials are finally adequately replicating much of its core data, and, crucially, that the new Biden administration can be trusted not to turn off the lights at a moment’s notice. Writing last week, however, Madrigal and Kissane warned that the landscape of coronavirus data is “persistently messy.” The pandemic response “has been nominally or actually based on the idea that we know what is happening across the country to a precise degree,” they wrote. “And although this idea—that we have solid data about the movements of the pandemic and the effects of the response—is much closer to truth now than it was a year ago, it’s very far from universally accurate.”

This week, Madrigal and Meyer expanded on that theme in an essay for The Atlantic, pinpointing key truths about pandemic data and officials’ handling of it. The initial pandemic response plan assumed the presence of good data, and failed to account for kinks at every part of the data-production pipeline, from the local, to the state, to the federal. We now have much better national-level data on cases, tests, hospitalizations, and deaths—but this still offers a rather messy compilation of snapshots and time lags, not a real-time “window” into the current state of the pandemic, as officials sometimes imply. (Deaths often substantially lag cases, and commonly go unreported for weeks.) At least five states still “have disturbingly incomplete testing data”; nationally, tens of millions of rapid antigen tests appear to be unaccounted for. “Data might seem like an overly technical obsession,” Madrigal and Meyer write. “But data are how our leaders apprehend reality.”

The press, too. Over the course of the pandemic, responsible news organizations have pledged to “follow the science” and extolled the virtues of data-driven analysis, but as Madrigal and Meyer note, looking at data is less like “peering into a crystal ball” than “watching city sewers empty into a wastewater-treatment plant”—it’s more useful than not looking, “but you shouldn’t delude yourself about what’s in the water.” And yet, as I wrote in the early days of the pandemic, reporters moved quickly to paint unduly certain pictures with extremely incomplete information; reported cases and deaths, for example, were routinely relayed as “totals,” not as clear undercounts, which, of course, they were.

Sign up for CJR's daily email

Over time, data flows improved; testing, in particular, expanded, even prior to the election of a president who doesn’t view it as a conspiracy to make him look bad. But testing remains highly imperfect. Per the Tracking Project, reported daily tests hit a high-water mark close to 2.3 million in December—a figure, some experts then calculated, that was adequate to identify most symptomatic COVID cases and close contacts, but still fell well short of the daily rate that would be needed to properly control viral spread by identifying asymptomatic people. From a public-health standpoint, these metrics are subject to healthy scientific debate; from a journalistic standpoint, every additional test is an extra confirmed truth about the world, and yet media-industry leaders have never pushed with any real consistency or urgency to frame expanded testing, as I suggested last year, as a freedom of information issue. (We’re better at demanding to see information the government is hiding than information the government is failing to collect.) In the absence of a further massive ramp-up, the percentage of tests coming back positive often offers more useful insights than flat numbers, yet many outlets haven’t pivoted to this metric with any consistency either. And in recent weeks, testing rates have generally declined. This could be a function of fewer people needing one (though, again, it depends how you define “need”), but it could also reflect public exhaustion with mitigation measures, bad weather, states’ shift to vaccine infrastructure, and other contingencies.

As I wrote recently, journalists covering the pandemic still have to grapple, over a year in, with massive uncertainty. We still haven’t figured out how best to communicate this—not only around new challenges like vaccines and variants, that currently attract the bulk of our attention, but around the very basic building blocks that we use to define and measure surges and lulls, sickness and death. The visibility of these underpinning realities is still murky. Some outstanding journalists—not least at the Tracking Project—have focused on finding outlines of clarity, and for that we owe an enormous debt of gratitude. Many of us have instead acclimated to the murk, preferring to see what’s in front of us, even though it isn’t the full picture.

Below, more on the pandemic:

  • Two guiding lights: CNN’s Kerry Flynn profiled The Atlantic and Stat, two publications, she writes, that have “excelled at guiding readers through an unprecedented and scary pandemic” while other newsrooms have struggled. “Rather than race to break news or go for quantity of stories, they each focused on explanatory pieces about the realities of the pandemic,” Flynn writes. “The leaders of both newsrooms used the same word, multiple times, to describe the source of their success: ambition.”
  • A messaging disaster: In recent days, European countries including France, Germany, and Italy have suspended use of AstraZeneca’s coronavirus vaccine following reports of blood clots in recipients—even though the number of patients affected is small, there is no evidence of any causal link, and the EU and the World Health Organization have strongly said that rollouts should continue. As the New York Times reports, “it appears increasingly clear that the suspensions have as much to do with political considerations as scientific ones,” posing a messaging challenge for health officials and for the press. (This is not the first time the AstraZeneca vaccine has become embroiled in a media and political storm, as I reported recently. The US has yet to approve the vaccine.)
  • Withdrawn: After a German publisher released A Corona Rainbow for Anna and Moritz, a children’s book that seeks to explain the pandemic, Chinese diplomats in Germany filed a formal legal complaint, pointing to a line in the book in which a character describes the coronavirus as having “come from China.” In response, the publisher halted deliveries of the book and committed to printing a new version with the China line taken out. A spokesperson said that the meaning of the original wording “has proven to be far more open to interpretation than we had intended.” Deutsche Welle has more.


Other notable stories:

ICYMI: NPR’s tale of two crises at the Capital Gazette

Has America ever needed a media watchdog more than now? Help us by joining CJR today.

Jon Allsop is a freelance journalist. He writes CJR’s newsletter The Media Today. Find him on Twitter @Jon_Allsop.