Claire Merchlinsky

Our Polling Trauma

July 8, 2020

The brain learns in different ways. Mastering French verbs requires repetition. A single distressing experience, however, can impart a lasting lesson in moments; a car crash survivor, for instance, might come away with a deep fear of riding in cars. A lesson like that is hard to unlearn, even if its basis is irrational. For anyone who analyzes political polls—including me, a neuroscientist hooked on election forecasting—the 2016 presidential race may have constituted a similarly traumatic event, one that left the mind reeling.

Before that November, many political reporters, accustomed to conventional presidential candidates, found the idea of a Donald J. Trump victory unthinkable, even though the data pointed in a different direction. National opinion polls suggested that Hillary Clinton was favored to win by an average of three percentage points; in models that took into account the Electoral College, her advantage looked even smaller, about two percentage points. Rationally, the race should have been presented in news coverage as a close battle. And that’s what came to pass on Election Day, when Trump’s win was one of the narrowest on record. Yet the outcome felt like a shock.

I was among those who made a miscalculation. I used to joke on the Princeton Election Consortium, my election analytics website, when I was highly certain about races: I’d eat a bug if the results were to turn out contrary to my expectation. I tried to calibrate my assertions so that I would lose the bet no more than once in a lifetime. In 2016, I met my quota. I’d overestimated the precision of my model, making a final prediction of a Clinton win with 93 percent probability. Those who later saw me consume a cricket on CNN witnessed the bet’s payoff.

I could have avoided that embarrassing experience had I not converted polling margins into win probabilities. In my prediction, I should have emphasized that a two-percentage-point difference in public opinion between the polls and the outcome might be enough to give the election to Trump. That distinction is important because journalists and readers tend to err toward excessive certainty. If you say, for instance, that Joe Biden has a 70 percent chance of winning this November, a reader might hear that he has a 100 percent chance. Indeed, Nate Silver’s model in 2016 was appropriately uncertain, assigning Clinton a 71 percent chance of becoming president—and allowing about a 29 percent chance of the opposite happening—but Silver was criticized for being “wrong.”

Consider a different context: if you’re offered a chance to play Russian roulette and the gun is one-third loaded—a 33 percent probability of a disastrous outcome—don’t play. Or take covid-19: no science-savvy journalist would report that the outbreak is on the decline based on a probability from a mathematical model. With stakes that high, one would describe the trajectory of the pandemic based on direct measures, such as the trend in the number of deaths per day. Verifiable numbers tell readers whether conditions are getting better or worse without making predictions about the future.

After the upset of 2016, many political journalists might be tempted to fear polling data. Paying attention to opinion surveys, these reporters worry, could lead the public astray. But psychological research shows that a disturbing memory, born of trauma, can focus on the wrong triggering event. If we become afraid of polls, I think we are learning a false lesson. It isn’t that polls were inaccurate during the last presidential election—viewed in terms of percentage-point margins, they weren’t far off. The problem is that our brains may have turned an emotional experience with polling into a lasting trauma.

Elections are hard to predict and can be intensely emotional.

 

During the Gilded Age, the United States witnessed massive disruption—there were technological innovations such as railroads, a shift from an agrarian to an industrial society, and deep social divisions over the rights of formerly enslaved Black people. It was commonly believed that the end of the world was near. These conditions feel familiar in America today, as questions of race and immigration loom large; the economy has again been remade by technology; and climate change, nuclear weapons, and the coronavirus pandemic make Armageddon seem imminent. The Gilded Age, too, was characterized by polarizing political conflict. And, like the modern examples of George W. Bush and Trump, two popular-vote losers won the Electoral College and became president: Rutherford B. Hayes and Benjamin Harrison.

In such close circumstances, elections are intrinsically hard to predict. At the same time, politics can take on an intensely emotional tone. Based on recent data, it would seem there is a correlation between strong political feeling and stability in public opinion. For example, since Trump’s election, the level of public approval (and disapproval) of the president has remained as firm as it’s been since Gallup started tracking those numbers, in the forties. According to the FiveThirtyEight average, which combines data from different pollsters, Trump’s approval rating has stayed within the range of about 37 to 43 percent for almost his entire presidency. Trump is the very model of stability—as the least popular incumbent in the history of polling.

Trump’s job approval numbers stayed within that same narrow range even as the coronavirus spread across the United States. Contrast his performance with the ratings of governors and other world leaders, many of whom saw large jumps in popularity as the pandemic unfolded. Also recall Jimmy Carter’s initial ratings leap during the Iranian hostage crisis, and Bush after 9/11, when his approval shot from 55 percent to 90, the highest any president has ever had. Compare Bush’s thirty-five-point spike with Trump’s “coronavirus bounce,” a mere two percentage points lasting just a few weeks and ending with a net decline. We can interpret this as voter entrenchment: people either find Trump a threat to the republic or think that he’s made America great again. Under fiercely emotional conditions, we don’t take time for introspection or the slow evaluation of evidence. Instead, we rely on the resources that we have most readily available: our prior beliefs and prejudices. Almost nothing can change our minds.

That stability helps Trump stay afloat as a contender for reelection. In campaign after campaign, incumbent presidents have won a share of the vote close to their approval rating. Based on geographic patterns from 2016, Trump needs only about 48 percent of the two-party vote (that is, the total of Republican and Democratic votes) to have a shot at winning the Electoral College. At the upper end of his range of approval, thanks to his committed base, Trump would be within a few points of that threshold.

Of course, as the election approaches, a better indicator than presidential approval rating will be direct polling of the Biden-versus-Trump race. In an analysis of election-year presidential polls going back more than sixty years, I have found that the total range over which support for candidates moved up and down from May to November—the variability—used to be much greater than it is now. Shortly after Newt Gingrich’s takeover of Congress, in 1994, and the founding of Fox News, in 1996, opinion became more locked in—fluctuating half as much as it did before, or even less. At this point, many Trump supporters and opponents believe, perhaps because of how fixed voters are in their positions, that he’s a lock for reelection.

There’s no need for political reporters to be skittish; the polls of the past few years have largely been accurate.

 

That belief is false. It can be overcome by focusing not on national surveys or models, but on polls in individual states. Journalists covering the 2020 elections would be wise to remind readers that state-level polling averages usually land within a few points of final outcomes. If you’re inclined to go deeper, the National Council on Public Polls offers a handy journalist cheat sheet. (But don’t get into fine-grain technical questions like sampling. It’s best not to second-guess the methods of a particular pollster—that runs the risk of motivated reasoning, which in the cognitive-science world means you’re seeking a certain answer based on your bias.) You can also seek out confirmation for findings by viewing multiple polls.

Covering state and local elections can not only help us overcome a fear of polling. It can also foster interest in a subject that news consumers can actually do something about: down-ballot races, where citizen engagement is often weak. Along with gerrymandering and electoral reform, that’s my main focus now: tracking where political races are tightest. It helps my readers learn where their efforts will make the most difference, because close elections can be turned by activism.

In 2018 and 2019, polls were pretty good at identifying close campaigns. In 2018, for instance, Democratic congressional candidates led Republicans in national surveys by a median of 8.5 points. That year, Democrats retook control of the House with a popular-vote margin of 8.6 points. In 2019, polls showed that, in Louisiana and Kentucky, the top candidates for governor were within a few percentage points of each other. Democrats took both races, each by less than three points; in both states, campaigns and get-out-the-vote work proved pivotal.

There’s no need for political reporters to be skittish; the polls of the past few years have largely been accurate. By continuing to present the data, journalists can walk back the trauma of 2016—and reveal important truths about the American electorate. Predictions can be off, to be sure; the key is to keep coverage simple, put facts in context, and not overreach. It takes many experiences, but fear can be unlearned.

Sam Wang is a professor of neuro­science at Princeton University and the director of the Princeton Election Consortium.