FLORIDA—Late Saturday night, the Tampa Bay Times and the Miami Herald released the results of a new Mason-Dixon survey of Florida Republicans. One didn’t need to see the poll numbers to know the newspapers believed the results were dramatic.
Times political editor Adam Smith wrote:
Mitt Romney needed Florida to resuscitate his campaign after a South Carolina routing, and on Tuesday, Florida is poised to deliver big.
A new Tampa Bay Times/Miami Herald/Bay News 9 poll found Romney easily beating Newt Gingrich among likely Republican primary voters, with 42 percent support to Gingrich’s 31 percent. Rick Santorum trails with 14 percent, followed by Ron Paul at 6 percent.
And Marc Caputo, political writer for the Herald, wrote:
Newt Gingrich swaggered into Florida as a Republican front-runner, but now he’s close to slipping out as an also-ran against a resurgent Mitt Romney.
Gingrich is badly trailing Romney by 11 percentage points, garnering just 31 percent of likely Republican voters heading into Tuesday’s presidential primary, according to a Miami Herald/El Nuevo Herald/Tampa Bay Times poll released late Saturday night.
Both reporters went to considerable length to describe the complexities of the campaign and the reasons for Romney’s apparent resurgence and Gingrich’s apparent slide. Where they failed, however, is in what they did not report.
Newspaper polling (and the coverage of such polling) has long been a pet peeve of mine. Smith, Caputo and others with whom I worked as political reporter have been subjected to my railing about newspaper polls countless times. I have written about the issue and what newspapers can do better on my blog, Crowley Political Report.
In Caputo’s and Smith’s stories, there is a clear example of what is wrong with newspaper polls and the coverage thereof. Both reporters wrote that their poll showed that Romney has a 24-point lead over Gingrich among Hispanic voters—52 to 28 percent. But readers were not told important details about these numbers. I interviewed Brad Coker of Mason-Dixon Polling and Research who conducted the poll for the newspapers. He said the survey of 500 registered, likely voting, Republicans included just 75 Hispanics. Of those 75 Hispanics surveyed, the number consisted “heavily” of Cuban Americans who live in Miami-Dade county.
Now, the argument can be made that Miami-Dade Cuban Americans will be the overwhelming majority of Hispanic voters in the Florida primary, as Caputo explains in his story. But neither Caputo nor Smith tells readers that their Hispanic survey is, in fact, not a comprehensive look at Florida Hispanic voters. Coker told me that if he were doing a detailed survey of Hispanics he would have surveyed 400 not 75.
Once you realize that the survey sample is 75, it begs the next question: What is the margin of error? The Times and Herald reported that the margin of error for the 500 Republicans surveyed was 4.5 percent. True. But what they did not tell readers was the margin of error for the 75 Hispanic voters surveyed, which Coker told me was plus or minus 12 percent. That means that the percentage of Hispanics who support Romney ranges from 40 percent to 64 percent and the percentage supporting Gingrich ranges from 16 percent to 40 percent.
What does it really mean? That there is little statistical value in the number of Hispanics surveyed. This is a common problem when news organizations report subgroups in their polls. Often the numbers surveyed are too small to reveal any meaningful information.
The American Association For Public Opinion Research warns journalists:
Sample sizes below 100 will have a large margin of sampling error—plus or minus 10 percentage points for a sample size of 100 and increasing as the sample size declines. Journalists should avoid reporting on groups this small unless there is a compelling reason to do so, and then only after consulting with an independent polling expert.
(Here’s a very useful chart for journalists that offers information on margin of error for subgroups).