This past week, two polls (one from Zogby, one from Rasmussen) found that the governor’s race in Texas was closer than expected. According to both polls, incumbent frontrunner Rick Perry had dropped down to less than 35 percent of the vote.
“Conventional wisdom in the governor’s race has been that none of the governor’s four opponents would have a chance to beat him if he gets more than 35 percent of the vote Nov. 7,” reported the San Antonio Express-News. “The Rasmussen poll put Perry’s re-election support at 33 percent, and the Zogby poll had it at nearly 31 percent.”
Daily papers around the state reacted by pumping up the results of the polls. From the Houston Chronicle: “Polls Find Perry Loss Isn’t Out of the Question.” From the Fort Worth Star-Telegram: “Push is on for candidates to break out: Challengers say recent polls show Perry can be defeated.” From the Express-News: “Two polls show Perry defeat is possible.”
Afterwards, Perry aides questioned the reliability of the polls — one of which, the Zogby interactive poll, relies upon an untraditional methodology that has raised some eyebrows in past elections. According to its Web site, Zogby interactive polls tap a self-selected group of respondents who sign up for surveys online and then respond to specific questions via email.
So was the Perry camp correct to question the Internet-based poll?
It’s a question with relevance far beyond the Lone Star State. For the mid-term elections, Zogby International has teamed up with the Wall Street Journal to create an interactive, Battleground States Poll (the entire methodology can be found here) which will be gauging (and perhaps shaping) voters’ preferences in 18 Senate races and 19 gubernatorial races in the weeks to come.
When reached by phone last week, Cliff Zukin, a political science professor and polling expert at Rutgers University, suggests that journalists should generally be wary of any Zogby interactive poll.
“The Zogby stuff, on scientific grounds, is quite questionable,” says Zukin. “Online, Internet, opt-in polling, where people volunteer to be respondents, doesn’t really have a basis in scientific validity. There are two kinds of samples in the world. There are probability samples, and there are non-probability samples.”
The Zogby interactive polls, says Zukin, clearly fall into the latter camp. “With probability samples, when everybody has a known chance of being selected, you can make pretty valid inferences about the population from which it is drawn,” says Zukin. “You can’t do that at all with self-selected surveys. That’s a problem.”
Another problem with Internet-based polling, says Zukin, is that, in general, Web and email-based surveys tend to overvalue the opinions of young people. A group that is notoriously lousy at showing up to actually vote.
“Internet coverage is now about two-thirds of the population,” says Zukin. “But it’s really age-skewed and, to a lesser extent, education-skewed, in the wrong way for voters. It’s younger people who are online. It’s older people who are not online. It’s older people who vote. And younger people who don’t.”
“It’s certainly not the gold standard,” says Zukin.
It’s a point that wasn’t lost on at least one revered political journalist in Austin — the Texas Monthly‘s Paul Burka. Writing last week on his recently launched blog, Burka did a nice job of unpacking the shortcomings of both polls. In particular, his critique of the Zogby-Journal Battleground States Poll should be required reading for journalists reporting on midterm elections around the country. (Burka also provides some interesting thoughts about the automated phone call methodology that underlies the Rasmussen poll.)
“The [Zogby-Journal] poll’s Web site describes the process as ‘interactive’ — that is, it’s an Internet poll, based on a database of individuals who have signed up to participate,” writes Burka. “It is not a random sample; the polling organization solicits responses by e-mail. In addition, the poll takers make about 20 to 50 phone calls in the state where a race is taking place. The poll does not mention a screen for likely voters.”
“As I have written repeatedly, the poll that I think is the least credible is Zogby/Wall Street Journal/Battleground States,” adds Burka. “I can’t believe the Journal allows its name to be attached to this so-called poll.”
Felix Gillette writes about the media for The New York Observer.