the news frontier

Survey Question: Do You Trust This Poll?

Local news sites informally collect community opinions
February 16, 2012

Every week, Phoenix-area hyperlocal news site InMaricopa asks its readers to participate in a brand-new poll; each usually gets at least a couple hundred responses. Publisher Scott Bartle says that in the past month they’ve started using the data from completed polls as the basis for a subsequent story. The subject matter varies widely in tone, from silly—“Are you celebrating Valentine’s Day or Arizona Centennial on Feb. 14?” (Neither: 47%)—to reflective—“If you had to do it all over again, would you have moved to Maricopa?” (No: 49%)—to serious—“Do you think [mayoral candidate] Christian Price should have challenged [opponent] Carl Diedrich’s signatures?” (Yes: 66%).

The outcome from the Price/Diedrich poll was later referenced by both candidates, which InMaricopa reported on. “Price said the poll results support his actions,” writes reporter Tim Howsare. “Diedrich pointed out the poll is not scientific and there is no way to know where the respondents live or if any were encouraged by his opponent’s campaign to respond in favor of their candidate,” which is, of course, true. The poll widget on InMaricopa, like most of these types of software, place cookies on a computer to track if someone has already voted, a system that’s easy to manipulate. A local message board explored the possibility that people voted multiple times; Carl Diedrich’s wife even chimed in to say she’s “never put actual stock in the results and neither does anyone who knows how web polls work.”

A number of local news sites host polls. They’re an easy way to get readers involved in the site, while showcasing a (non-scientific) tally for what’s on local residents’ minds. At Somerville Today, a hyperlocal in New Jersey, editor Loren Fisher says he puts up new questions as fast as he can come up them, but only if he thinks the question addresses a “hot topic” in town: “If there’s nothing super hot, I’m not going to throw up, ‘What’s your favorite color?’” He says he’s never had people complain about a lack of technical sophistication (“I think people realize that 100 people responding isn’t very scientific”). But some hyperlocal editors have encountered residents who are upset with their casual opinion-gathering.

Maggie Kulbokas, the managing editor for Massachusettes news sites Cape Cod Today and Plymouth Daily News, runs polls on both sites, and says their most popular polls are either goofy or political. Cape Cod Today, the older of the two sites, used to get single-digit responses to their polls. But as the years went by it became a quite popular feature, and now polls get a triple digit response. She says there are times when people take issue with the questions, especially the political ones. “We are pretty tongue-in-cheek about them,” says Kulbokas, referring to last week’s question: “Pretend you’re a Republican and vote in our Primary.” “You have to take it with a grain of salt,” says Kulbokas.

Amy Senk, the founder of Corona del Mar Today, says she has consciously chosen to stay away from serious topics in her polls after this past summer. When people were protesting a new pet store, saying it bought animals from puppy mills, Senk asked: “Should Newport Beach Ban Sales of Non-Shelter Puppies in Pet Stores?” The poll agitated one of her regular readers, Dan, who wrote in the comments: “I voted 26 times ‘i do not have an opinion’ to make the point this survey is unreliable.”

“He was really worked up about it, he wanted a disclaimer explaining that they weren’t scientific,” says Senk. “So I decided maybe I would just stick to Girl Scout cookies.” She’s keeping it light until she finds a good way to communicate that this is not a refined survey. “If I want to do public controversies than I should make clear that it’s for fun,” says Senk. “Maybe I should do a poll over whether people think the polls are scientific.”

Sign up for CJR's daily email

To read more about InMaricopa, click here.

To read more about Somerville Today, click here.

To read more about Cape Cod Today, click here.

To read more about Plymouth Daily News, click here.

To read more about Corona del Mar Today, click here.

Alysia Santo is a former assistant editor at CJR.