There has been institutional resistance among traditional media outlets toward using the full potential of the Web, Pogue said. His early efforts at Web reporting for The New York Times were an example of that resistance.
Editors were reluctant to invest in videos, graphics and blogs; there was poor interactivity and Pogue received very little feedback on his work. The paper’s mentality was that such efforts did not sell ads and generate
revenue, but Pogue argued that they would “improve the brand.” Then the Times “opened it up … and everything changed.” The Times added a comments section under blog posts, for example, and Pogue received 1,335 comments over the first weekend.

Citing the value of “instant feedback,” Pogue then returned to his main point about user-generated reviews, showing a slide of stampeding buffalo that drew laughs from the crowd. He then listed a number of Web sites that
review everything from consumer electronics, to doctors, to cars, to boyfriends. One of the best, he said, is the Internet Movie Database (imdb.com), which includes user reviews of movies. This is where the illogical becomes logical. Even if there are a few “jerks” and bad reviews, they will be marginalized by the thousands of reviewers chiming in. “The cumulative wisdom will steer you in the right direction,” Pogue said. Yet there are still poorly managed comment and review sites, he added, citing YouTube as one example. But the advantages of social media are clear. Even in the case of fake reviews and businesses trying to “game the system” for their own benefit, Pogue said, “the real stuff greatly overwhelms the bogus stuff.” And innovations that address these problems keep coming, Pogue explained. TripAdvisor, for example, is a Web site where “you can search and
sort not just for how good a hotel is, but for how good it is for your kind of person.” Pogue also cited Shopping.com, where consumers are not only able to search for the cheapest products, but for the cheapest products from reliable vendors. “There are technological solutions to a lot of this stuff,” he said.

None of this, however, means that professional reviewers are obsolete, Pogue cautioned. One of the advantages of full time critics is that consumers can get to “know his tastes,” he said, adding that he and his wife “steer their
ship” by a New York Times film critic whom they think is reliably wrong about children’s movies. When it comes to the relationship between professional and amateur consumer reporters, Pogue said, there has not been
as much displacement as most people think. Most user reviews are simply “add-on,” he said. Plus, he added, there are inherent problems with user reviews, which tend to be self-selecting for people who have had particularly bad experiences with a product or service. This is known as the “Squeaky Wheel Syndrome.” And no matter how hard we try, there will always be some gaming of the system, Pogue warned, citing Microsoft’s practice of sending laptops to reviewers and asking them to “dispose of them as they saw fit” when finished. Of course, most kept them, so the practice amounted to a form of payola. It is also easy to disseminate false, but incredibly damaging, rumors about products and services anonymously on the Web.

Pogue closed by calling for the development of a “Code of Ethics” among user review Web sites: take responsibility for words and comments you allow; when you believe that someone is unfairly attacking another, take action; and do not allow anonymous comments. Pogue noted that this has worked at many Web sites and that, employing another technological innovation, some sites now allow reviewers to review each other and the quality of comments, so that only the best rise to the top of the page.

Panel Discussion: Are Consumers the Right Watchdogs?

After Pogue finished, Kevin McKean, the vice president and editorial director of Consumer Reports, took the stage to moderate a five-person panel discussion on the question, “Are Consumers the Right Watchdogs?” McKean
started by noting that the “financial collapse” has made consumer reporting and protection issues “even more timely.” He then introduced Mike Hoyt, the executive editor of Columbia Journalism Review, who outlined a three-part package in the September/October 2008 issue of the magazine called “The Rise and Fall - and Rise - of Consumer Journalism.” The authors of the three articles - Evan Cornog, David Cay Johnston and Trudy Lieberman — all took part in the conference. (The articles can be found at www.cjr.org )

Curtis Brainard is the editor of The Observatory, CJR's online critique of science and environment reporting. Follow him on Twitter @cbrainard.