The first thing you notice about NewsTrust Baltimore, an online aggregator of stories from local news sources, is how friggin’ civil all the commenters are. Consider one reader’s critique of a story about the Supreme Court’s decision upholding the Westboro Baptist Church’s right to protest at military funerals:

This story does a good job of putting a difficult and emotionally charged story into context and providing the background needed to understand the issues involved in the court case.

Elsewhere in this comment section, NewsTrust readers debate how effective the story’s mixture of AP and local CBS affiliate content is, and consider whether they catch a whiff of bias in the lede. Other stories on the site are similarly dissected, with users sharing even-handed opinions and critiques on articles covering everything from state government hiring practices to “Baltimore’s fledgling food-truck scene.”

Thoughtful, measured critiques probably sound like a dream for readers and writers accustomed to the flame wars fought in many online news comment threads. And though relatively few people used NewsTrust Baltimore during its six-month lifespan, their sobriety of purpose set an example that others would envy.

But the NewsTrust Baltimore experiment raises the question: Is civility sustainable? Unable to get continued foundation support, and not close to being commercially viable, the project finished without hinting at a workable business model. And the relatively small number of articles that sparked robust discussion suggests that there are limits to how deep news consumers want to get in the news-critiquing business.

The project—which was actively curated from February through July of this year and still features new user-updated content—localized the online social networking/media watchdogging tools developed by the national nonprofit NewsTrust. A small staff aggregated each day’s Baltimore news from a variety of legacy and new media—personal blogs like to online-only news sites like the Baltimore Brew to the Baltimore Sun, the area’s major daily. Excerpts of those stories appeared on the NewsTrust Baltimore homepage; clicking through led to the story at its original online location framed by NewsTrust’s reviewing tools.

The site’s users took over from there. Readers could give a story one to five points, rating an article on qualities like whether it was factual, well-sourced, well-written, and enterprising. (Disclosure: I was a Baltimore journalist until April. One story of mine was posted to the site. It got a four, based on one review.) Energetic reviewers could go further and write narrative reviews, highlight quotes, and add links. Stories that got positive reviews and ones from news sources already ranked highly could receive more visible placement on the NewsTrust Baltimore site.

Fabrice Florin, NewsTrust’s executive director, says that the reader reviewing process has a dual purpose. “One is to help rate the news based on quality,” he explains. “But that’s really a means to an end and the real purpose is to help each and every one of us who participates in the process become more discriminating as a news reader.”

In order to review a story, readers first have to create a public profile. Reviewing readers then receive their own ranking, which rises or falls based on how active they are, how experienced they consider themselves, and how other readers and NewsTrust staff rate them.

All of these rankings and reviews help maintain the site’s sense of focus and civility—but they can also make the site seem intimidating to a first-time visitor. Mary Hartney, NewsTrust Baltimore’s local editor and a former Baltimore Sun staffer, acknowledges that the reviewing method asks a lot of media-saturated readers.

There’s the element of: ‘I’m already signed up for a lot of social services, do I really want to sign up for another? What am I going to get out of it?’ Fabrice would tell you this: Reviewing a story well on NewsTrust—it takes work, it takes some effort, and I think that can turn some people off, for better or for worse.

This begins to explain the project’s modest success in getting readers to become reviewers. The site received 21,000 unique visitors during its six-month pilot. (By comparison, the Baltimore Sun’s website reported 4.8 million unique visitors for August.) Of those visitors, 535 became members. About 60 percent of these were people who were in some way affiliated with NewsTrust (students in classes collaborating with NewsTrust, media partners, NewsTrust staffers, etc.).

Of the sixty members featured for being active on the site, only about ten weren’t in some way affiliated with the project. One of NewsTrust Baltimore’s unanswered questions is whether participation would have grown if the project was around longer, and, if the numbers swelled, would the civility have survived?

Bruce Wallace is a freelance radio and print journalist. He lives in Brooklyn.