Building an actively involved audience ranks high on the priority list for news organizations, but as outlets around the country experiment with different approaches to this goal, the question remains: how do news organizations gauge audience engagement? A recent survey from the J-Lab, which was answered by 278 “digital-first” news startups, found that most of the respondent sites lack the tools to measure whether they are turning their users into impactful supporters—the kind of visitors that contribute to the site’s long-term survival.

About eight out of 10 respondents said the conversion of users into “advertisers, donors, content contributors or volunteers,” is an important bit of information that remains difficult to quantify. Unique visitors and page views remains the top metric for measuring audiences, but respondents to the survey expressed their desire for better tools to assess how particular engagement strategies are working. “We feel those numbers only give us part of the information we need,” wrote one survey respondent. “We’re interested not just in breadth of engagement but more in depth of engagement.”

While “winning eyeballs” has remained the strategy of many news outlets, this approach, “prompts them to think of audience size as synonymous with audience engagement,” says the report. Comments and social media likes, retweets, and follows are also an important part of how news sites are increasing and assessing their reach, but this metric isn’t very telling when it comes to quality or impact of those interactions.

“Better tools can be built, online publishers need to be trained to use them, and site supporters — from funders to advertisers — should require better measure of engagement and impact,” reads the report, which wraps up with some recommendations for journalism schools and organizations, news sites, and funders to help encourage the development of innovative engagement-tracking tools.

“Audience engagement has just started to unfurl as an issue for news organizations,” concludes the report. A better understanding of how these interactions between news outlets and users evolve “would both increase the effectiveness of news organizations in the long haul and assist in identifying paths to financial sustainability.”

If you'd like to help CJR and win a chance at one of 10 free print subscriptions, take a brief survey for us here.

Alysia Santo is a former assistant editor at CJR.