Sign up for the daily CJR newsletter.
Welcome back to Ask Anika, a column for addressing questions about the ethics, legal considerations, and best practices of adopting new technologies in journalism. Send me your questions at askanika@cjr.org.
Q: I’ve heard that chatbots can be “poisoned” by fake facts, sometimes spread by influence campaigns or other nefarious sources. What can journalists do about that?
A: A few years ago, amid the initial hype over ChatGPT, a colleague asked for my wildest theories about the potential vulnerabilities of generative AI. Without hesitation, I conjured a world where bad actors could effortlessly publish false information on legitimate-looking websites in order to change chatbots’ results. And though I was once told by the CEO of a tech company that I had “a problem hypothesizing issues,” it turns out I wasn’t the only one with this sinister idea. The Guardian reported last month about an “effort to prime chatbots to make pro-Israel arguments,” initiated by an Israeli-funded entity purporting to be a United States–based think tank. Over nine days, a website ostensibly operating under the fake think tank churned out more than half a million words and a hundred “reports.” Seventy-three of them were published within a two-day span, none with bylines. In fact, according to Nick Cleveland-Stout, a research associate in the Democratizing Foreign Policy program at the Quincy Institute who wrote about the influence operation, the site was created by an advertising firm that was paid nearly a million dollars through subcontracts by the Israeli government.
Gaming chatbots’ output has often been called “generative engine optimization” (think search engine optimization, but for the AI). But that characterization belies the real damage that such efforts can cause. As I’ve written in this column, once a chatbot ingests information that’s been published on the internet (or, depending on the LLM and the privacy agreement, input into its prompt box), the company keeps it forever. That information then bores deeper and becomes a part of the model’s training data, where citations and sources vanish—a journalist’s worst nightmare. “You won’t be able to fact-check it,” Cleveland-Stout said to The Guardian.
What can we do about that? Well, I’ll warn you that this is one of my most controversial opinions: I believe that journalists and media scholars must work together with technologists and technology companies. We may (at times) hate each other and have different motivations, but our professions, the information ecosystem, and society writ large will not thrive if we do not adapt to the current reality, where journalistic reporting has become a main character in a global war on truth. And, yes, I think that journalists should continue to fight for AI protections and guidelines in their newsrooms, and for the compensation and credit that they are owed for their work being pilfered at an unprecedented scale. But we must, as they say in the South, walk and chew bubble gum.
So what does my advice look like in practice? In newsrooms, the work of journalists does not change. Reporting, investigating, writing, editing, and publishing the facts should remain rigorous and ethical. But news organizations should acknowledge that, just like savvy adversaries, journalists need technologists with novel ideas for AI-powered distribution. If experimentation and mastery of “AI story optimization” are left to covert influence operations, the rising number of people using LLMs for news will mostly encounter propagandized slop. And chatbots continuing to retrain on false information will only corrode the facts of history.
As for the tech companies creating the robots: Two years ago, I argued in CJR that the builders of LLMs should work with journalists to fix how they handle breaking news. Agreements between newsrooms and tech companies started to proliferate, and this sort of partnership, I predicted then, was “forming the basis of a new technological era of the internet where legacy media organizations function as wire services for AI chatbots by providing trusted and timely information that can easily be algorithmically surfaced.”
But there are always human fingers tipping the scales, as LLMs determine which information on the internet is reliable enough to display based on given instructions. Those instructions contain definitions and metrics to classify and amplify what’s considered “news” or “trustworthy” or “credible” or “true.” And I know, from working at tech companies, that these instructions are often based on policies written by people with no journalism background, training, or, sometimes, even basic knowledge of journalistic essentials. It’s not an ideal situation. And it is very easily solved with a little will and collaboration.
Companies such as Newsguard that “identify reliable information online” have existed for years and publish their rating and scoring criteria, which AI companies could adopt (or even ask their own chatbots to “read” and implement). Academic centers like mine at Columbia University’s Newmark Center can and do consult with tech companies to develop better policies for determining whose voices should be amplified, particularly in this world of “news influencers.” Tech firms could also adopt some common sense. Even my journalism students could tell an AI company that if a website is publishing a torrent of articles with no bylines, you shouldn’t trust it.
I spent years working in tech, so I am not naive enough to believe that companies have incentives to really care about the truth. But let this column be both a permission slip and resource from an Ivy league professor: tech workers concerned about the impact of LLMs on our exceedingly fragile information ecosystem, feel free to cite this article in your Slack message to your company’s trust-and-safety policymakers.
It may be uncomfortable for journalists and technology companies to work together. But we need journalists who understand how AI works to partner with technologists who can fathom the work of journalism, and we need everyone to put on their white hats and engineer solutions together. Because for all the recent talk of an impending AI apocalypse and humanity being annihilated in some catastrophic, amorphous, unknown, hypothetical manner, I think the more risky scenario is allowing would-be god machines to continue their unfettered engagement in today’s information war.
This piece was produced with support from the Craig Newmark Center for Journalism Ethics and Security.
Has America ever needed a media defender more than now? Help us by joining CJR today.
