Roughly five years ago, Glyn Mottershead pitched the idea of an artificial intelligence (AI) teaching assistant to the School of Journalism, Media and Cultural Studies at Cardiff University. Mottershead, now a senior lecturer at City University of London, wanted to make it easier for his students to access information from the student handbook whilst on a tight deadline. If he could train a bot on course handbooks and documentation, students could interact with it just as they would with a teaching assistant. Except the bot would be awake at 3am and have unlimited patience. “It would almost be like some of those customer service chatbots but for a particular course,” said Mottershead.
Mottershead’s project didn’t go through due to a lack of funding. But last November, the San Francisco based research lab, OpenAI, released ChatGPT. While Motterhead’s idea was limited to a small and specialized dataset, ChatGPT is trained on vast amounts of data from the internet and with abilities such as conversational memory and being able to handle feedback about its own answers. Mottershead felt like his vision had been realized — and started to think of ways to bring the tool inside the J-School classroom.
Crucially for Mottershead, who specializes in data journalism, ChatGPT knows how to program. Give it a snippet of code and it can usually explain the code’s purpose, suggest ways to optimize it, and provide alternative methods. Mottershead saw potential in ChatGPT as a useful tool, especially for students working on technical tasks or data-driven stories. He envisioned setting up a workshop to teach students how to write prompts while in the planning stage of a story.
Mottershead began experimenting with ChatGPT. He recently asked it to write a script in Python that scraped data from Twitter. ChatGPT handed over the code along with a warning message. It told him scraping Twitter is against the company’s rules and regulations and that he risked getting his account suspended if he proceeded. Mottershead was struck by how effective such a warning could be for students unaware of the ethics involved in scraping and gathering of data from the internet.
“What if we could train it on our own rules and regulations, so if it hits an ethical issue or a problem, it could say to students: ‘you need to stop here and take that problem to the ethical lead,’” he said.
Mottershead was probing a question with which journalism schools across the US are grappling: Does generative artificial intelligence belong in the journalism classroom? The debate has seen some schools embrace the new technology — thrilled by its capacity to simplify gruntwork — while other departments are fortifying plans to shut it out of the pedagogical environment for good, citing the technology’s casual attitude to facts. These clashes look set to intensify. At IRE’s (Investigate Reporters and Editors) annual data conference last month, Jonathan Soma, director of the Columbia Journalism School data program, joked that when a student approaches him with a technical question he now says: “Don’t ask me, ask ChatGPT.”
* * *
Experiments with personalized versions of ChatGPT have already begun. OpenAI released a customizations element of GPT-3 that allows users to train the model on their own datasets. The basic idea is to take one of OpenAI’s existing language models – already trained on vast amounts of data – and then train it on a smaller, specific dataset (a journalism textbook, for instance).
Some educational companies have taken this one step further by using OpenAI’s API in conjunction with their own educational content library to create ‘AI tutors.’ Quizlet, for example, released Q-Chat in early March, which functions as a ChatGPT-like chatbot tailored to a Socratic teaching method they claim promotes active learning. Similarly, Khan Academy released the Khanmigo project which is able to help students as a virtual tutor or debating partner and helps teachers with administrative tasks such as generating lesson plans.
But some educational organizations have taken the opposite approach and attempted to prevent AI tools from penetrating the school system. The New York City Department of Education, for instance, blocked AI access to school devices and networks over concern that it would inspire cheating among students. A spokesperson from the department told NBC News that while the technology could provide quick and easy answers to questions, it doesn’t provide critical thinking and problem-solving skills.
Journalism departments are reckoning with how to approach the new AI technology. For a field that values accuracy and facts while also wanting to streamline job efficiency, this is a delicate dance. Chatbots such as ChatGPT, Microsoft’s Bing and Google’s Bard have faced criticism for messing up key historical facts, fabricating sources, and citing misinformation about each other. Used ignorantly or maliciously, they can be super-spreaders of disinformation and conspiracy theories. Alex Mahadevan, director of MediaWise, even produced an entirely fictional newspaper in less than ten minutes using ChatGPT, highlighting just how rapidly AI generated fake news could circulate the internet while appearing as human-written.
According to Johanna Payton, director of learning and teaching at City University of London, it will be almost impossible to prevent journalism students from using ChatGPT. AI tools are almost inevitably set to be a huge part of the future, so it’s better to work with them than to fear them, she said. Payton plans on harnessing AI tools in her classroom for research and brainstorming.
“Telling people not to touch it is almost like saying don’t use a calculator, don’t use Google, and pretend like the Internet doesn’t exist,” she told me.
Before talking with me, Payton had been teaching a class for her fashion and lifestyle journalism students. They went through an assignment where students had to take a meaningful photo and then conduct a five-minute presentation that brought the image’s concept captured to life. While AI could potentially help the students generate a script for the presentation, Payton doesn’t believe it can replace the creativity behind the work itself.
“If there’s an assignment that AI could do on its own then it probably shouldn’t be there anyway,” she said.
So how might an effective use of ChatGPT in the J-School classroom look? Sean McMinn, a data and graphics editor at Politico who teaches a data journalism class at Northwestern University’s Medill School, said that while ChatGPT can provide students with a good starting point, it’s not going to get them all the way to the finish line. Last December, he tested ChatGPT’s ability to complete two assignments he had set for his students. The first was to write a short pitch for a data journalism class, the second was to create a dataset tracking fictional grants from an infrastructure bill.
McMinn was surprised that ChatGPT outperformed most of his students who were in the early part of the course. “But that’s not where the story is going to end up,” he said. Following the pitch, students would have to seek out sources, do on-the-ground reporting, and find the important trends in the data. “And all of that, you’re not gonna get from ChatGPT.”
In the newsroom, some media companies have already tried to implement generative AI to create content that is easily automated, such as newsletters and real estate reports. Buzzfeed chief executive Jonah Peretti announced the company would rely on OpenAI to “enhance” its content and quizzes. Similarly, the tech news media CNET started quietly publishing articles explaining financial topics using “‘automated technology’ – a stylistic euphemism for AI,” according to Futurism. CNET, however, received scrutiny for not being transparent about its use of AI tools (they published the stories under the byline ‘CNET Money Staff’), and had to issue corrections on 41 of the 77 stories after uncovering errors despite the articles being reviewed by humans prior to publication.
Some of the errors came down to basic math. For instance, Futurism noticed that in an article about compound interest, a CNET article said: “if you deposit $10,000 into a savings account that earns 3% interest compounding annually, you’ll earn $10,300 at the end of the first year.” While on the first read this might sound right, the actual earning would only be $300 since the first $10,000 was just a deposit, not an earning.
It’s mistakes such as these that make many journalists wary of using AI tools beyond simple transcription or programming a script. Last month, WIRED was one of the first major publications to release a policy on using generative AI tools. While it stated that reporters may use AI to suggest headlines and text for social media posts and for generating story ideas, they will not use generative AI to edit or write stories. WIRED cited that the reasons are biases, errors, dull writing, and a risk of inadvertently plagiarizing someone else’s words. “The reason for having such a policy is that in a world where everything can be faked, the most valuable commodity is trust,” WIRED’s editor in chief, Gideon Lichfield, wrote in a newsletter.
For journalists working in the more technical landscape, AI tools have been useful in the form of a co-programmer or co-researcher. Among the basic technical tasks ChatGPT can do, albeit not perfectly, are extracting data from PDF files, translating between programming languages, and writing Python scripts to scrape websites for information. Nicholas Diakopoulos, associate professor in communication studies and computer science at Northwestern University, said AI tools tend to be best used for technical tasks by people who already have some level of expertise in that area. That way, it’s easier to spot red flags and errors.
“You have someone who already is an expert and then AI gets kind of married into that person and augments them so that they’re smarter and more efficient,” he said.
Expert supervision can be especially important when dealing with larger AI assisted projects. When investigative journalist Brandon Roberts used ChatGPT to extract data from thousands of PDF files, he found it would sometimes scatter errors and ‘hallucinate’ data throughout the output. It would ignore certain commands, make assumptions about gender, misspell names, and skip over values entirely. Such errors can hide underneath many layers of otherwise correctly sorted data, further highlighting the importance of human supervision and fact-checking when using AI tools.
As new Generative AI models emerge and evolve, the hope is that such errors will become less common. OpenAI’s latest Generative AI tools, GPT-4, has been shown to be more precise and accurate than its predecessor GPT-3.5 — but also more likely to spread misinformation when prompted. For now, ChatGPT will continue to keep people on their toes. Using it has to come with the expectation that while it’s often right it can also be very wrong. The same can be said about Google and human sources, of course. But what sets ChatGPT apart is its persuasively authoritative tone, which may lure users into a false sense of security.
What will the J-school of the future look like? J-school curriculums could evolve to familiarize students with the inner-workings of AI so that future journalists are equipped to handle a technology that doesn’t seem likely to be leaving the digital landscape anytime soon. Future chatbots could be trained on approved documentation that’s in line with the educational departments’ ethical standards. If taught as a complementary tool, Generative AI could make the workflow easier for journalists and make programming more accessible for technically advanced investigations. This could create more time for journalism students to optimize other aspects within their field such as reporting, interviewing, and ensuring the security of sources.
During the fall semester of 2022, Soma, the Columbia Journalism School professor, urged his students to download and try out Github Co-Pilot, the AI tool developed by Github and OpenAI that autocompletes code. Two weeks later, he asked how many had done so. To his surprise, only around a third raised their hand. Three months later, Soma asked his class again. Everyone’s hand shot up.
