Prior to Elon Musk’s purchase of X, once called Twitter, the platform was a playground for academic research. Thanks to X’s free application programming interface, or API, thousands of papers were written based on its data. That all changed in February when Musk put the API behind a paywall. A new survey reveals that over 100 projects were canceled, halted, or pivoted to other platforms as a result of these changes. Despite hateful and inaccurate posts surging on X, researchers are now left with the fewest options for studying it.
The Coalition for Independent Technology Research, a group of researchers and journalists who work to defend the right to study the impact of technology on society, surveyed researchers affected by the new policies put in place by Musk. Of the 167 researchers surveyed, about half said they are increasingly worried about the potential legal repercussions of studying the platform. This concern comes after X’s July lawsuit against the Center for Countering Digital Hate, a nonprofit group that documented the rise of hate speech towards minorities on X since Musk acquired the platform. Several researchers surveyed called for efforts to regulate social media, emphasizing the importance of regulation that gives them access to social media data while being protected from litigation.
Tow talked to Josephine Luktio, Assistant Professor at the University of Texas and one of the researchers behind the study, about what they found and what the future holds for academic research on social media platforms.
This interview has been edited for length and clarity.
SG: Prior to Elon Musk’s takeover, Twitter’s API allowed researchers to gather millions of tweets per day for free. Now, researchers have to pay $100 per month for just 10,000 tweets. Why was the free API such an important tool in academia, and what did it allow researchers to study?
JL: For a lot of researchers, both academics and civil society researchers, Twitter was a really important platform to study because many politicians, journalists, and public figures were on Twitter. Researchers who wanted to understand those public figures needed to study Twitter. From that perspective, the API was essential to gather data and information about public conversations of ongoing current events and social topics. We relied on the Twitter API as well as Twitter’s relationship with researchers to provide that access.
Relative to other social media platforms, Twitter has historically been quite generous with its API access. The Twitter API became useful not only for doing research about important current events and what public figures were saying but also as an educational tool to show researchers how to responsibly use an API.
Hate speech spiked on X since Musk purchased the site last year. For instance, antisemitic content shot up by 61 percent just two weeks after the purchase. If researchers can’t study X via the API anymore, what are some other ways to keep track of hate speech on the platform?
It’s tricky because prior to the closure of the Twitter API, but after Elon Musk purchased it, there was really important research coming out about the increase in hate speech and trolling attacks. Not just for anti-Semitic content but for uses of the N-word and harassment of LGBTQ+ activists. I noticed that a lot of that research has been stilted in some way because of the Twitter API closure.
There are researchers who have been attempting to do it within Twitter’s new paid structure, but there’s no way to make that sort of research generalizable. It’s such a small number of posts and messages that you’re basically looking for a needle in a haystack.
I think moving forward, people will be trusting that research less and less. There are researchers who have considered alternative or non-traditional approaches to data collection, for example, data crawling or scraping. But because Elon Musk is so litigious, not only to these scraping strategies but also to anyone who speaks critically about Twitter, there is this chilling effect among researchers who are willing to actually call Elon Musk or X out for the increase in this sort of content.
That seems related to X’s July lawsuit against the Center for Countering Digital Hate?
Absolutely. Of the 167 responses to our survey about 50 percent, perhaps a little bit more, did mention that they were concerned about legal actions that they might be incurring as a result of continuing to study this platform.
Now that academics don’t have access to the free API, the survey found that researchers plan to pivot to other platforms. Reddit was by far the most popular platform of choice, followed by TikTok and YouTube. What makes these platforms attractive for researchers?
Similar to Twitter, these are platforms that have tried to engage with researchers in a variety of ways to provide access. Another reason why I think these platforms are popular to study is because they’re popular to use. Reddit, Facebook, Instagram, and TikTok are among the most popular platforms, at least among American adults eighteen and older. So it doesn’t surprise me that people are trying to turn from one popular platform to another popular platform.
The platforms that a lot of folks listed as transitioning into are platforms that have some form of API access. They don’t provide nearly as much data as Twitter does. But I think these are still really important platforms to study. That being said, some platforms have been really hesitant about providing a lot of data or will place limits on that data. For example, TikTok has very specific limitations if you want to use their researcher API. For instance, you need to run your projects by them. That has made a lot of researchers both eager to study the platform but also hesitant about what that relationship with the platform might look like.
So, will we see more studies on Reddit and TikTok?
I do think that there will be more research on other platforms. I don’t know if it’s ever going to be at the scale of Twitter data or Twitter research as we’ve seen it previously. We might be entering the stage where the golden era of social media research might be over.
Do you think the future of academic research on X is dead?
I’ll be upfront and say I’m kind of speculating. I can’t say with certainty what will happen. But I do think there are two potential Silver Linings that give me some hope about the future of social media research. The first is that more research methods rely on user consent. One growing area of research is data donation research, where you ask a user in a survey to also provide social media data. I think that sort of work is becoming more popular because it has the backing of user consent.
The other thing is that politicians are increasingly recognizing that independent social media research is really important. Particularly in Europe, we are seeing efforts to effectively require social media companies to make some sort of data available to researchers. I don’t know what that looks like tangibly, like how that’ll manifest for researchers and what that process looks like. But I am optimistic that politicians and public figures are starting to recognize the importance of doing this research because, certainly, social media platforms are not going to go away. They do a lot of great and terrible things for society. I think it’s really important for researchers to understand what those great and terrible things are.
More than 100 studies were canceled or suspended due to the changes to the API. What are some examples of projects that were shut down?
One big example that I was especially frustrated by was this great tool called Botometer. It was developed by a team at Indiana University and was specifically used to detect bots on Twitter. This is a tool that Elon Musk himself quoted when he was talking about the number of bots on Twitter, now X. That tool is effectively defunct now that they cannot access the Twitter API.
Musk’s restrictions on gathering data on Twitter have limited researchers’ ability to study real-time events like the spread of misinformation around the attack on Israel, the airstrikes in Gaza, and the upcoming 2024 election. What do you think are some of the implications on our understanding of these events?
I think the implication is that our understanding of these events will be much worse. Twitter provided us a window into understanding how public discourse around these events unfolds. Especially the importance of political elites and journalists in shaping the way people thought about these things. I imagine that that will still be important in the 2024 election, and it’ll still be important as current events unfold. We just won’t be able to study them or understand how they’ve changed.
Is there anything that surprised you about the findings from the survey?
It didn’t necessarily surprise me but I think it is really important to recognize that the closure of the Twitter API is something that doesn’t just impact academic researchers, but it absolutely impacts journalists. It impacts civil society researchers and things that we’re trying to do to understand the media ecology and the information environment and the potential harms within it. When we talk about the closure of the API, we tend to focus on individual projects or topics like mis- and disinformation and hate speech. But really, the reality is that everything has been shut. So even looking at things like expressions of joy and connectivity on Twitter, people can’t study that anymore either. So it’s not just different types of pockets of research. All research pertaining to Twitter has effectively ended.
