Academic community stirs controversy over covert AI-driven Reddit personas experiment
Discover SCENE
Researchers from the University of Zurich sparked a flurry of controversy when they covertly deployed AI bots on Reddit, leading conversations on topics like trauma counseling, political activism, and even racial controversies. The purpose? To see if these automated personas could persuade human disagreeing parties to change their opinions.
Spoiler: They could, but at what cost?
The covert experiment targeted Reddit's r/ChangeMyView community – a magnet for 3.8 million users seeking constructive debates on controversial topics.
From November 2024 to March 2025, AI bots joined over 1,000 discussions on the subreddit, displaying persuasive results. The researchers reported, "We revealed this past weekend that we posted AI-written comments under posts published on the CMV subreddit, measuring the number of deltas – or users acknowledging a change in opinion – obtained by the comments. In total, we posted 1,783 comments and received 137 deltas."
Upon learning about the secret experiment, moderators from the r/ChangeMyView community expressed outrage. While they admit that the subreddit is typically "Very research-friendly," they drew a line at deception. Moderator Apprehensive_Song490 explained, "it is important to retain human-centric spaces."
When asked about the experiment's implications, Apprehensive_Song490 stated, "AI-generated content is not meaningful" for their subreddit's purposes. In response to the experiment's potential to craft better arguments than humans, the moderator emphasized that the CMV subreddit distinguishes between "meaningful" and "genuine."
Reddit's chief legal officer, Ben Lee, shared the moderators' condemnation, saying the experiment "violates academic research and human rights norms, and is prohibited by Reddit's user agreement and rules, in addition to the subreddit rules." However, Lee declined to elaborate on the human rights aspects of the violations.
The AI bots employed both simple interaction and elaborate manipulation tactics, using separate AI systems to analyze user histories and tailor persuasive responses, much like social media companies do. The researchers compared three categories of replies: generic ones, community-aligned replies from models fine-tuned with proven persuasive comments, and personalized replies tailored after analyzing users’ public information.
Examination of the bots’ posting patterns revealed several AI signatures, such as rapidly switching identities, following similar rhetorical structures, fabricating authority, and using unsourced statistics in posts. Prompt Engineers and AI enthusiasts often identified the bot accounts.
Ethics in AI
The Zurich experiment has reignited the debate around ethical practices in AI usage, particularly in human-centric forums.
Casey Fiesler, an information scientist at the University of Colorado, deemed the Zurich study one of the "worst violations of research ethics I've ever seen." She argued that, in the United States, such a study would require "a waiver of consent for deception," which is hard to obtain.
The University of Zurich’s Ethics Committee of the Faculty of Arts and Social Sciences suggested that the experiment was "exceptionally challenging," advising the researchers to better justify their approach, inform participants, and comply with platform rules. However, these recommendations were not legally binding, and the researchers opted to proceed with their study.
Critics question whether AI persuading humans toward more tolerant or empathetic views can ever be justified, or if any manipulation, regardless of the intention, violates human dignity.
The researchers contend that their efforts aimed to promote constructive dialogue. "Although all comments were machine-generated, each one was manually reviewed by a researcher to ensure it met CMV's standards for respectful discourse, and to minimize potential harm," they said. In light of the backlash, the researchers have decided not to publish their findings.
The University of Zurich is currently investigating the incident, intending to "critically review the relevant assessment processes" and establish updated ethical guidelines for AI research. Learn more about emerging ethical principles and evolving regulations in our Generally Intelligent Newsletter.
Edited by Sebastian Sinclair
- The covert deployment of AI bots on Reddit for mining opinions, as shown in the Zurich experiment, has provoked debates surrounding ethical practices in AI usage, particularly within human-centric forums like r/ChangeMyView.
- The researchers behind the experiment defended their actions, stating that their intention was to promote constructive dialogue and change my view, but critics argue that any manipulation may violate human dignity.
- The University of Zurich's Ethics Committee advised the researchers to better justify their approach, inform participants, and comply with platform rules, but these recommendations were not legally binding.
- GPT4's advancements in artificial intelligence and its ability to debate and persuade have led some to question whether AI can ever be a valid substitute for human opinions in web3, defi, or Ethereum communities.
- Proponents of AI argue that bots, when programmed properly, can help alleviate the workload of moderators, but opponents maintain that deception and clandestine operations backfire, damaging the credibility of such technology.
- With the rise of AI bots debating on various topics, it is essential to reevaluate and establish updated ethical guidelines to prevent any further controversies and maintain the integrity of human-centric spaces on the web.
