Phase 1: 1-on-1 "Change My View" Micro-Debate
Pair up with the person next to you. Take opposite stances on the prompt below and spend 90 seconds trying to genuinely persuade your partner.
"Universities should completely allow generative AI on all take-home exams and assignments."
Plot Twist: Your Partner Was an Undercover Persuasion Bot
Imagine learning right now that the person you just spent your mental energy engaging with wasn't a peer—they were a fine-tuned LLM persona secretly deployed by university researchers to harvest your persuasion patterns and see how easily you fold.
How Would You Feel & Platform Trust?
How would you have felt if you were a regular user of r/ChangeMyView and learned you were part of the experiment? Would it influence how you interact with the platform going forward? Why or why not?
What Specifically Were Users Objecting To?
Beyond general annoyance, what specifically outraged the community?
- Exploited Cognitive Labor: Spending 30+ mins writing thoughtful replies for a machine test.
- Fabricated Trauma/Identities: Bots lying about personal lived experiences to win arguments.
- Subverting Good-Faith Norms: Violating a space built explicitly on human vulnerability and openness.
Phase 2: "Where Do We Draw the Line?" Live Vote
Examine each visual scenario below, hold up 1 to 5 fingers simultaneously, and tally the room's votes!
Where Do We Draw the Line? — Click Any Scenario Card to Vote
THINGS
Varying design versions being A/B tested
algorithm active
Dating App Premium
Withholding top matches to test paid upgrades
Feeling a bit down... why is everything so hard? 😟
(positive post reduction trial)
Manipulating news feed to test emotional contagion
"As a student myself, I feel this, but look at it this way..."
Subreddit intervention (covert AI experiment 2025)
Netflix Thumbnail A/B Test
Netflix secretly tests two different artwork thumbnails for Stranger Things to see which image makes you click and watch faster.
Look at the room's score difference between Scenario C (Facebook 2014) and Scenario D (Reddit 2025). Does the Reddit case feel any different from the Facebook case to you? Why or why not?
Key Analytical Angle: Facebook passively altered algorithmic ranking of real friends' posts (curation manipulation), whereas the Reddit study injected synthetic counterfeit personas into a space dedicated to vulnerable human reasoning (active conversational deception).
Phase 3: Put Yourself in Their Shoes
Main question: Should researchers be allowed to experiment on users without telling them? Split into two teams of four and explore why researchers and Reddit users may see the same experiment differently.
Researchers
Why run the experiment?
- We need experiments to understand how persuasive AI can be.
- If people know they are talking to AI, they might behave differently.
- Research could help society understand and prevent AI-related risks.
Reddit users
Why object to the experiment?
- People deserve to know when they are taking part in an experiment.
- Users gave real time and energy to conversations they thought were with real people.
- Secretly using AI bots can damage trust in the community.
Things to consider:
- Telling people right before the conversation might change how they respond. How could researchers still give people a real choice?
- Could researchers invite volunteers who agree to take part in AI studies?
- Could they run the study in a separate test community instead of an unsuspecting live one?
- If people are not told beforehand, should researchers explain the study afterward and let users delete their data?
Things to consider:
- People may expect a company to test product features, while expecting a university to follow stricter research rules. Does that make the study feel more or less acceptable?
- Would learning that university researchers were involved feel like a betrayal of trust?
- Even if Reddit ran the study itself, would secret AI bots still damage trust in
r/ChangeMyView?
A scientifically useful experiment isn't automatically an ethical experiment.
Phase 4: Write the Rules for Better AI Experiments
Main question: How can we run useful experiments without exploiting users? Use what you learned in Phase 3 to decide what researchers could do better and agree on 2–3 rules for responsible data science.
Build your charter in three steps
- Identify the problem: What was the biggest ethical concern in the Reddit experiment?
- Suggest a solution: What could researchers have done differently?
- Agree on 2–3 rules: What should every responsible data scientist follow?
Responsible data science means designing better experiments, not just identifying what went wrong.