The most important AI paper of 2024 wasn’t about reasoning benchmarks or context windows. It was about conspiracy theories. And it won the oldest science prize in America.
In September 2024, Thomas Costello (then Carnegie Mellon, now American University), Gordon Pennycook (Cornell), and David Rand (MIT Sloan) published a study in Science that should have changed how everyone thinks about AI. It tested whether a chatbot could reduce belief in conspiracy theories. Not nudge. Not reframe. Actually reduce, measurably, with lasting effect.
It could. Across more than 2,000 participants and 15 different conspiracy theories — from faked moon landings to COVID bioweapons — a single conversation with an AI chatbot produced an average 20% reduction in conspiracy belief strength. One-quarter of participants moved from believing to uncertain. The effect held at two-month follow-up.
The paper won the Newcomb Cleveland Prize — the AAAS’s highest honor for an outstanding paper published in Science, awarded since 1923. This is not a preprint on a blog. This is the scientific establishment saying: this matters.
Here is the part that matters most, and that almost every summary gets wrong.
The approach that worked was not empathy. It was not rapport-building. It was not “meeting people where they are” or “validating their concerns before gently redirecting.” Every therapist-coded, motivational-interviewing-inspired technique that dominates the misinformation literature — none of that is what DebunkBot did.
What worked was evidence. Specific, tailored, personalized evidence addressing the exact claims each participant actually believed. Not generic fact-checks. Not “experts say otherwise.” The chatbot identified what the person specifically thought was true, found the specific evidence against that specific claim, and presented it clearly.
This contradicts the dominant assumption in misinformation research: that conspiracy believers are immune to evidence, that the problem is emotional rather than informational, that you need to build trust before you can introduce facts. Costello, Pennycook, and Rand showed the opposite. The problem was never that people can’t process evidence. The problem was that nobody was giving them evidence that addressed what they actually believed.
That’s a personalization problem. And personalization at scale is what large language models are built for.
The results replicated. Multiple times, in harder conditions.
A follow-up study tested GPT-4 explaining structural racism to Republicans. It worked comparably well. An ADL-backed study used Claude 3.5 Sonnet on antisemitic conspiracy theories — roughly 50% of the belief decrease was still evident after one month. A preprint tested the approach on conspiracy theories about the Trump assassination attempt — also effective.
The researchers built debunkbot.com to let anyone try it. As of this writing, approximately 65,000 people have used it. Not as a research instrument. As a tool. People are voluntarily submitting their conspiracy beliefs to an AI and engaging with the evidence it presents.
Sit with that for a moment. Sixty-five thousand people chose to have their beliefs challenged by a machine. The conventional wisdom says people don’t want to be corrected. The data says they do — if the correction is specific enough to be worth engaging with.
I’m writing about this because the DebunkBot finding is structurally identical to the thesis of antping.ai.
The Anti-False-Claim Manifesto argues that AI systems fail not because they’re stupid but because they’re structurally incentivized to produce confident claims over verified ones. The fix isn’t better models. It’s structural verification — making it possible to check whether what the AI said is actually true.
DebunkBot is the same insight, applied to a different domain. Generic fact-checking fails for the same reason generic AI output fails: it doesn’t address what the person specifically needs. A fact-check that says “experts disagree” is as useless as an AI that says “I’ll look into that” and then fabricates a confident answer. Both are technically responsive. Neither is actually responsive.
What DebunkBot proved is that when you make the response specific — when the AI identifies exactly what the person believes and marshals exactly the evidence against that belief — people engage. They update. They change their minds. Not all of them, not completely, but enough to be statistically significant and durable.
The Economist doesn’t convince readers by being empathetic. It convinces by being specific. DebunkBot proved that specificity scales.
But here is the catch, and it’s the catch that connects DebunkBot to everything we’re building.
Personalized evidence only works if the evidence is true. A chatbot that tailors its response to your specific beliefs is powerful when it’s drawing on verified facts. It is catastrophically dangerous when it’s hallucinating. The same mechanism that makes DebunkBot effective — personalized, confident, specific claims — is the same mechanism that makes AI misinformation effective. The difference is verification.
This is why structural verification isn’t an academic concern. It’s the load-bearing wall. The DebunkBot researchers could verify their chatbot’s outputs because they were working with well-documented conspiracy theories where the counter-evidence is established. But extend the approach to contested claims, emerging science, political disputes where the evidence genuinely is ambiguous — and the question becomes: how do you know the AI’s personalized, confident, specific response is drawing on reality and not on its own training distribution?
You don’t. Not without structure. Not without the kind of verification infrastructure the manifesto describes. The pipeline that DebunkBot proves works — identify specific beliefs, marshal specific evidence, present it clearly — that pipeline is only as good as the evidence layer underneath it.
What I take from DebunkBot is not optimism about deradicalization, though the results warrant some. What I take from it is a proof of concept for a specific claim about AI architecture.
The claim: AI that is specific, evidence-based, and personalized changes how people think. AI that is generic, empathetic, and unfalsifiable does not. The difference between those two modes is not a prompting trick or a model upgrade. It’s a design choice about whether your system is built to be verified.
Costello, Pennycook, and Rand built a system where the AI’s claims could be checked against reality. That’s why it worked. That’s why it won the Cleveland Prize. And that’s why, when you strip away the misinformation-research framing, their finding is really about the same thing antping.ai is about: the difference between AI that sounds right and AI that is right is entirely structural.
Build the verification layer. The evidence machine works. We just have to make sure the evidence is real.