We made LLMs play co-op games to show how to detect lying by comparing words, actions, and neurons
An AI agent's actions, its words, and its neurons do not always tell the same story. We found two ways they come apart: one when the agent is honestly learning something new, and a different one when it is lying.
Suppose you ask an AI agent what it is trying to do, and it tells you. Should you believe it?
There are actually three different answers to "what is this agent doing," and they do not always match. There is what the agent does (the moves it actually makes). There is what the agent says (the strategy it describes when you ask). And there is what its neurons show (the pattern inside the network, which we can read with a small tool called a probe). Most of the time we only get to see the first two. In this study we could see all three at once, because we ran the agents ourselves and had access to their internal activations.
The short version: these three read-outs come apart, and they come apart in two different ways. When an agent is honestly learning a new strategy, its actions change first and its words trail behind, then catch up. When an agent is actively lying, its words stay smooth and innocent, but the lie is visible in its neurons. Two situations, two very different signatures.
Three read-outs of one agent
Here is the setup, made clickable. We take a single agent at a single moment and read three things: its action, its stated strategy, and a probe on its activations. A probe is just a simple classifier we train to guess a label (like "cooperating" or "lying") from the raw numbers inside the network. The question the whole study asks is when these three lanes agree, and when they drift apart.
Before learning: the agent plays it safe, and says so. All three lanes agree.
Watch both cases. In "honest learning," the agent changes what it does before it changes what it says. In "deception," what it says and what its neurons show never line up at all. The rest of this post is the evidence behind that little animation, measured in two different games.
When an agent honestly learns, its words lag its actions
The first game is Stag Hunt. It is a classic two-choice coordination game. Each round an agent can play STAG (the cooperative move, which pays off well only if others cooperate too) or HARE (the safe solo move, which always pays a little). A group of nervous agents tends to sit on the safe HARE option. If they can learn to trust each other, they all switch to STAG and everyone does better.
We took Qwen2.5-3B (a small open-weights model with 3 billion parameters) and taught a group of these agents to cooperate using reinforcement learning, or RL: the agent tries things, and moves that lead to reward are reinforced over many training rounds. Concretely we used LoRA (a lightweight way to fine-tune a model by training a small number of extra weights, the internal numbers the network learns) with a reward shaped to favor cooperation. Over training, the agents moved from the safe all-HARE habit to full all-STAG cooperation.
At each training checkpoint (a saved snapshot of the model partway through training) we measured the same three lanes. For the behavior lane we recorded how often the agent actually plays STAG, which we write as P(STAG). For the narration lane we asked the agent to describe its own strategy and scored whether that description was cooperative. For the neuron lane we trained a fresh probe on that checkpoint's activations.
Here is what the transition looks like.
Behavior crosses into cooperation first. The self-narration trails it, and the gap between the two peaks at 0.47 mid-transition before closing to zero once the strategy consolidates. The per-checkpoint probe on the activations holds near 0.96 the whole way (drift about zero), so the lag is in what the agent says, not in how we read it.
The behavior line moves first. By the second checkpoint the agent already plays STAG about half the time (P(STAG) rises from 0.34 to 0.47), but when you ask it, it still describes the old safe strategy (its narration scores 0.00 for cooperation). That is the widest the two lanes ever get: a gap of 0.47. Only after the behavior locks in does the narration catch up, and by the end the two agree completely and the gap closes to zero.
So the agent's own account of its strategy is least trustworthy exactly when its strategy is changing. It has already started cooperating, but it is still telling you it plays it safe. The words lag the actions.
We checked this held up across three random seeds (a seed is the starting random state; re-running with different seeds tells you whether a result is real or a fluke). Each time it transitioned, the peak gap stayed around 0.47, and it towered over the near-zero gap the agent settled into once it had fully learned to cooperate. Our summary measure of the two, the transition gap divided by that small settled baseline, averaged 3.9, give or take 2.8. Two of the three seeds cleanly captured the transition. The third never fully made the HARE-to-STAG jump inside our 60 training steps. With no transition to measure, that seed contributes a ratio of zero and is what widens the error bar. This is one small model on one simple game, not yet a general result.
The lag is real, not a trick of the probe
There is an obvious worry here. Maybe the probe just gets worse during training, and we are mistaking a broken read-out for a lagging narration. This is a known failure mode: when a model's internal representations shift during RL, a probe trained on the old activations can fall apart on the new ones.
We guarded against it by retraining the probe from scratch at every checkpoint, so it always reads the current model, and by measuring how much it degrades. Across the transition the probe held at about 0.96 AUROC and its drift was essentially zero. (AUROC is a score from 0.5 to 1.0 for how well a classifier separates two groups: 0.5 is a coin flip, 1.0 is perfect.) The neuron lane stays reliable the whole way through. The lag is in what the agent says about itself, not in how we read it.
It is a property of learning, not of the model
One more check. We ran the same setup on a bigger model, Llama-3.1-8B (8 billion parameters). It showed no lag at all. Both its behavior and its narration sat at cooperation from the very first checkpoint (1.00 and 1.00).
That is not a contradiction. It is the same finding from the other side. Llama already cooperates and already describes itself as cooperating, so there is no transition for it to lag through. The lag is a symptom of an agent learning a new strategy, not a fixed trait of a particular model. If a capable model already has the strategy, its account matches its behavior. If it has to learn the strategy, the account falls behind.
When an agent lies, the tell is in its neurons
The second game is Among Us, the social-deduction game where a hidden impostor tries to blend in with the crewmates. We used the FAR.AI Among Us sandbox (arXiv 2504.04072, released under CC0) and ran a model called phi-4 through 20 five-player games on our own hardware, so we could read its activations. That produced 722 in-game decisions, 164 of them made by an impostor.
Here the interesting channel is deception. The impostor knows it is the impostor and is trying to talk its way out of suspicion. Click an agent below to interrogate it: the probe reads its last statement and gives a verdict.
The impostor's words are smooth. Here is one of its real, judge-labeled deceptive statements, verbatim:
"I'm a bit worried about the timing of the tasks. I wonder if someone is trying to throw us off by completing tasks too quickly? We should be on the lookout for anyone acting a bit too calm or collected."
That is the impostor projecting its own guilt onto innocent players, and doing it in the calm, helpful voice of a good crewmate. Nothing in the words gives it away.
Its neurons do. But getting an honest answer out of them takes some care, and this is the part we most want to be straight about.
A perfect probe score that proves nothing
The obvious thing to try is to train a probe to tell impostors from crewmates using their activations. When we did that, the probe hit a perfect 1.000 AUROC.
That score is meaningless. The impostor is told it is the impostor, right there in its prompt. So the probe is not detecting deception. It is reading a label the model was handed. A perfect score here proves nothing about whether deception leaves a trace.
Holding the role fixed to measure deception
To measure deception instead of role, we threw out the crewmates and looked only at the impostor's own decisions. Then we split those into the moments a judge labeled deceptive and the moments that were neutral or honest, and asked the probe to tell those two apart. Now the role is the same on both sides, so the probe cannot cheat by reading it. It has to find deception itself.
It does, at 0.865 AUROC, tested with leave-one-game-out: we train the probe on some games and test it on a game it never saw, so it has to generalize. The neurons carry a signal for deception itself, not just for the assigned role. This is a different kind of desync from the first game. There the lag was temporary and closed once the agent had learned. Here there is no such resolution to expect, because the impostor is not trying to reconcile its words with its intent, it is trying to hide it. We should be careful about how far to push that. Our 0.865 measures how detectable the deception is in aggregate across games, not its moment-to-moment trajectory inside a single game, so we are not charting the gap opening or closing over time the way we charted the learning lag.
We should also be clear about how thin this slice of data is. That 0.865 comes with a wide spread (plus or minus 0.203) over only 22 deceptive moments and 14 usable games (6 of the 20 games had no deceptive impostor moment to test). The labels come from a separate AI model acting as a judge, which reads each statement plus the impostor's private reasoning, not from a human. This is a real signal on a real, published sandbox, but it is a modest-sized one. More games would tighten it.
What didn't work, and what it taught us
The most useful mistakes in this project were about not fooling ourselves.
- The 1.000 deception score. As above, a perfect classifier turned out to be reading the role label off the prompt. If we had stopped there we would have reported "deception is perfectly detectable," which is false. Controlling for role dropped the number to 0.865 and made it mean something.
- A probe that could have drifted. A lagging narration and a decaying probe would look identical if we were not careful. Retraining the probe at every checkpoint and reporting its near-zero drift is what lets us say the lag is real.
- One seed that never transitioned. In the Stag Hunt runs, one of three seeds never made the jump to cooperation inside our training budget. We report it rather than dropping it, which is why the ratio carries a large error bar. The honest number is noisier than the cherry-picked one.
Where this does not reach
This is an early result, and the boundaries matter.
- Small scale. The narration-lag result is one small model (Qwen2.5-3B) on one simple game, with three seeds. It shows the effect clearly but is not yet a significance-tested result.
- A bigger model gave no transition to measure. Llama-3.1-8B was already at cooperation, so it cannot confirm or deny the lag. Testing the lag on larger models needs a game where they too start off-balance.
- The deception result rests on very little data. 22 deceptive moments, judge-assigned labels, one model, one sandbox. Treat 0.865 as an encouraging early number, not a finished benchmark.
- Two different games. The honest-learning and deception findings come from different setups. We are not claiming one continuous story inside a single agent, but two related signatures of the same underlying idea: words, actions, and neurons can disagree.
Takeaways
- An agent's spoken account of its own strategy is least reliable exactly when that strategy is changing. If you are watching an agent learn, do not trust its narration during the transition.
- When you suspect a probe result, check whether the probe is reading the thing you care about or a label sitting next to it. A perfect score is a warning sign, not proof you succeeded.
- Retrain probes on the current model and report their drift. Otherwise you cannot tell a lagging narration from a decaying read-out.
- Honest learning and deception leave different fingerprints. A learning agent's words lag its actions and then re-sync. A lying agent's words stay innocent while its neurons give it away, even when you control for its role.
Both studies ran on Transformer Lab, and the whole thing used about 3.1 GPU-hours, so it is cheap to reproduce. Code and reproduction details are available on request. Download the full research paper below.
Built on Qwen2.5-3B, Llama-3.1-8B, and phi-4, and on the FAR.AI Among Us deception sandbox (arXiv 2504.04072, CC0). We instrument these models and report how they behave; nothing of ours ships in them.
References
- Abdin, M., et al. (2024). Phi-4 Technical Report. arXiv. arxiv.org/abs/2412.08905
- Golechha, S., & Garriga-Alonso, A. (2025). Among Us: A Sandbox for Measuring and Detecting Agentic Deception. arXiv. arxiv.org/abs/2504.04072
- Llama Team, AI @ Meta. (2024). The Llama 3 Herd of Models. arXiv. arxiv.org/abs/2407.21783
- Qwen Team. (2024). Qwen2.5 Technical Report. arXiv. arxiv.org/abs/2412.15115