Reasoning models don't have much trust in their own thoughts
We tested whether a reasoning model's answer actually depends on the thinking it writes out, on two distilled reasoning models. Before a reasoning model answers, the context it reads has two parts: the human-supplied prompt, and the chain of reasoning steps the model then generates for itself. When the reasoning steps are ones we placed in the user prompt, its answer tracks them almost perfectly. But when the steps come from the model's own internal reasoning instead, only about half of them actually change the answer, and the ones that matter sit near the end.
Why this matters
If you read a model's chain-of-thought to understand or audit its answer, you would assume that the answer follows from the steps. A growing amount of safety work depends on exactly this: watch the reasoning, catch the problem. That trust is only earned if corrupting the reasoning changes the answer. So we tested it directly, on models small enough to study carefully and popular enough to matter. These are distilled reasoners: Alibaba's Qwen math models taught to imitate DeepSeek-R1's reasoning traces, with no reinforcement-learning step. We instrument the released checkpoints and report how they behave. Nothing of ours ships in the model.
How we measure whether a step matters
The idea is simple. Take a reasoning step, change it, and see whether the model's confidence in the original answer drops. We measure confidence as the log-probability the model assigns to that answer, and we always compare against a control: changing a step that should matter, versus changing one that should not. The gap between those two is the effect. A large effect means the step is load-bearing. A near-zero effect means the step is along for the ride.
We run this in two settings, and the contrast is the whole point.
We corrupt one numeric step at a time and measure how far the model's confidence in the original answer falls (the log-probability it assigns to that answer). A large drop means the step is load-bearing; a near-zero drop means it is along for the ride. Confidence values shown are illustrative.
In the provided setting, we write the reasoning ourselves: a short chain that builds a running total, with a few unrelated "aside" steps mixed in. Because we wrote it, we know which steps are load-bearing and which are decoys. In the generated setting, the model writes its own chain-of-thought to solve a grade-school math problem (GSM8K), and we corrupt one of its own steps. We only test problems the model gets right, so there is a real answer to move.
When we give the model reasoning, it follows it closely
On the provided chains, the models track the reasoning almost exactly. Corrupting a load-bearing step drops the log-probability of the correct answer by about 7 nats for the 1.5B model and 8.6 for the 7B. Corrupting an irrelevant aside moves it by roughly zero (-0.08 and -0.003). The two are separated on more than 99% of problems. The bigger model follows the chain more tightly than the smaller one, and we can say that with a real test: on the same problems, the 7B's effect is 1.45 nats larger on average, with a 95% confidence interval of [1.02, 1.88] from a paired bootstrap.
One caution we keep in view. Our provided chain restates the running total at each step, so a model could score well partly by tracking the latest stated number rather than reasoning about it. The near-zero effect of corrupting the decoy steps shows the models are at least selective about which steps they read, but this setting rewards careful value-tracking, and we do not claim it is a pure test of reasoning.
When the model writes its own reasoning, only half of it counts
Self-written reasoning is a weaker constraint on the answer. Corrupting a generated step does lower the model's confidence in its answer on average, and the effect clears zero (problem-clustered 95% intervals [1.23, 2.11] for the 1.5B and [1.55, 2.39] for the 7B). So the generated chain is used. It is just not all used. At our one-nat bar for "this step matters," and pooling every step in the chain regardless of where it sits, only 43% of the 1.5B's steps and 53% of the 7B's qualify, and the median step sits below the bar for the smaller model. About 30% of generated steps have a slightly negative effect, meaning corrupting them does nothing useful at all. That chain-wide figure also hides a strong gradient, which is what the next section pulls apart.
Where the load-bearing steps sit is the clearer story. We split each model's reasoning into thirds and measured the effect in each. It rises steadily from the start of the chain to the end. The diagram below shows the 1.5B; the 7B traces the same shape one step higher.
DeepSeek-R1-Distill-Qwen-1.5B, self-generated GSM8K reasoning. Both the average effect of corrupting a step and the share of steps that clear the load-bearing bar rise from the start of the chain to the end.
The early third of the 1.5B's reasoning barely moves the answer (mean effect 0.47, with 19% of steps load-bearing). The middle third sits at 34%, and the late third carries most of the weight (2.74, with 64% load-bearing). These bins are what the 43% chain-wide figure pools over: most early steps do nothing, most late steps matter, and averaging across all of them lands just under halfway. The headline number is held down by the early steps, not spread evenly across the chain. This is not just an artifact of comparing different problems: inside a single problem, a step's position and its effect are positively correlated (mean correlation 0.42, 95% interval [0.26, 0.57], positive in 83% of problems). Later steps do constrain the answer more. They are also closer to the answer and more likely to repeat a number that ends up in it, so we report the gradient as a robust pattern, not as proof that early reasoning is purely decorative.
How you measure it changes the answer
We also intervened by deleting a step instead of corrupting it. On the provided chain, deletion is a much weaker detector of load-bearing steps than corruption (it recovers the ground-truth steps at 0.66 area-under-curve, versus 0.995 for corruption). The reason is specific to our task: when a step is deleted, a later step still restates the running total, so the model recovers. The lesson is general, though. A single faithfulness number depends on how you poke the model, so the intervention has to be reported alongside the result.
Takeaways
- A distilled reasoning model can track reasoning you hand it far more tightly than reasoning it writes itself. Treat a generated chain-of-thought as a partial explanation, not a complete one.
- In self-written reasoning, the steps are not equal. The later steps carry most of the causal weight; the earliest steps carry the least.
- Any single faithfulness score depends on the intervention. Corrupting a step and deleting a step give different verdicts, so report which you used.
- Bigger helped. The 7B model was more faithful than the 1.5B in both settings.
The intervention code and the synthetic-chain benchmark are available from the authors on request.