This post used to be a placeholder while the paper was still in progress. Now that the paper is written up, I have turned it into a shorter companion: not a full reproduction of the experiments, just the main things worth taking away.

I. What Accuracy Hides

The starting point was a simple but uncomfortable observation. A model can trace variable assignments quite well, then become brittle once arithmetic is introduced; it can handle explicit for loops, yet struggle with semantically equivalent unrolled sequences. In our experiments, Value Tracking reaches 70.8% Resolved, while Loop-unrolled — which only requires sequential arithmetic — reaches 28.0%.

Final accuracy tends to flatten these differences. Two models with similar accuracy can still fail for very different internal reasons. So we shift the question: instead of only asking what information is encoded at a layer, we ask whether the model has already solved the problem by that layer.

The answer may already be present in the hidden state before it is organized into a form the model can use for generation. We therefore distinguish information availability — when the answer is externally readable — from information readiness — when it is usable by the model's own decoding pipeline.

II. The Dual Diagnostic Framework: Probing × CSD

We build a synthetic benchmark of six task families spanning data flow, control flow, and their combination: Value Tracking, Computing, Conditional, Function Call, Loop, and Loop-unrolled. Each model is evaluated on 24,300 samples, and every answer is a single digit so the target stays one token across tokenizers. We then use two layer-wise diagnostics side by side:

• Linear Probing: a logistic classifier trained per layer tests whether the answer is already linearly recoverable from the hidden state — this measures availability.
• Context-Stripped Decoding (CSD): inspired by Patchscopes (Ghandeharioun et al., 2024), we extract the last-token hidden state at layer ℓ and inject it into a target prompt that keeps only the question suffix — the entire code context is stripped away — letting the model continue its forward pass from layer ℓ+1. The key adaptation is baseline logit subtraction: a clean forward pass on the bare target prompt yields the language prior, which we subtract from the patched logits to isolate the signal carried by the hidden state itself. This measures readiness.

If CSD produces the correct answer after the code context is removed, the hidden state is self-contained: the model's own decoding pipeline can finish the job without looking back at the code. The agreement and disagreement between Probing and CSD across depth then gives us a cross-section of the internal reasoning process.

III. Brewing: Information Precedes Self-Decoding

A stable ordering emerges from the layer-wise scan. We define two landmark layers: the FPCL (First Probe-Correct Layer) and the FJC (First Joint-Correct Layer, where probing and CSD are simultaneously correct). In the anchor setting (Qwen2.5-Coder-7B):

• FPCL occurs at mean normalized depth 14% — the answer becomes linearly readable remarkably early;
• FJC arrives only at 50% depth — the model itself catches up far later;
• The interval between them is brewing: it averages 10.7 layers (38% of total depth), and the ordering FPCL < FJC holds across every experimental configuration.

Figure 1: The brewing-to-resolution lifecycle (Qwen2.5-Coder-7B, Computing). Each panel shows one sample trajectory: blue ridges are layer-wise Probing distributions, orange are CSD. Probing becomes correct first; CSD catches up several layers later. FPCL and FJC bound the brewing interval, after which trajectories diverge into (a) Resolved, (b) Overprocessed, (c) Misresolved, or (d) Unresolved.

Code reasoning is therefore not a last-layer event. The answer often appears early, but in a form that is not yet friendly to autoregressive generation. The model then spends a substantial part of its depth turning that answer into something it can actually output. Computing the answer and making it output-ready behave like separable jobs.

IV. Four Resolution Outcomes Cover All Trajectories

After brewing, trajectories do not all converge the same way. Based on whether FJC exists, whether the final output is correct, and the CSD confidence over the tail window (last quarter of layers), every trajectory falls into exactly one of four categories:

All four outcomes carry substantial mass. They are better understood as common branches of internal computation, not as rare edge cases.

V. The Taxonomy Is Causally Real

The taxonomy is only useful if it survives intervention. If FJC and the four outcomes reflect real internal structure, targeted edits should produce predictable, outcome-specific effects. We test this with three experiments:

• Activation patching at FJC: patching the source hidden state into a neutral template flips the answer at FJC in 20–44% of cases, versus only 3–18% at pre-brewing layers — FJC is a causally privileged point where information first becomes readable.
• Layer skipping for Overprocessed: naively replacing the final hidden state with the FJC state rescues only ~10% — but diagnosis reveals a representation norm mismatch. An alpha-blend injection (0.7·original + 0.3·FJC state) restores correctness in 47.8% of Overprocessed samples on average. The gap between injection modes is itself the proof: the FJC state carries the correct information but differs geometrically from late-layer representations.
• Re-injection for Unresolved: injecting the FPCL hidden state — where information first became readable — into the penultimate layers rescues 22–38% of samples, while Resolved controls remain at 84–100%. A substantial fraction of Unresolved samples are "not finished," not "fundamentally incapable."

Each intervention targets a different branch of the taxonomy, and each produces effects consistent with its branch and inconsistent with the others. Component attribution further localizes the rewrite to the network's late segment (layers 22–27 of 28): driven by attention in the Qwen family, but by the MLP in Llama/DeepSeek — the late-segment rewrite is stable across models, while its responsible component is family-specific.

VI. Every Code Primitive Has a Unique Failure Fingerprint

Figure 2: (a) Task→Outcome Sankey diagram (Qwen2.5-Coder-7B); ribbon width proportional to sample fraction. (b) Resolved% under increasing difficulty for each task. Three of six tasks fall below 30% Resolved.

The aggregate 41.5% Resolved rate masks dramatic variation across tasks, and controlled difficulty sweeps expose bottlenecks specific to each:

• Computing: the largest Overprocessed share (35.6%) — the model extracts the information but destabilizes mid-computation. Increasing arithmetic steps from 2 to 4 drives Overprocessed from 25.4% to 47.5%. Subtler still: pure addition yields a far higher Misresolved rate (17.5%) than mixed add-multiply (6.4%) — simpler operators produce more confident wrong answers.
• Function Call: the most severe degradation of any task. Call depth 1→3 collapses Resolved from 61.1% to 2.5% and drives Unresolved to 57.4%. Function-call indirection is the single largest bottleneck we observe, delaying FPCL by a factor of 2.4×.
• Conditional: solid overall (59.2% Resolved), but boolean_flag conditions yield 18.5% Misresolved (vs. 6.5% for numeric comparison) — Boolean evaluation is a latent bottleneck independent of nesting depth: the model confidently selects the wrong branch.
• Loop vs. Loop-unrolled: the cleanest causal contrast — identical arithmetic, differing only in syntax. Loop achieves higher Resolved (35.5% vs. 28.0%) and earlier information readability. Under dual-variable tracking, Unresolved doubles from 23.5% (Loop) to 53.6% (unrolled): loop syntax acts as a structural scaffold; once unrolled, the cues vanish and the same computation becomes harder. Trajectory-matched analysis shows the late MLP (layers 25–26) writes the correct digit into the residual stream more forcefully for loops — and transplanting that MLP write into the unrolled twin closes 82.6% of the readiness gap.

VII. Across 16 Models: Stable Scaffold, Divergent Capability

Figure 3: (a) Normalized FJC distributions (KDE) for 16 models, colored by family; (b) FJC existence rates; (c) normalized brewing duration. Despite variation across families, all 16 model-level means fall within a 24–42% band.

Extending the analysis to 16 models across Qwen2.5-Coder, Qwen2.5, Qwen3, DeepSeek-Coder, CodeLlama, and Llama-2 (0.5B–14B) reveals a clean dissociation:

• The mechanism is stable: brewing is already present at 0.5B, and normalized brewing duration falls within 24–42% for all 16 models. Coder and Base variants of the same architecture place their task-level FJC positions almost identically (Pearson r = 0.901; mean shift of only 0.67 layers).
• Capability diverges: scale shifts mass from Unresolved/Misresolved toward Resolved (0.5B: 18% Resolved → 14B: 50.3%); code-specialized training improves the Computing outcome mix; CodeLlama/Llama-2 collapse on Computing (7–8% Resolved) with the bottleneck in CSD capability rather than probing — while DeepSeek-Coder, on the same architecture, reaches 21.3%, pointing to code pretraining data quality, not architecture, as the differentiator.

Scaling does not create a new computational mechanism; it sharpens an existing one. Training changes how well the model resolves tasks (capability) far more than it changes where the resolution transition occurs (mechanism).

VIII. Watching Brewing Without Ground Truth

Everything above relies on ground-truth labels — a luxury deployment doesn't have. Fortunately, the lifecycle leaves observable signatures in the model's own output distributions. From the CSD digit softmax we derive two GT-free signals — entropy and max-confidence — and find that a sustained tail-layer entropy rise is the strongest single detector of Overprocessed samples (AUC 0.71–0.86): the signature of overprocessing is rising uncertainty, not merely falling confidence. A closed-form Resolution Functional built from endpoint statistics alone achieves a Resolved-vs-Rest AUC of 0.850.

This discriminative power is genuinely dual: remove either lens and refit, and four-class balanced accuracy collapses from 0.62 to about 0.40 — the strongest features (probe–CSD argmax agreement and divergence) simply cannot be computed from one lens alone.

IX. Why It Matters: No Single Early-Exit Policy Can Work

The two dominant failure modes make opposite demands:

Any single early-exit policy is therefore self-defeating: whatever helps Overprocessed hurts Unresolved, and vice versa. This motivates outcome-aware inference — and the 0.850 AUC of the GT-free signals suggests that monitoring which internal state the model currently occupies is a practical path, not a fantasy.

Experimental Motivation for Recursive Transformers

This connects directly to a direction the community is actively pursuing: architectures that refuse to lock the decoder into a fixed layer count — recursive/looped transformers that reuse weights and iterate on demand (e.g., LoopFormer), Mixture-of-Depths routing that allocates depth per token, and dynamic depth-scaling designs like the Inner Thinking Transformer. All of these share one premise: different inputs need different amounts of computational depth. Our taxonomy supplies sample-level experimental evidence for exactly that premise:

• Unresolved (23.7%) proves the fixed layer budget is genuinely too short for some inputs — re-injection rescues 22–38% of them, so additional computation demonstrably helps: these samples simply never finished;
• Overprocessed (26.4%) proves that surplus depth is not merely wasted but actively destructive — continuing the forward pass overwrites an already-formed correct answer.

A fixed-depth transformer sends every sample through the same layer budget. In our data, that budget is too long for roughly a quarter of samples and too short for another quarter. The key question is not simply whether to exit early, but which internal state the sample is currently in.

Going one step further, every adaptive-depth architecture faces the same core question: when should iteration stop? Our results identify what the halting criterion should track: not token-level confidence, but readiness — exit when the answer becomes self-decodable (an FJC-like event), keep iterating while still brewing, and brake before refinement flips into corruption. This is the observability layer that recursive and depth-adaptive architectures assume but do not themselves supply — and our GT-free signal system at AUC 0.850 is a working prototype of that layer.

There are also clear boundaries. Our diagnostics operate at layer granularity, so they do not yet identify the exact attention heads or MLP sublayers that drive the brewing-to-resolution transition. The benchmark uses short single-primitive programs, so the behavior still needs to be tested on more compositional multi-statement code. Two deeper questions also remain: why do late layers corrupt correct answers, and is availability-before-readiness a consequence of residual-stream geometry or a contingent property of current training?

Citation

@article{guo2026brewing,
  title  = "From Brewing to Resolution: Tracing the Internal
            Lifecycle of Code Reasoning in LLMs",
  author = "Guo, Yifu and Chen, Siyue and Lu, Yuquan and Lin, Jiaye
            and Xu, Zishan and Lin, Jianbo and Zhang, Siyu and
            Yang, Cheng and Li, Junxin and Li, Yujia and Huo, Yu
            and Wang, Ruixuan",
  year   = "2026"
}

Key References

1. Ghandeharioun, Asma, et al. "Patchscopes: A unifying framework for inspecting hidden representations of language models." ICML (2024).
2. Belinkov, Yonatan. "Probing classifiers: Promises, shortcomings, and advances." Computational Linguistics (2022).
3. Belrose, Nora, et al. "Eliciting latent predictions from transformers with the tuned lens." arXiv:2303.08112 (2023).
4. Halawi, Danny, et al. "Overthinking the truth: Understanding how language models process false demonstrations." ICLR (2024).
5. Meng, Kevin, et al. "Locating and editing factual associations in GPT." NeurIPS (2022).
6. Schuster, Tal, et al. "Confident adaptive language modeling." NeurIPS (2022).
7. Gu, Alex, et al. "CRUXEval: A benchmark for code reasoning, understanding and execution." ICML (2024).
8. Raposo, David, et al. "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models." arXiv:2404.02258 (2024).
9. Chen, Yilong, et al. "Inner Thinking Transformer: Leveraging dynamic depth scaling to foster adaptive internal thinking." ACL (2025).
10. Jeddi, Ahmadreza, et al. "LoopFormer: Elastic-depth looped transformers for latent reasoning via shortcut modulation." ICLR (2026).