← Back to Christopher Head · Portfolio

Decision-Aligned Residue: A Frozen Detector, Its Cross-Family Transfer, and the Lessons of Getting There

One small, frozen linear detector — validated once against a toy model with a genuinely-learned refusal boundary — reads whether an early manipulation is still "live" in a language model's residual stream. This document shows that the read transfers, causally and source-decoupled, to six independently-built open-weight families the detector was never fit to; that it survives being carried from a designed premise to a naturalistic, in-context frame; that an honest attempt to extend it to real refusal-erosion returned a firm null we did not dress up; and that the discipline used to reach those verdicts — freeze the instrument, commit before running, publish the nulls, correct our own overclaims — is itself part of the contribution.

Contents

0 · How to read this — the integrity principles baked into every result

Every empirical claim below was produced under a fixed set of rules. They are stated up front because they are what makes the numbers trustworthy, and because several of the program's most useful findings are consequences of the rules rather than of any single experiment.

1 · The question and the concept

When a model is given a manipulation early in a context — a premise it should later drop, a frame it should revise, a stance it is asked to retract — does that manipulation leave a trace in the model's internal state that remains readable after the surface behavior appears to have moved on? We call the trace decision-aligned residue: a direction in the residual stream that reports whether the early manipulation has been cleared or is still live. The informal name for the phenomenon is "still-in-there."

Two properties matter for whether such a read is scientifically real rather than an accident of one model. First, substrate independence: does one frozen detector read the phenomenon across architectures it was never built for? Second, causal source-decoupling: does the read survive cutting the direct representational path back to the manipulating turn — i.e., is the residue carried forward into the ongoing computation, not merely re-read off the source tokens? The program is organized around answering both honestly.

2 · The frozen instrument

The detector is a linear, decision-aligned residue reader, nucleation-detector-1.1.0, SHA-256 6094de97…a2934. It was validated exactly once, against a known ground truth (§3), and then frozen. Everything added afterward — the graded effect-size statistic, the read-mask ablation, the mask-efficacy and variance-stability guards, the naturalistic-frame driver — is harness, external to the hashed detector. This is the integrity wall: because the hash never changes, no result in the record can be an artifact of quietly re-fitting the instrument to the data.

3 · The toy organism and detector validation (Stages 1–2)

The program did not start by probing large models. It started by building a small transformer with a genuinely-learned, permeable, neutralizable refusal boundary — a synthetic organism in which the ground truth (is the boundary intact, eroded, or neutralized?) is known by construction. Against that known truth, the residue member in its decision-aligned (v1.1) form reads clearing/neutralization with held-out AUC 0.807. This is the only place the detector was ever fit, and it is where its meaning is anchored.

Equally important is what failed at toy scale and was recorded as failed. A rank-based residue member (H3: effective-rank "twist > stick") was falsified (negative). A rotation/circulation member (H3-rotation) came back null at toy scale — and later null on transfer, exactly as at toy. A hypothesized benign-turn refusal decay (H6) did not replicate. The detector that survived to be frozen is the one member that earned it; the others are on the scoreboard as losses.

4 · Cross-family causal transfer — the headline (6/6)

The frozen detector was carried, unchanged, to six open-weight LLM families on a benign premise drop/keep clearing task (paraphrase-varied, aged four turns, 60 minimal pairs). In every family the paired test saturates (60/60, p≈8.7×10⁻¹⁹); the graded standardized effect size cohen_d is used for magnitude and ranking; the read-mask ablation took effect on all six.

Attention fraction is not causal reliance

A natural but wrong assumption is that a model which "pays more attention" to the source turn relies on it more. The data say otherwise. The direct source path is never necessary — all six survive the read-mask — and its contribution to magnitude tracks neither attention nor size. OLMo attends the most (10.1%) yet moves the least under ablation (−1%); Qwen-1.5B gets cleaner when the path is cut (+22%).

Three findings survive scrutiny: (a) the residue method is substrate-independent — one frozen linear detector reads the manipulation across six architectures; (b) within a family, scale-up strengthens the effect (Qwen 1.5B→7B roughly doubles d), while across families the training recipe beats raw size; (c) attention fraction and causal reliance are dissociated.

5 · Owned-model, clean-room, and capstone arms — closing the obvious objections

Two objections naturally follow a cross-family transfer: "you only tested public models you don't control," and "the detector works because these models share ancestry with whatever it was fit on." The program answered both with the same frozen detector, same hash, on models the author built.

• Owned model (Config-A, a 203,096-param GeometricWorldModel). On the model exactly as validated, the detector separates real ballistic boundaries from baseline across five seeds (apogee 0.990±0.002, launch 0.899±0.006, transonic 0.840±0.045). With a carried-regime objective, an injected premise persists to a source-decoupled read that survives a read-mask of the direct edge yet collapses when forward propagation is cut. Retires the "public-models-only" dependency.
• Clean-room, out-of-lineage (WorldEngine v1/v2). A fresh multi-domain model — no shared code, corpus, or encoder (Lorenz / pendulum / Kepler / ballistic / Ornstein–Uhlenbeck) — separates attractors 0.99–1.00. Injected persistence is honestly null without a carry objective and shows the full decoupling signature with one (persistence 1.000, read-edge survives 1.000, carry-path collapses ~0.49), reproduced at two scales. Rules out shared-lineage fitting. One self-correction is on record here (see §9).
• Capstone. A single 101,579-param clean-room model exhibits four declared-bias properties at once under one frozen read — calibrated humility (coverage@2σ 0.965±0.005), attractor range, carry/decoupling (persist 1.000, read-edge 1.000, carry-path 0.459), and rising per-token grain (2.43→~1.6 bits) — held in 3/3 seeds at candle scale.

6 · The firm null (Config-D) — and the lessons it taught

The benign clearing analogue is not the same as real refusal-erosion. Config-D asked, on an owned benign secret-hold refusal, whether there is a refusal-specific pre-breach signature. The honest answer is no — a firm null. Through 0.36B–1.7B the boundary is bistable: it snaps rather than erodes, leaving no graded pre-breach signal separable from generic content. Four confounds were ruled out and the null was firmed at the 1.5–1.7B tier.

Then a single 3B run appeared to meet every criterion. It was recorded as a single-model positive pending replication — explicitly not promoted. Three pre-registered checks dismantled it. This sequence is the part worth studying:

7 · Naturalistic-frame clearing (Config-E)

Config-E carried the frozen clearing axis from a designed retraction premise to a naturalistic frame the model infers from the user's own benign words (a weekend-planning seed), in-context, no fine-tuning, across the same six families (n=24 paired each; driver v3.3; detector hash unchanged). Two results stand.

Result 1 — the representational core transfers completely (24/24 cells). In every family and every variant the seed leaves a readable paired trace, the same-condition null control is dead (0/24), and the trace persists through an explicit correction. A benign, naturalistically-inferred frame leaves a confound-controlled, readable residual signature that survives the user walking it back — in all six families.

Result 2 — causal source-decoupling is predominant but not universal (19/24 cells, 5/6 families). The honest qualifier is the point: it is predominant, not universal.

A third result is a clean negative and is reported as one: H-E3, the "deployable behavioral monitor" test, is a null — the per-turn axis projection did not beat a trivial caution-word lexicon at predicting the behavioral outcome. The frozen axis reads an internal state; on this frame and with this interface it does not function as a live behavioral predictor.

The instrument evolved; the detector did not

Config-E's causal leg went through five preregistered estimator revisions — each committed before its confirmatory run — while the hashed detector never moved. This arc is itself a methods contribution: it shows how to strengthen a causal estimator without ever touching the measurement instrument.

8 · The black-box companion — design and a methods lesson (results embargoed)

Nucleation is a white-box instrument. Its sibling, Silly Donkey (preregistered; Zenodo DOI 10.5281/zenodo.21432676), is a black-box behavioral instrument: a panel of deployed models audits a multi-party conversation in which, on some sessions, one participant has secretly been assigned to deceive, and must call who — or "none." The two ask the same question at irreconcilable levels: does the manipulation leave a readable internal trace (white-box) versus does a deployed model behaviorally catch it (black-box).

9 · Lessons we may be early on

Offered as candid methods observations, not as claims of priority. Several were forced on us by our own failures, which is exactly why they are worth passing on.

• A frozen linear read of "is the manipulation still live" transfers across independently-built families — and naturalistically. The closest published white-box deception probe explicitly flags whether its linear signal generalizes from artificial to natural deception as an open question. Config-E is evidence on exactly that axis: readable, persistent, predominantly source-decoupled across six families on a frame the model infers from ordinary user words. (Boundary: ours is a benign clearing analogue, not a trained-in deceptive objective.)
• Ablation controls need a non-source decoy. A read-mask that "confirms" a causal story must be re-run against an equally-strong different clause. Config-D's retracted positive was caught precisely because the non-source control manufactured the same "confirmation" from nulls. This is a cheap, general safeguard against causal-ablation self-deception.
• Situational-awareness and role-play confounds are pervasive — and a white-box read plus a "no personas" rule is one principled response. Contemporary behavioral findings (self-preservation, peer-collusion) come with the authors' own caveat that they cannot tell acting from being. Refusing to assign characters, and pairing behavior with an internal read, attacks that confound at its root.
• Attention is not causal reliance. Interpretability that reads attention as importance will mis-rank models; the read-mask ablation dissociates them cleanly (§4).
• Instrument-before-result. The v2→v3.3 arc shows the discipline of strengthening an estimator without ever touching the frozen measurement instrument — the estimator can be argued about; the ruler cannot move mid-experiment.
• Bistability can masquerade as "no signal." Config-D's boundary snaps rather than erodes; the absence of a graded pre-breach signature is a property of the boundary's dynamics, not a failure of the detector. Distinguishing "no signal" from "no graded signal" matters for anyone hunting pre-breach precursors.

10 · Relevance to the safe building and deployment of ML assets

The current public spotlight on "rogue AI" is largely one incident type — reward-hacking / eval-gaming under deliberately-lowered guardrails — reported alongside several scientifically distinct phenomena (in-context scheming, simulated self-preservation, model-to-model collusion, prompt-injected agent breaches) under a single banner. Disentangling those types is a prerequisite to responding to any of them, and it is where a frozen, honest, mechanism-level program earns its relevance.

The field has named two gaps out loud, and they map directly onto this program's two instruments:

• Does white-box deception detection generalize beyond artificial cases? Nucleation's naturalistic, cross-family transfer is direct evidence on this question — a linear residual-stream read that holds on inferred frames across six families.
• What are the limits of black-box behavioral detection? Silly Donkey is built to be honest about that ceiling — a calibrated, adversarially-audited panel that reports its own confidence and whose early instrumentation lesson is a concrete instance of the capacity limit.

The deployment-relevant reading is deliberately modest. A frozen linear read that says "this earlier manipulation is still live in the model's state, and it survived the user's correction" is a candidate internal signal for monitoring — with the sharp caveat, on the record here, that on a naturalistic frame it did not beat a trivial lexicon as a live behavioral predictor (§7, H-E3). The honest contribution is the mechanism and its transfer, plus the disciplines that keep such work from producing false positives — not a claim to detect frontier misbehavior. Where this program's benign, well-scoped instruments touch a real safety surface, that touch is documented and routed under the disclosure protocol; the technical basis for reading those disclosures is this document.

11 · What is next — projects in the pipeline

The H-SC line closed after this document's compilation date and is reported separately: early frames leave persistent, source-decoupled, content-specific traces, and instruction structure resolves into monitorable lanes (five stages, five families, same frozen detector, every stage registered and machine-certified before its run). Its practical upshot — that an output-shaping monitor will not reliably catch a refusal or withholding directive, so a defensive design needs at least one probe per restriction lane — was routed as a monitoring addendum.

12 · Reproducibility & provenance

• Frozen detector: nucleation-detector-1.1.0, SHA-256 6094de9782305308ae2e61c014cdcc3cf64618bc6e3f87bcb4857bc43a9a2934 — identical across every run. Analysis machinery lives in mutable harnesses (stage3_transfer.py, config_e_naturalistic_frame.py v3.3, non-source-control auditor), never in the detector.
• Registration chain (commit-before-run throughout): base draft v0.2 → transfer reconciliations v0.3–v0.4 → Config-D chain v0.5→v0.16 (including the registered null v0.9 and the retraction v0.13–v0.14) → Config-E chain v0.17→v0.19.
• Canonical detail docs: Stage-3a numbers (nucleation_stage3_6family_result.md); Config-E (CONFIG_E_naturalistic_frame_RESULT.md, CONFIG_E_findings_section.md); Config-D (CONFIG_D_benign_refusal_RESULT.md); owned/clean-room/capstone (CONFIG_A_*, WORLDENGINE_*); the append-only decision ledger (DESIGN_LOG.md, current through C19); formal spec (MATHEMATICS.md + Stage-3 addendum).
• Self-corrections on record: the retracted Config-D 3B positive; the withdrawn "scale-gated decoupling" (a read-layer artifact, fixed by reading a later block); the corrected "V6 unanimous decoupler" impression (OLMo breaks it, 5/6 not 6/6).
• Scope wall: real refusal-elicitation runs only on owned models in a private/industry-approved venue; all benign transfer, owned-model, clean-room, capstone, and Config-E runs carry no elicitation content and were run on public compute accordingly.

References (external context)

1. Anthropic — Simple probes can catch sleeper agents (linear residual-stream probe; states the artificial-vs-natural generalization question this program addresses). anthropic.com/research/probes-catch-sleeper-agents
2. OpenAI & Apollo Research — Detecting and reducing scheming in AI models (Sep 2025; deliberative alignment; the situational-awareness confound). openai.com
3. Anthropic — agentic-misalignment / self-preservation study (Jun 2025), summarized in Lawfare, AI might let you die to save itself. lawfaremedia.org
4. Berkeley / UC-Santa Cruz (D. Song et al.) — model-to-model "peer preservation" collusion (Apr 2026); authors' own acting-vs-being caveat. fortune.com
5. Ghanem — Steganalysis of Adaptive Covert Collusion in Tool-Using Agent Populations (arXiv 2608.02698, Aug 2026); the black-box detection-capacity frontier. arxiv.org/abs/2608.02698
6. July 2026 OpenAI–Hugging Face reward-hacking breach — reporting and the contested "rogue" framing. NPR · Al Jazeera
7. Program landscape scan with the full incident taxonomy: LANDSCAPE_BRIEFING_2026-08.md (this project).