Acceptance Prediction before Drafting

Where the signal sits in the round

The target checks round r−1 in one pass. As it reads, it builds a state for every token in the prefix. Take that state at the last kept token, the frontier. Take it at four depths: layers 6, 12, 18 and 24. Join them into one vector. That is 14,336 numbers for the Qwen pair and 16,384 for Llama. Three things matter here. It is the target's state, not the drafter's. It is free, because the check just built it. And it is in memory before the next round drafts.

1. Already paid for

   Round r−1 is checked

   The target reads the whole prefix in one pass. It builds a state for each token. At the last kept token, that state just sits there in memory. We call the spot the frontier.

2. Our signal

   Read the frontier state

   One dot product over 14k numbers. It takes about 16 µs. That is 0.015% of a round. No draft compute has been spent yet. This is where our signal lives.

3. All prior signals

   Round r drafts

   Now the drafter runs. Every control signal we found in prior work shows up here, and not before. That list covers draft entropy, acceptance heads, confidence schedulers and tree pruners. By then the cost is paid.

Main result: the cached state predicts next-round acceptance

Work on the dev split left one candidate alive: the raw frontier state. We then scored the frozen test split once per protocol. The spec was registered in advance. Nothing at all was selected on test. P1 runs the dev-frozen pipeline out of fold, inside the test rows.P2 is the strict dev-to-test transfer. The scaler, probe, control columns, fill-in means, domain list and calibrator are all fit on dev alone. Test rows are then scored once. P2 is the one that matches serving.

Every interval clears zero. P(Δ≤0) = 0.000 in every cell. The dev split shows the same pattern: 0.670 → 0.742 on Qwen-v1, 0.618 → 0.731 on Qwen-v2, and 0.696 → 0.765 on Llama-v1. The numbers come back in a fresh container to about ±0.001 AUROC. That spread is BLAS rounding noise. It is ten times smaller than the CI half-widths.

What this rules out. The baseline already holds domain one-hots, and captures are spread evenly by domain. So the gain is not the probe telling domains apart. The baseline also holds the target's entropy and margin from the round before. In theory those are the free variables that drive greedy acceptance. So the result says one thing plainly: entropy and margin are not enough. The cached state holds more that bears on the choice.

Methodology

The study is built to repeat cleanly and to leak nothing. An exact engine. Sealed captures. A bar we set high on purpose. And a rule that no number here is ever typed by hand.

Steering the direction disrupts real acceptance

A gain in what we can predict is not enough on its own. It cannot tell a load-bearing state from a shadow. So on held-out test rounds we steer the target's state at the frontier. We do it with forward hooks: h ← h + α·σ·d̂. Here d̂ is the acceptance direction fit on dev. Next, σ is the mean state norm, which sets the dose scale. The dose α runs over {−2, −1, 0, +1, +2}. Each layer gets controls: random directions at the same norm, and shuffled ones. A no-op check confirms the hook site (fidelity 0.95 on Qwen-v1, 1.00 on Llama).

Steering along d̂ hurts real first-token acceptance 2–10× more than the controls do. The damage tracks the dose. It traces an upside-down U that peaks at α=0. It holds at all four probed layers, in both families. On Llama the real hit is 0.32–0.40, while the controls sit near zero. The gap holds even in narrow bins of the entropy change it induces. So the effect does not run through entropy alone. We read this at the level of the state and no further. The probed direction drives whether the target's next token can agree. We make no claim about the inner workings.

Calibration: where the odds are honest, and where they aren't

Raw probes over many columns are far too sure of themselves. Before calibration the combined ECE runs as high as 0.14. One global Platt map fixes most of that. The map is monotone, so AUROC does not move, and it cannot leak. Under P1 the test ECE falls back to0.016–0.024. Under P2 we freeze the map on dev out-of-fold scores. We then apply it to test scores from a model fit on all of dev. That shift in spread costs us some calibration on one setting. The combined test ECE is 0.037 on Qwen-v1, against 0.024 and 0.021 on v2 and Llama. We report it rather than patch it. The ranking metrics do not change.

Where the signal stops: first token only

Survival probes by length show where the signal lives. At k=1 the gain is strong and clean in all three settings. By k≥4 it is null to harmful. It is clearly worse than the baseline on v2 and Llama at k=8. That is the point where domain and history already predict long runs. A length controller run off the survival probe also fails to beat one run off the baseline. The one exception is a small win at a single draft-cost point.

AUROC gain on the dev split, domain controlled, 2,000-resample grouped bootstrap. Green marks a clean gain. Red marks a clean loss. The frontier state answers one question: will speculation connect at all next round? It does not say for how long. That sets it apart from work that predicts draft length before the draft. Those papers find that treating length as a class label does worse. It also points at the skip choice as the natural user of the signal.

What it costs, and where it actually matters

Reading the frontier state is free on the capture path. The states fall out of the check anyway. The probe is one dot product over 14k numbers. That is about 16 µs per round, or0.015% of a round. The honest caveat is engineering, not compute.

The decision proxy. We sweep a calibrated skip-or-draft choice across draft-cost ratios. When drafting is cheap (0.2× verify or less) every calibrated model says "always draft". No pre-round signal changes a thing there. Adding a skip arm at those costs even hurts, by 0.3% to 0.7%. At a draft cost of 1× verify or more, the probe cuts regret in a way the CIs back. That holds across a range of costs on Qwen-v2 and Llama, under both protocols, and on Qwen-v1 under P1. Under P2, Qwen-v1 clears the bar at one cost point only. We report that sensitivity rather than hide it. So the signal changes real decisions where drafting is dear: launch-bound drafters, large drafters, packed batch serving. Elsewhere it is a diagnostic.

It adds to control inside the round. It does not replace it. For adaptivelength, a draft-entropy stop inside the round is still stronger. It beats the best fixed length by +6.7% to +11.2% in serving efficiency across the three settings. On Qwen-v1 that is +11.2%, 95% CI [+10.3%, +12.1%]. A pre-round gate on frontier entropy wins back only a third to a half of that. The two signals pair up by timing. One admits or skips a round, and schedules it. The other stops a round that has begun.

Acceptance itself has shape: the atlas

Here is a second question, and a purely behavioral one. No probes. No inner states. Does acceptance have a shape we can read? We pre-registered one contrast on dev. It was HIGH (code_delimiter, operator, number, pooled) minus LOW (named_entity, reasoning_transition, pooled). We then fixed criteria T1–T4 and the rule that maps each one to an outcome. We committed all of it before the test split was unblinded. A script did the scoring. PASS 3/3.

The contrast runs +0.12 to +0.21 in acceptance rate. And it is not domain identity in disguise. It is clean and positive in all 15 domain cells across the three settings. The tails hold across families too. In every setting, code_delimiter,operator, number, clause_boundary and punctuationsit above the marginal. In every setting, named_entity sits below it. The reading is a plain one. A small drafter is good at the tokens that syntax forces on the spot. It misses the tokens that call for what the target knows, or for a choice the target has to make. Our domain marginals recover the known domain axis (Qwen-v1 dev: summarization 0.657 < chat 0.700 < code 0.813 < math 0.875). We treat that axis as a control. The category structure holds once we condition on it.

The half that failed. The claim as first posed had a second axis. It said acceptance would vary across the phases of generation. It did not survive. On both Qwen settings the phase spread is flat, and flat by a wide margin: 0.0084 and 0.0036, against a bar of ≤0.03 set in advance. Llama does show a steady rise (prefix 0.794 → mid 0.824 → late 0.853). But that is one family on its own, and a length or mix confound is plausible. So we record it and claim nothing from it. The atlas result is narrowed to the category axis.

Two disclosures. First, a cell we had named in advance as free to be null came out positive. Llama-code was flat on dev, at −0.006, sitting on a 0.913 ceiling. On test it landed at +0.0674 [+0.0304, +0.1097]. So the dev null was ceiling noise. Second, the weakest cell that survives is Qwen-v1 summarization. It comes in at +0.0544, with P(Δ≤0) = 0.027 and a percentile CI lower bound of −0.002. That clears the p<0.05 bar we set in advance. We flag it as the margin to watch in any re-run. One more oddity we record but do not explain: on Llama the special category accepts at 0.007 (n=883). The likely cause is a gap in how chat templates and special tokens are handled, not an acceptance effect. We leave it out of the contrast.

Negative results that back the claim

Limitations

• First token only. We make no claim about run length. A controller that picks length gets no help from it.
• We use the whole frontier state. A cheap version you could ship is still open.
• One decoding mode: greedy exact match. We did not test acceptance under sampling.
• Two families, one drafter each. Trained heads of the EAGLE class are out of scope by design. Ours is the cell where the drafter stands alone.
• We picked layers 6, 12, 18 and 24 for the drafter depth, then reused them on the deeper targets. A sweep tuned to the target may make the result stronger. It may also pin it to fewer layers.
• The decision study is a proxy over sealed traces. No end-to-end timing test was run.
• The harness runs one request at a time. We did not measure how batches interfere.
• Under the strict protocol, calibration gets worse on one setting. We report that rather than patch it.
• We describe and log the bf16 tie-breaking drift. We do not remove it.
• The atlas leans on one taxonomy, v1.0.0, whose categories overlap. It covers two families and two corpora. Its phase half did not survive. It describes behavior only. We do not tie it to the probe evidence.