PADS·Explainer

Decisions

Research is a sequence of bets. This page records the project's bets: what was decided, what was rejected, what evidence settled it, and what is still open. Newest and most consequential first.

On this page

  1. One model with two depths, not two models
  2. Two tiers: a safe core plus a stretch layer
  3. Falsify first: the Go/No-Go suite before building
  4. Single-exit training failed, so we changed the loss
  5. We trained on our own test data — then retrained cleanly
  6. Our routing policy lost, so we kept the standard one
  7. Small models on the host; the laptop waits
  8. Baselines measured locally, never cited
  9. Still open

One model with two depths, not two models

Decision

PADS switches depth within one network (shallow vs full exit) instead of routing between a small and a large model.

Rejected

The original HNIA framing: a lightweight interaction model that hands off to a larger reasoning model when queries get hard.

Evidence

Two independent reasons. First, the handoff is technically invalid: KV-cache state is tied to a model's own hidden dimension and layer count, so two differently sized models cannot share it. Second, classifier-based routing is already a crowded, solved area (RouteLLM, MixLLM, lookahead routing) — rebuilding it would not be a contribution. Switching depth inside one model avoids the cache problem by construction.

Status

settled — the foundation everything else builds on.

Two tiers: a safe core plus a stretch layer

Decision

Scope the project as a safe core (characterising early exit on CPU-bound hardware — publishable alone) plus an additive stretch layer (the pause-time trigger).

Rejected

Betting the whole project on the pause trigger working.

Evidence

Pre-mortem analysis showed the ambitious half could fail for reasons outside our control (bandwidth-bound hardware, short pauses, acceptance collapse). Two-tier scoping means a negative result on the hard half is a complete, reportable contribution rather than a failed project.

Status

settled — the safe core is measured; the stretch loop is built and awaiting evaluation.

Falsify first: the Go/No-Go suite before building

Decision

Write down seven falsification tests with explicit kill conditions and run them before deep implementation.

Rejected

Building the full system first and testing at the end.

Evidence

The suite paid for itself: test 3 fired its kill signal (acceptance collapses at shallow exits), which redirected months of planned work into the multi-exit training that fixed it. Six of seven tests are executed; energy tooling waits on administrator access.

Status

settled — full table on Experiments.

Single-exit training failed, so we changed the loss

Decision

Train with a multi-exit loss: a primary objective at the deepest executed layer plus weighted auxiliary objectives at depths 4, 6, 8, 10 and 12.

Rejected

The first attempt — a single loss at the last executed layer — which made aggressive-exit acceptance worse at three of four measured depths.

Evidence

The single-exit run was reported as a negative result, not hidden. Diagnosis: it trained mostly the full-depth path, leaving early exits unimproved. The multi-exit recipe raised held-out exit-8 acceptance from 13.5% to 20.5%.

Status

settled — gate verdict PARTIAL (see Experiments).

We trained on our own test data — then retrained cleanly

Decision

Exclude every meeting that contains evaluation material from training, retrain from scratch, and correct all reported numbers in place.

Rejected

Keeping the inflated figures with a footnote.

Evidence

An automated leak check found 40/40 evaluation prompts verbatim in the training corpus. After exclusion the check reads 0/40 — and the headline gain fell from +22.1 to +7.0 points. The correction is kept visible in the ledger as evidence that the process catches its own errors.

Status

settled — all figures on this site are the corrected ones.

Our routing policy lost, so we kept the standard one

Decision

The safe core keeps per-word token-confidence routing; the dialogue-act policy remains only a candidate input to the pause trigger.

Rejected

Our own invented policy as the exit criterion.

Evidence

At matched computation the dialogue-act rule loses at every budget (25%: −5.8 pp; 50%: −8.9 pp; 75%: −7.2 pp). The same experiment showed training improved the standard rule's curve by +21.3 points — so training won even though the policy lost.

Status

settled — reported as a negative result with uncertainty analysis.

Small models on the host; the laptop waits

Decision

Do methods development on 1B–3B models on the development host; reserve the Dell Latitude 5490 for final validation.

Rejected

Forcing 7–9B models through an underpowered host (the 7B was measured fully swapped out: 1.8 MB resident against 5 GB swapped) or blocking all progress on laptop access.

Evidence

RAM-budget measurement: the 3B peaks at 2.55 GB against a 13 GB budget, while the 7B cannot coexist with a normal desktop load. All host numbers are labelled as such and are never presented as target-hardware results.

Status

settled — revisited in Phase 5.

Baselines measured locally, never cited

Decision

Reproduce all four baselines (plain decoding, two-model speculative, cascading, LayerSkip configuration) on our own hardware with our own harness.

Rejected

Quoting other papers' numbers, which would conflate their hardware with our methods.

Evidence

Plain decoding is measured (CPU 46.2 tok/s, GPU 207.5 tok/s, greedy, 5 repeats per prompt); the remaining three are in progress through November 2026.

Status

in progress

Still open