Experiments
What was measured, what it showed, and what is still unmeasured. Every row cites the artifact it comes from; development-host results are labelled.
All numbers below were measured on the development laptop (Intel Core Ultra 7 155H, 15 GB RAM, RTX 4070 Laptop), not on the Dell Latitude 5490 the project targets. They are methodology and early-signal evidence, not target-hardware results.
On this page
Go/No-Go falsification suite
Seven experiments were pre-registered with explicit kill conditions before implementation. Six have been executed on the development host; the energy-tooling check needs root and is pending.
| # | Test | Result | Key numbers (development host) |
|---|---|---|---|
| 1 | Bandwidth vs compute | Partial — 7B did not fit host RAM (fully swapped); 3B mitigation branch executed | 3B Q4_1: 5.3 → 14.2 tok/s from 1 to 8 threads, saturating past 5–6 threads |
| 2 | Real pause-duration feasibility | PASS | AMI corpus, 29,297 silence pauses: median 1,050 ms; 96.6% fit a 164 ms extrapolated 7B step, 100% fit the measured 3B 70 ms step |
| 3 | Off-the-shelf acceptance rate | Kill signal fired at aggressive exits | Greedy, exit layers 4/6/8/10/12: 6.4 / 9.4 / 10.2 / 14.1 / 56.1% |
| 4 | Thermal stability | Throttling observed | 177 samples over ~35 min: max 103 °C, 16.9% of samples ≥ 95 °C |
| 5 | RAM budget | 3B fits comfortably; 7B does not fit this host | 3B Q4_1 peak RSS 2.553 GB against a 13 GB budget |
| 6 | Energy tooling | Blocked on root | turbostat/powertop/RAPL present; energy_uj unreadable without root; USB power meter is the pre-planned fallback |
| 7 | Literature scoop check | Partial overlap, not scooped | Two systems require citation (arXiv:2606.13450, arXiv:2607.23204); 32/32 arXiv identifiers verified, none fabricated |
Multi-exit training and the leakage correction
The off-the-shelf checkpoint's acceptance collapses at shallow exits, so we continued-train it with the LayerSkip recipe — layer dropout plus a multi-exit loss — using LoRA adapters.
The first training run appended every AMI meeting, including the three the evaluation set is drawn from: 40/40 evaluation prompts appeared verbatim in the training corpus. The reported gain was inflated. The model was retrained with a held-out split (leak check 0/40) and all numbers below are the corrected ones. The contaminated figures were +22.1 pp at exit 8; the honest figure is +7.0 pp.
| Exit layer (of 16) | Baseline (N=40) | Held-out trained (N=34 non-empty) | Empties as failures (N=40) |
|---|---|---|---|
| 4 | 9.16% | 3.84% | 3.26% |
| 6 | 12.08% | 19.34% | 16.44% |
| 8 | 13.48% | 20.46% | 17.39% |
| 10 | 16.55% | 29.97% | 25.48% |
| 12 | 53.95% | 57.89% | 49.20% |
Acceptance = fraction of drafted tokens the full-depth model accepts, greedy decoding, 40 AMI conversational turns. The trained model emits empty generations on 6/40 prompts (baseline 0/40) — an unresolved quality regression. Gate verdict: PARTIAL (exit 8 clears the 20% target by 0.5 pp; the conservative view misses it).
Exit-policy comparison (Phase 2)
Two policies decide how deep to compute: token confidence (the standard per-token criterion) and the dialogue-act policy (a turn-level decision from the trained classifier). They are compared at matched compute — equal fractions of positions routed to full depth — by agreement with the full-depth oracle.
| Compute budget | Token confidence | Dialogue act | Difference |
|---|---|---|---|
| 25% deep | 55.2% | 49.4% | −5.8 pp |
| 50% deep | 76.0% | 67.1% | −8.9 pp |
| 75% deep | 91.6% | 84.5% | −7.2 pp |
Negative result for the dialogue-act exit policy: it is behind token confidence in every point estimate at every matched budget, and never beats it — the safe core keeps token confidence, and the dialogue-act signal remains only a candidate pause-trigger input. Under the assumption-free uncertainty convention the fixed-budget deficit is not statistically resolved at n=40; the direction is consistent across all three conventions (sensitivity table in the run directory).
The same experiment shows the trained model's token-confidence curve beats the base model's by +21.25 pp at 50% deep (95% CI 18.45–24.15) and +12.82 pp at 75% deep — a convention-robust result, and the strongest evidence so far that multi-exit training genuinely improves early-exit quality on CPU-bound models.
Baselines (Phase 3, in progress)
The PRD requires four baselines reproduced locally rather than cited from other papers. The first — plain decoding — is measured; the rest are running.
| Configuration | Hardware | Tokens/s | Peak RSS | TTFT |
|---|---|---|---|---|
| Plain decoding, llama.cpp, LayerSkip 1B Q4_K_M | CPU, 8 threads | 44.89 ± 0.33 | 1.204 GB | 3.60 ms (derived) |
| Plain decoding, llama.cpp, LayerSkip 1B Q4_K_M | GPU, 8 threads | 200.73 ± 2.70 | 1.114 GB | 0.42 ms (derived) |
| Two-model speculative decoding | CPU + GPU | in progress | ||
| Plain cascading (0.5B → 1B) | GPU | in progress | ||
| LayerSkip published configuration | GPU | from Phase 1/2 measurements (acceptance 6.4–56.1% by exit layer) | ||
38 measured runs per arm (2 warm-up runs discarded), 40 AMI prompts, 128-token generations, 0 empty generations in both arms. TTFT is derived from the prompt-processing rate because this llama.cpp build prints only aggregate rates. Energy is null — RAPL needs root.
Not measured
- Target hardware. No reported number comes from the Dell Latitude 5490 yet — that is Phase 5.
- Wall-clock speedup for PADS proper. The pipeline is a prototype without cache-aware depth switching; latency claims would be invalid, so none are made.
- Generation-level quality (ROUGE-L / exact match) at matched compute — the Phase 2 comparison measures routing agreement with the oracle, not free-form generation quality.
- Energy and thermal on the target — energy tooling is blocked on root; the thermal run was a host substitute.
- Audio turn-taking. The current signal is a text-based proxy; VAP-style audio prediction is not implemented.
- The 15% empty-generation regression in the trained model is unresolved and is carried as a known quality issue.