Progress
Where the PADS project stands: what is measured and done, what is blocked, and what is explicitly left for next semester. Every number below names the artifact it came from.
Every measurement on this page was taken on the development host (a laptop with a discrete GPU used for training), not on the Dell Latitude 5490 target hardware. Wall-clock speedup for PADS proper is not measured, audio turn-taking is not implemented, and energy measurement is blocked on root access.
Overall: about 63% — and how that number is computed
We score six project blocks equally — Foundations & Go/No-Go, Phase 1 safe core, Phase 2 exit policies, Phase 3 baselines, Phase 4 turn-taking + PADS end-to-end, Phase 5 target hardware + manuscript — as 1.0 when done, 0.0 when not started, and fraction-of-plan-steps-done when in progress. That gives (1 + 1 + 1 + 0.76 + 0 + 0) / 6 = 3.76 / 6 ≈ 63%. The 0.76 for Phase 3 is 16 of 21 plan steps done in docs/superpowers/plans/2026-09-23-pads-phase3-baselines.md (Tasks 1, 3, 4 complete; Task 2 two-model speculative 0 of 4 steps; Task 5 comparison done except the proposal-timeline update). The PoC voice loop was unplanned extra work and is counted as a bonus, not in the denominator. The honest reading: the safe core is finished and publishable; PADS proper (Phases 4–5) has not started.
Phase checklist
| Phase | State | Evidence |
|---|---|---|
| Foundations & Go/No-Go falsification suite | done | 6 of 7 tests executed on the development host (energy tooling pending a root-only command); pause premise holds on AMI speech — experiments/go_no_go_results.md |
| Phase 1 — safe core (LayerSkip multi-exit training, held-out) | done | Held-out retrain + acceptance, leak check 0/40 — experiments/runs/2026-09-22T18-22-08Z-heldout-retrain/acceptance_heldout.md; plan 23/23 steps in docs/superpowers/plans/2026-09-22-pads-phase2-exit-policy.md Task 3b context |
| Phase 2 — exit policies (token-confidence vs dialogue-act) | done | Matched-compute comparison with cluster bootstrap — experiments/runs/2026-09-22T19-18-00Z-phase2-policy-eval-heldout/comparison.md; plan Tasks 1–4 + 3b + 3c all checked |
| Phase 3 — baselines (plain, speculative, cascading, LayerSkip config) | in progress | 3 of 4 baselines measured; two-model speculative blocked — experiments/runs/phase3-comparison/comparison.md; plan 16/21 steps |
| PoC voice loop (wav → STT → trigger → LLM → TTS, CPU-only) | done | Working offline demo, 15.0 s total, 1.34 GB peak — experiments/runs/poc-smoke/POC_NOTES.md, timings.json, demo_run.log |
| Phase 4 — turn-taking proxy + PADS end-to-end | planned | Next semester (Jan – Feb 2027); needs streaming STT partials and real pause audio per experiments/runs/poc-smoke/POC_NOTES.md §Next step |
| Phase 5 — target hardware + manuscript | planned | Next semester (Mar – Apr 2027); Dell Latitude 5490 validation, energy/thermal protocol, IEEE manuscript |
Headline numbers
Where each number comes from
| Number | Source artifact |
|---|---|
| 96.6% pause fit; 1,050 ms median pause; 164.1 ms step | experiments/runs/2026-09-21T19-05Z-test2-pause-durations/test2_verdict_7b.txt; method in experiments/go_no_go_results.md Test 2 row |
| Held-out L8 13.48% → 20.46% (17.39% conservative) | experiments/runs/2026-09-22T18-22-08Z-heldout-retrain/acceptance_heldout.md and metrics_extracted.json (baseline_L8 / trained_L8); ledger table in experiments/go_no_go_results.md |
| Safe-core +21.25 pp at 50% deep [18.45, 24.15] | experiments/runs/2026-09-22T19-18-00Z-phase2-policy-eval-heldout/comparison.md; ledger verdict in experiments/go_no_go_results.md (held-out token 0.7598 vs base 0.5473) |
| Classifier 0.859 accuracy / 0.856 macro-F1 | experiments/results/dialogue_act_realdialog_metrics.json (accuracy 0.8589, macro-F1 0.8559; majority baseline 0.5494) |
| Plain CPU 46.18 ± 0.18 / GPU 207.50 ± 1.23 tok/s | experiments/runs/2026-09-22T21-23-19Z-phase3-baselines-5x/summary.json (plain_cpu / plain_gpu per_prompt_mean); comparison in experiments/runs/phase3-comparison/comparison.md |
| PoC 15.0 s total; 1.34 GB peak; p_deep 0.7401 | experiments/runs/poc-smoke/timings.json (total_wall_s 14.981), experiments/runs/poc-smoke/demo_run.log (Maximum resident set size 1,407,804 KB), experiments/runs/poc-smoke/POC_NOTES.md |
Four values were recomputed from source JSONs/logs rather than copied from prose: (1) plain CPU tok/s mean 46.18 and (2) GPU 207.5015 read from summary.json per_prompt_mean means; (3) classifier accuracy 0.8589 read from dialogue_act_realdialog_metrics.json; (4) pause-fit 96.6% read from test2_verdict_7b.txt. Also confirmed: PoC peak RSS 1,407,804 KB in demo_run.log (≈1.34 GB), PoC total 14.981 s in timings.json, trained-L4 acceptance 0.0384 in metrics_extracted.json (matches the 3.84% ledger row, validating that file), and 6 empty-generation skips in the held-out trained L8 eval log (the 15% rate). All matched.
What's blocked / open
Two-model speculative baseline blocked
llama.cpp speculative decoding needs draft and target to share a vocabulary. The only local target GGUF is llama-arch (models/gguf/layerskip-1b-Q4_K_M.gguf); the only local small drafts are qwen2-family. Three options, none taken yet: (1) download a small llama-arch draft GGUF (e.g. TinyLlama) to pair with the LayerSkip target; (2) run the arm entirely in qwen2 (local Qwen2.5-0.5B draft + local Qwen2.5-7B GGUF target) — valid speculation measurement, different model family; (3) drop the arm and treat the self-speculative LayerSkip-config row as the speculation comparator. See experiments/runs/phase3-comparison/comparison.md row 3 note.
15% empty generations open
The held-out adapter produces empty generations on 6/40 eval prompts at every exit layer vs 0/40 for the base model. Cause unknown; the conservative empties-as-failures view (L8 17.39%) misses the ≥ 20% gate. See experiments/runs/2026-09-22T18-22-08Z-heldout-retrain/acceptance_heldout.md.
Energy measurement needs root blocked
RAPL energy_uj reads permission-denied non-root; temperature-only sensors otherwise. Blocked until someone runs the documented sudo turbostat command. See experiments/go_no_go_results.md Test 6 row and experiments/results/test6_energy_tooling_host.log.
No target hardware yet planned
All numbers are development-host numbers. Dell Latitude 5490 validation, sustained thermal/energy protocol, and audio turn-taking all wait for Phases 4–5 next semester. The ledger's "Not measured" lists carry forward in experiments/go_no_go_results.md.
Negative and partial results, stated plainly
- Dialogue-act exit policy is a negative result: behind token-confidence at every matched budget (−5.8/−8.9/−7.2 pp at 25/50/75% deep), never winning in 10/10 held-out pointwise comparisons. The safe core keeps token-confidence; dialogue-act survives only as a pause-trigger input.
- Held-out gate verdict is PARTIAL: L8 clears ≥ 20% by 0.46 pp on non-empty samples (20.46%) but fails the conservative view (17.39%); the contamination that inflated the original 35.61% is documented, not hidden.
- Cross-arm tok/s comparisons use three different timers (llama-cli subprocess, HF generate() wall, benchmark.py in-process with no warm-up discard) — no speedup claim is made across instruments.
Ledger: experiments/go_no_go_results.md (Phase 3 section, 2026-10-28). Comparison: experiments/runs/phase3-comparison/comparison.md. Plans: docs/superpowers/plans/2026-09-23-pads-phase3-baselines.md, docs/superpowers/plans/2026-09-22-pads-phase2-exit-policy.md.