Pause-Aware Depth Scheduling
Using the natural silence before a person finishes speaking as free compute time for the expensive part of language-model inference — on hardware with no discrete GPU.
The one-sentence bet
Conversational systems on CPU-only devices pay the full serial cost of every reasoning layer after the user finishes speaking, and the user waits for all of it. But human turns contain a short, predictable silence before they end — typically a few hundred milliseconds. PADS asks whether that silence can be used to begin full-depth computation early, using self-speculative early exit inside a single model, so that by the time the turn actually ends some or all of the reasoning cost is already paid for.
Routing between a small and a large model is a crowded, solved problem, and it cannot hand off KV-cache state between differently sized models — caches are tied to a model's own hidden dimension and layer count. PADS never switches models. It switches depth within one network: early layers draft, later layers verify, and the pause buys time to start the later layers sooner.
How it works
Two tiers, so a negative result is still a result
Safe core measured
A systematic characterisation of self-speculative early exit on CPU-bound hardware, plus a dialogue-act-driven exit policy compared against the standard token-confidence policy at matched compute. Independently publishable.
Stretch layer in progress
PADS proper: using the pause to start full-depth computation before the turn ends. Needs a turn-taking signal and real pause timing. If it shows no gain, it is written up as a limitation — the safe core stands on its own.
Status at a glance
| Phase | What it covers | State |
|---|---|---|
| Foundations & Go/No-Go | Literature gap, pre-registered falsification suite, environment, hardware characterisation | done |
| Phase 1 — safe core | LayerSkip reproduction, multi-exit training, real dialogue-act classifier, held-out acceptance | done |
| Phase 2 — exit policies | Token-confidence vs dialogue-act routing at matched compute, with uncertainty | done |
| Phase 3 — baselines | Plain decoding, two-model speculative, cascading, LayerSkip configuration reproduced locally | running |
| Phase 4 — pause-aware end-to-end | Turn-taking proxy, real pause distributions, PADS vs safe core | planned |
| Phase 5 — target hardware & manuscript | Dell Latitude 5490 validation, energy/thermal protocol, IEEE manuscript | planned |
Every measurement on this site was taken on the development host (a laptop with a discrete GPU used for training), not on the Dell Latitude 5490 that the project's claims are about. Development-host numbers are labelled as such throughout. Wall-clock speedup for PADS proper is not measured, audio turn-taking is not implemented, and energy measurement is blocked on root access.
Explore
Architecture
The full system, training, evaluation, and deployment pipelines in diagrams.
Technology
Every technique and tool used, explained: early exit, LayerSkip, LoRA, GGUF, policies, statistics.
Experiments
What was measured, what it showed, and what remains unmeasured.
Glossary
Definitions for every technical term used on this site.