Explainer: PADS from zero
No background assumed. If you know what a voice assistant is, you know enough to start. Technical words link to the glossary.
On this page
Why voice assistants feel slow on a laptop
Talk to a voice assistant on a powerful server and it answers quickly. Run the same kind of assistant on an ordinary laptop with no graphics card, and every thoughtful answer takes noticeably longer. The laptop is not broken — it is doing exactly what it was asked to do, just slowly. The question this project asks is: can we make it feel faster without changing the answers?
What a language model actually does
A language model does not write sentences. It predicts one word at a time. Each word is cut into pieces called tokens, each piece becomes a list of numbers, and those numbers pass through a stack of sixteen identical processing blocks called a transformer. At the end, every possible next word gets a probability, the winner is picked, and the whole pass runs again for the next word. One word, one full pass — that is why answers take time. On a laptop the pass is slow because moving data in memory, not arithmetic, is the bottleneck (see Technology for why).
The shortcut: guessing early
Here is the key trick. If the model is trained the right way — a recipe called LayerSkip — then the first eight blocks already make a decent guess at the next word. So instead of always running all sixteen blocks, you can run eight, guess, and then run the remaining eight to check the guess. This is called self-speculative decoding: draft early, verify with the rest, on the same model.
If the check disagrees with the guess, the guess is thrown away and the checked answer is used. A thrown-away guess can never change the final answer. That guarantee is inherited from standard speculative decoding, and everything in this project depends on it.
The pause: free time nobody uses
When people talk, they pause briefly just before finishing a turn — typically a few hundred milliseconds of silence. Today's systems compute nothing during that silence: they wait for you to finish, then start thinking. We measured 29,297 real pauses and found the typical pause is about a second — roughly six times longer than one useful computing step. The pause is free wall-clock time, sitting unused.
Putting it together: what PADS does
PADS combines the shortcut with the pause. While you are still speaking, two small predictors watch your partial sentence: one estimates whether your turn is about to end, the other estimates whether your question will need deep thinking. Only when both are confident does the system start the deep computation inside the pause. Otherwise it stays shallow. When your turn actually ends, the early work is either confirmed — the answer is ready sooner — or discarded, which is always safe.
Being wrong costs a little wasted electricity; being right saves the user waiting. So the trigger fires only when both signals are confident — false alarms stay cheap and rare by design.
What we have actually proven
All four are development-host measurements, not target-laptop results. Full provenance on Experiments.
What we have not proven
- That any of this is faster on the target laptop — no number here comes from it yet.
- That the pause trigger itself saves time end to end — the loop is built and demonstrated, not yet evaluated.
- That our dialogue-based routing beats the standard rule — it lost, and we report that plainly (see Decisions).
- That the trained model is strictly better — it sometimes outputs nothing at all (15% of test prompts), an open issue.