What happens when a draft token is rejected?
In exact speculative sampling, a replacement is drawn from a corrected residual distribution proportional to max(0, p minus q), not simply from the original target distribution.
Answered in
Speculative Decoding: Faster Tokens Without Changing the Target DistributionA draft model proposes tokens and a target verifies them. Exact speculative sampling keeps the target distribution; speedups depend on the workload.
Read the full analysisOther questions this article answers
More system design & architecture questions
- Does a virtual display sandbox an agent?
- Can invisible text hijack a screenshot-only agent?
- Is MCP inherently stateless?
- Does MCP automatically secure an exposed tool?
- Does speculative decoding always double speed?
- Does Wasm guarantee that untrusted code cannot escape?
- Does WASI 0.2 run arbitrary desktop Python or Bash unchanged?
- Is PostgreSQL ts_rank_cd a BM25 implementation?
Every answer on Crashtech is written by the editor of the article it comes from — never auto-summarised. Browse all answers or the System Design & Architecture beat.