Model Serving · Software component

Draft Length Controller

Software componentModel ServingModelsarc:DraftLengthController

A runtime control component that tracks speculative acceptance over a sliding window and raises or lowers the draft length K, disabling speculation when acceptance falls below a threshold.

Responsibility. Keeps speculative draft length matched to the observed acceptance rate.

Also known as: Adaptive K adjustment, Dynamic draft length

Direct neighbourhood (hover for relationship types)

Relationships

writes dependency

Speculative Decoding Configuration Ch7.1A

monitors assurance

Speculative Decoder Ch7.1A

Design guidance

SHOULD increase K when acceptance exceeds 0.8 and decrease it when acceptance falls below 0.5, over windows of about 100 verifications.

Quantitative guidance

As stated by the sources; verify before use.

Dynamic K adjustment recovers 20-30% speedup (Ch7.1A).

Classification

Patterns: Sliding-window adaptive control

Sources

Ch7.1A: T. Nguyen, "Advanced Implementation with Nvidia NEMO Framework and Nvlink," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.1A. ISBN: 9798244538229.