Self-Speculative Decoding Head

Model assetModel ServingModelsarc:SelfSpeculativeDecodingHead

Lightweight prediction layers attached to the target model that predict several future tokens simultaneously, enabling speculation without a separate draft model.

Responsibility. Provides speculative drafts with minimal memory and no inter-model coordination.

Also known as: Medusa heads, EAGLE heads

Variant of Draft Token Proposer abstract

When to choose. Choose when memory is constrained or batch size must be preserved (e.g., edge devices, high-batch serving).

Direct neighbourhood (hover for relationship types)

Relationships

deployed on structural

Edge GPU Device Ch7.1A

alternative to variability

Speculative Draft Model abstract Ch7.1A

Quantitative guidance

As stated by the sources; verify before use.

10-15% memory overhead, 20-40 GPU-hours training, alpha=0.6-0.75, 1.8-2.5x speedup (Ch7.1A).
Medusa heads add 8-12% memory, keeping batch 14-15 of 16: net 2.0x vs 1.4x for two-model (Ch7.1A).
On Jetson Orin 48GB, enables 70B (vs 34B) models with <1s latency for 3-step reasoning (Ch7.1A).

Classification

Technologies: MedusaEAGLE

Sources

Ch7.1A: T. Nguyen, "Advanced Implementation with Nvidia NEMO Framework and Nvlink," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.1A. ISBN: 9798244538229.