Agentic AI Reference Architecture

Model Serving · Data artifact

Priority KV Cache Eviction Policy

Data artifactModel ServingModelsarc:PriorityKVCacheEvictionPolicy

An eviction policy that preserves caches of high-priority requests and evicts low-priority ones under memory pressure.

Responsibility. Evicts lowest-priority KV caches first.

Variant of KV Cache Eviction Policy abstract

When to choose. Choose when requests carry differentiated SLAs (e.g., premium versus free tier).

specializesis target of alternativeTois target of alternativeTois target of alternativeToKV Cache Eviction Policy: specializesKV Cache Eviction PolicyCost-Aware Hybrid KV Cache Eviction Policy: is target of alternativeToCost-Aware Hybrid KV Cac…FIFO KV Cache Eviction Policy: is target of alternativeToFIFO KV Cache Eviction P…LRU KV Cache Eviction Policy: is target of alternativeToLRU KV Cache Eviction Po…
Direct neighbourhood (hover for relationship types)

Relationships

alternative to variability

Sources

  1. Ch4.4: T. Nguyen, "Performance Profiling and Optimization," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.4. ISBN: 9798244538229.