Agentic AI Reference Architecture

Model Serving · Data artifact

Cost-Aware Hybrid KV Cache Eviction Policy

Data artifactModel ServingModelsarc:CostAwareKVCacheEvictionPolicy

A hybrid eviction policy combining recency, priority and recomputation cost, biasing eviction toward short caches that are cheap to regenerate.

Responsibility. Evicts KV caches minimizing combined recency, priority and recomputation cost.

Variant of KV Cache Eviction Policy abstract

When to choose. Choose in production when recency, priority and recomputation cost must be balanced together.

specializesalternative toalternative toalternative toKV Cache Eviction Policy: specializesKV Cache Eviction PolicyFIFO KV Cache Eviction Policy: alternative toFIFO KV Cache Eviction P…LRU KV Cache Eviction Policy: alternative toLRU KV Cache Eviction Po…Priority KV Cache Eviction Policy: alternative toPriority KV Cache Evicti…
Direct neighbourhood (hover for relationship types)

Relationships

alternative to variability

Sources

  1. Ch4.4: T. Nguyen, "Performance Profiling and Optimization," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.4. ISBN: 9798244538229.