Agentic AI Reference Architecture

Model Serving · Data artifact

LRU KV Cache Eviction Policy

Data artifactModel ServingModelsarc:LRUKVCacheEvictionPolicy

An eviction policy that removes KV caches of requests that have not generated tokens recently.

Responsibility. Evicts least-recently-active KV caches first.

Variant of KV Cache Eviction Policy abstract

When to choose. Choose for interactive sessions where recent activity signals continued engagement and idle sessions can be suspended.

specializesis target of alternativeTois target of alternativeToalternative toKV Cache Eviction Policy: specializesKV Cache Eviction PolicyCost-Aware Hybrid KV Cache Eviction Policy: is target of alternativeToCost-Aware Hybrid KV Cac…FIFO KV Cache Eviction Policy: is target of alternativeToFIFO KV Cache Eviction P…Priority KV Cache Eviction Policy: alternative toPriority KV Cache Evicti…
Direct neighbourhood (hover for relationship types)

Relationships

alternative to variability

Sources

  1. Ch4.4: T. Nguyen, "Performance Profiling and Optimization," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.4. ISBN: 9798244538229.