Agentic AI Reference Architecture

Model Serving · Data artifact

FIFO KV Cache Eviction Policy

Data artifactModel ServingModelsarc:FIFOKVCacheEvictionPolicy

An eviction policy that removes the oldest requests' KV caches first.

Responsibility. Evicts the oldest KV caches first.

Variant of KV Cache Eviction Policy abstract

When to choose. Choose when all requests have similar priority.

specializesis target of alternativeToalternative toalternative toKV Cache Eviction Policy: specializesKV Cache Eviction PolicyCost-Aware Hybrid KV Cache Eviction Policy: is target of alternativeToCost-Aware Hybrid KV Cac…LRU KV Cache Eviction Policy: alternative toLRU KV Cache Eviction Po…Priority KV Cache Eviction Policy: alternative toPriority KV Cache Evicti…
Direct neighbourhood (hover for relationship types)

Relationships

alternative to variability

Sources

  1. Ch4.4: T. Nguyen, "Performance Profiling and Optimization," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.4. ISBN: 9798244538229.