Agentic AI Reference Architecture

Observability & Evaluation · Data artifact

Exhaustive Evaluation Sampling Policy

Data artifactObservability & EvaluationObservability & Evaluationarc:ExhaustiveEvaluationSamplingPolicy

An evaluation sampling policy that evaluates every production trace.

Responsibility. Selects all production interactions for automated evaluation.

Also known as: 100% evaluation

Variant of Evaluation Sampling Policy abstract

When to choose. Choose when comprehensive per-trace ground truth justifies ~2-5% of inference cost and ~500 ms added latency per request.

specializesalternative toalternative toEvaluation Sampling Policy: specializesEvaluation Sampling PolicyRandom Evaluation Sampling Policy: alternative toRandom Evaluation Sampli…Strategic Evaluation Sampling Policy: alternative toStrategic Evaluation Sam…
Direct neighbourhood (hover for relationship types)

Relationships

alternative to variability

Quantitative guidance

As stated by the sources; verify before use.

Classification

Quality attributes
Maintainability (ISO/IEC 25010)

Sources

  1. Ref8.03: "Agent Evaluation Frameworks and Metrics," unpublished reference note (03-Agent-Evaluation-Frameworks.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note