Observability & Evaluation · Data artifact

Service Level Objective Specification

Data artifactObservability & EvaluationObservability & Evaluationarc:ServiceLevelObjectiveSpec

A specification of target latency percentiles, minimum throughput and maximum cost per request against which serving configurations and monitors are validated.

Responsibility. States the performance and cost targets a deployment must meet.

Also known as: SLO specification, SLA target, Latency SLO, Availability SLO, Error budget definition

Direct neighbourhood (hover for relationship types)

Relationships

configures structural

is read by dependency

constrains control

Design guidance

SHOULD validate candidate configurations against every SLO dimension and escalate conflicting dimensions to stakeholders for negotiation.
SHOULD scale out instances when latency passes but throughput misses; when latency fails even at minimum batch size, only model optimisation or GPU upgrades help.
SHOULD state user-facing latency targets on P95 (e.g., 95% of requests under 2 s) and track P99 separately for architectural diagnosis; MUST NOT rely on average-only targets.
MUST state the concurrent load (RPS) under which each latency target or measurement applies.
SHOULD target TTFT under 1 s and token generation above 20 tokens/s for interactive streaming agents; batch agents SHOULD target end-to-end latency.
SHOULD express reliability targets as measurable SLOs (e.g., 99.9% of requests succeed within 500 ms) over a trailing window.
SHOULD define separate SLOs for infrastructure reliability and for safety-validation success.

Quantitative guidance

As stated by the sources; verify before use.

Example SLO: p50 < 150 ms, p95 < 300 ms, p99 < 500 ms, >= 500 RPS, <= $0.50 per 1,000 requests (Ch4.2).
Worked-example SLOs: P95 < 8 s (recommendations); ReAct agent 2 s p99 SLA (Ch4.2).
Targets: availability 99.9%, P95 latency <100ms, error rate <0.1%, SLA compliance >99% (Ref7.16).
High-latency alert at 2s matches a typical 2.5s SLA with 500ms buffer (Ch7.2).
User abandonment accelerates beyond 5-7 s; task completion declines 7-10% per additional second; satisfaction for 2 s responses is ~40% higher than for 5+ s (Ch8.1).
Users read ~250 words/min (~4 words/s); generation below 20 tokens/s feels sluggish (Ch8.1).
E-commerce agent SLO: 99.9% availability over trailing 30 days -> 0.1% error budget (43.2 min/month) (Ch8.2A).
Infrastructure SLO 99.9% availability (0.1% budget); safety SLO 99% validation success (1% expected adversarial-input baseline) (Ch8.2B).
Latency targets p50 < 1-2 s, p95 < 3-5 s, p99 < 10-15 s, max < 30 s; availability 95% basic, 99% standard, 99.9% critical, 99.99% enterprise (Ref8.09).
Quality targets: success rate > 95%, user satisfaction > 4.0/5, hallucination rate < 2%, tool-call correctness > 98% (Ref8.09).

Classification

Patterns: SLO targeting
Quality attributes: Performance efficiency (ISO/IEC 25010)Cost efficiency
Risks mitigated: Unprioritised latency-throughput tradeoffsImpossible conflicting requirements discovered late

Sources

Ch4.2: T. Nguyen, "Deployment and Scaling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.2. ISBN: 9798244538229.
Ch6.5: T. Nguyen, "Production RAG Systems," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.5. ISBN: 9798244538229.
Ch7.2: T. Nguyen, "Performance Optimization and Production Monitoring," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.2. ISBN: 9798244538229.
Ch8.1: T. Nguyen, "Latency Metrics," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.1. ISBN: 9798244538229.
Ch8.2A: T. Nguyen, "Error Rates and Reliability," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.2A. ISBN: 9798244538229.
Ch8.2B: T. Nguyen, "NeMo Guardrails Integration," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.2B. ISBN: 9798244538229.
Ref7.16: "Production Monitoring and Operations for Agentic AI," unpublished reference note (16-Production-Monitoring-Operations.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
Ref8.03: "Agent Evaluation Frameworks and Metrics," unpublished reference note (03-Agent-Evaluation-Frameworks.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
Ref8.09: "Chapter 8 Summary: Run, Monitor, and Maintain," unpublished reference note (09-Chapter-8-Summary.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note