Observability & Evaluation · Data artifact

Failure Taxonomy

Data artifactObservability & EvaluationObservability & Evaluationarc:FailureTaxonomy

A versioned classification scheme of agent failure categories with domain-specific severity and fix-effort weights used to categorise and prioritise evaluation failures.

Responsibility. Defines failure categories and their severity and effort weights.

Also known as: Error Taxonomy, Hallucination severity classification, Hallucination taxonomy, Three-tier agent error taxonomy, Planning/execution/verification failure tiers, Agent execution / system / data error categories

Direct neighbourhood (hover for relationship types)

Relationships

configures structural

Evaluation Failure Analyzer Ch3.4 Ch3.10 +1

Design guidance

SHOULD encode the domain's risk tolerance in category severity weights.
SHOULD distinguish critical, high, medium and low severity hallucinations by impact.
SHOULD distinguish mechanisms: training data contamination, temporal misinformation, confabulated citations, contextual distortion, multi-agent coordination failure.
SHOULD classify every agent failure as planning (reasoning), execution (infrastructure) or verification (silent incorrect output) before choosing a remedy.
SHOULD match remediation to tier: prompts, examples or fine-tuning for planning; retries, circuit breakers and capacity for execution; validation, guardrails and human review for verification.
SHOULD track remediation effectiveness per tier; simultaneous decrease across tiers suggests a broader (e.g., load-related) root cause.

Quantitative guidance

As stated by the sources; verify before use.

Categories: tool selection, parameter, reasoning, retrieval, timeout, hallucination; example severity 6/5/7/5/3/9 and effort 7/8/3/4/6/2 on 1-10 scales (Ch3.4).
If 80% of errors are execution failures (e.g., payment API timeouts), prompt changes accomplish nothing (Ch8.2A).
Categories: agent execution (hallucination HIGH, tool call MEDIUM, decision loop HIGH), system (OOM CRITICAL, latency timeout HIGH, database connection CRITICAL), data (missing/corrupt MEDIUM, API response MEDIUM-HIGH) (Ref8.06).

Classification

Patterns: Severity-weighted prioritisationTaxonomy-driven diagnosis
Quality attributes: Maintainability (ISO/IEC 25010)
Risks mitigated: Misapplied mitigations from conflated failure typesOne-size-fits-all remediation applied to the wrong failure tier

Sources

Ch3.4: T. Nguyen, "Tuning Model Parameters for Production Performance," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.4. ISBN: 9798244538229.
Ch3.10: T. Nguyen, "Efficiency Metrics," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.10. ISBN: 9798244538229.
Ch8.2A: T. Nguyen, "Error Rates and Reliability," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.2A. ISBN: 9798244538229.
Ref8.06: "Error Troubleshooting and Incident Response for Agent Systems," unpublished reference note (06-Error-Troubleshooting-Incident-Response.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
Ref8.07: "Agent Health Checks and Diagnostics," unpublished reference note (07-Agent-Health-Checks-Diagnostics.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note