Agentic AI Reference Architecture

Observability & Evaluation · Data store

Quality Improvement Backlog

Data storeObservability & EvaluationObservability & Evaluationarc:QualityImprovementBacklog

A tracked list of identified reasoning-quality problems, each with priority, owner and success criteria, feeding targeted improvements and re-evaluation.

Responsibility. Tracks evaluation findings as actionable improvement tasks.

Also known as: Improvement hypothesis backlog

is read byis written byis written byis written byis written byTraining Pipeline Orchestrator: is read byTraining Pipeline Orches…Feedback Prioritizer: is written byFeedback PrioritizerEvaluation Result Analyzer: is written byEvaluation Result AnalyzerTrace Pattern Miner: is written byTrace Pattern MinerIntent Performance Analyzer: is written byIntent Performance Analy…
Direct neighbourhood (hover for relationship types)

Relationships

is read by dependency

is written by dependency

Classification

Patterns
Structured improvement loop
Risks mitigated
Improvement loop failure

Sources

  1. Ch3.9: T. Nguyen, "Reasoning Quality," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.9. ISBN: 9798244538229.
  2. Ch10.1: T. Nguyen, "Conversational UI," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 10.1. ISBN: 9798244538229.
  3. Ref10.03: "User Feedback and Iterative Improvement," unpublished reference note (03-User-Feedback-Iteration.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  4. Ref10.05: "The Data Flywheel: Continuous Improvement Loop," unpublished reference note (05-Data-Flywheel-Continuous-Improvement.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note