Agentic AI Reference Architecture

Observability & Evaluation · Data store

Time-Series Metrics Store

Data storeObservability & EvaluationObservability & Evaluationarc:TimeSeriesMetricsStore

A database of timestamped metric samples collected from agents and infrastructure, queryable for rates, percentiles and trends.

Responsibility. Persists and serves metric time series for dashboards and alerts.

Also known as: Time-series database, TSDB

is read by; is written by; receives telemetry fromis read by; is monitored byis written byis read byis read byis read byis read byis read byis written byreceives telemetry fromis read byis read byis written byis read byis read byis read byreceives telemetry fromis read byToken Cost Meter: is read by; is written by; receives telemetry fromToken Cost MeterAlert Manager: is read by; is monitored byAlert ManagerMetrics Collector: is written byMetrics CollectorMetrics Dashboard: is read byMetrics DashboardQuality Drift Detector: is read byQuality Drift DetectorSLO Monitor: is read bySLO MonitorMetric-Driven Autoscaler: is read byMetric-Driven AutoscalerCanary Rollout Controller: is read byCanary Rollout ControllerFairness Monitor: is written byFairness MonitorTrace Exporter: receives telemetry fromTrace ExporterAgent Behavior Anomaly Detector: is read byAgent Behavior Anomaly D…Execution Monitor Console: is read byExecution Monitor ConsoleProduction Quality Monitor: is written byProduction Quality MonitorCompliance Evidence Collector: is read byCompliance Evidence Coll…Statistical Comparator: is read byStatistical ComparatorBottleneck Analyzer: is read byBottleneck AnalyzerGPU Telemetry Exporter: receives telemetry fromGPU Telemetry ExporterCompliance Dashboard: is read byCompliance Dashboard+22 more (see relationships)
Direct neighbourhood (hover for relationship types)

Relationships

is read by dependency

is written by dependency

receives telemetry from dynamic

is constrained by control

is monitored by assurance

Design guidance

Quantitative guidance

As stated by the sources; verify before use.

Classification

Technologies
PrometheusPrometheus TSDBPromQLInfluxDB
Quality attributes
Maintainability (ISO/IEC 25010)

Sources

  1. Ch4.1: T. Nguyen, "Introduction to AI Agent Deployment and Scaling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.1. ISBN: 9798244538229.
  2. Ch4.3: T. Nguyen, "Container Orchestration and Edge Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.3. ISBN: 9798244538229.
  3. Ch6.2B: T. Nguyen, "Production Vector Database Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.2B. ISBN: 9798244538229.
  4. Ch6.5: T. Nguyen, "Production RAG Systems," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.5. ISBN: 9798244538229.
  5. Ch7.2: T. Nguyen, "Performance Optimization and Production Monitoring," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.2. ISBN: 9798244538229.
  6. Ch7.3: T. Nguyen, "NeMo Agent Toolkit Profiling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.3. ISBN: 9798244538229.
  7. Ch8.1: T. Nguyen, "Latency Metrics," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.1. ISBN: 9798244538229.
  8. Ch8.2A: T. Nguyen, "Error Rates and Reliability," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.2A. ISBN: 9798244538229.
  9. Ch8.2B: T. Nguyen, "NeMo Guardrails Integration," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.2B. ISBN: 9798244538229.
  10. Ch8.3: T. Nguyen, "Token Economics and Architecture," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.3. ISBN: 9798244538229.
  11. Ch9.4: T. Nguyen, "Fairness and Bias Mitigation," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 9.4. ISBN: 9798244538229.
  12. Ch10.1: T. Nguyen, "Conversational UI," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 10.1. ISBN: 9798244538229.
  13. Ch10.4: T. Nguyen, "Human-in-the-Loop," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 10.4. ISBN: 9798244538229.
  14. Ref7.15: "Advanced Agentic AI Optimization Techniques," unpublished reference note (15-Advanced-Agentic-Optimization.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  15. Ref7.16: "Production Monitoring and Operations for Agentic AI," unpublished reference note (16-Production-Monitoring-Operations.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  16. Ref7.18: "Chapter 7 Summary: NVIDIA Platform Implementation," unpublished reference note (18-Chapter-7-Summary.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  17. Ref8.02: "Machine Learning Monitoring in Production," unpublished reference note (02-ML-Monitoring-Production.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  18. Ref8.09: "Chapter 8 Summary: Run, Monitor, and Maintain," unpublished reference note (09-Chapter-8-Summary.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  19. Ref9.06: "Auditing and Compliance Monitoring for AI Systems," unpublished reference note (references/Chapter 9 - Safety, Ethics, and Compliance/06-Auditing-Compliance-Monitoring.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  20. Ref9.09: "Compliance Automation and Tools," unpublished reference note (09-Compliance-Automation-Tools.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  21. Ref9.10: "Chapter 9 Summary: Safety, Ethics, and Compliance," unpublished reference note (10-Chapter-9-Summary.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  22. Ref10.04: "Change Management and Organizational Adoption," unpublished reference note (04-Change-Management-Adoption.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note