Agentic AI Reference Architecture

Human Oversight · Human role

Platform Operator

Human roleHuman OversightExperience & Human Oversightarc:PlatformOperator

An on-call engineer who responds to alerts, investigates scaling anomalies and intervenes manually in caching and traffic distribution.

Responsibility. Responds to alerts and manually intervenes in platform behaviour.

Also known as: On-call engineer, Operator, SRE, Platform team, IT administrator, Fleet administrator, On-call operator, Operations team

receives escalation from; is triggered by; receives data frommonitors; invokestriggers; receives escalation frommonitorsreceives escalation fromoverridesreceives escalation frominvokesmonitorsmonitorsinvokesinvokesmonitorsinvokesinvokesinvokesreceives escalation frominvokesAlert Manager: receives escalation from; is triggered by; receives data fromAlert ManagerRollout Manager: monitors; invokesRollout ManagerEdge Update Orchestrator: triggers; receives escalation fromEdge Update OrchestratorInference Server: monitorsInference ServerRetry Handler: receives escalation fromRetry HandlerLoad Balancer: overridesLoad BalancerIncident Manager: receives escalation fromIncident ManagerMetrics Dashboard: invokesMetrics DashboardSelf-Managed Vector Index Store: monitorsSelf-Managed Vector Inde…Autoscaler: monitorsAutoscalerGPU System Profiler: invokesGPU System ProfilerCanary Rollout Controller: invokesCanary Rollout ControllerGPU Partition: monitorsGPU PartitionCache Invalidator: invokesCache InvalidatorVersion Rollback Controller: invokesVersion Rollback Control…Bottleneck Analyzer: invokesBottleneck AnalyzerTool Error Classifier: receives escalation fromTool Error ClassifierFeature Flag Service: invokesFeature Flag Service+6 more (see relationships)
Direct neighbourhood (hover for relationship types)

Relationships

invokes dependency

is triggered by dynamic

receives data from dynamic

receives escalation from dynamic

triggers dynamic

overrides control

monitors assurance

Design guidance

Quantitative guidance

As stated by the sources; verify before use.

Classification

Patterns
Manual cache purgeManual traffic weighting
Quality attributes
Maintainability (ISO/IEC 25010)

Sources

  1. Ch1.8: T. Nguyen, "Scalability and Production Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.8. ISBN: 9798244538229.
  2. Ch2.8: T. Nguyen, "Error Handling and Resilience," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 2.8. ISBN: 9798244538229.
  3. Ch3.6: T. Nguyen, "Trace Analysis and Execution Debugging," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.6. ISBN: 9798244538229.
  4. Ch3.9: T. Nguyen, "Reasoning Quality," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.9. ISBN: 9798244538229.
  5. Ch3.10: T. Nguyen, "Efficiency Metrics," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.10. ISBN: 9798244538229.
  6. Ch4.1: T. Nguyen, "Introduction to AI Agent Deployment and Scaling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.1. ISBN: 9798244538229.
  7. Ch4.2: T. Nguyen, "Deployment and Scaling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.2. ISBN: 9798244538229.
  8. Ch4.4: T. Nguyen, "Performance Profiling and Optimization," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.4. ISBN: 9798244538229.
  9. Ch4.5: T. Nguyen, "NVIDIA NIM and Triton Inference Server," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.5. ISBN: 9798244538229.
  10. Ch4.6: T. Nguyen, "TensorRT-LLM and NVIDIA Fleet Command," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.6. ISBN: 9798244538229.
  11. Ch4.7: T. Nguyen, "Scaling Strategies," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.7. ISBN: 9798244538229.
  12. Ch6.2B: T. Nguyen, "Production Vector Database Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.2B. ISBN: 9798244538229.
  13. Ch6.3A: T. Nguyen, "ETL Pipeline Fundamentals," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.3A. ISBN: 9798244538229.
  14. Ch6.3B: T. Nguyen, "ETL Worked Example - Load Phase & Pipeline Integration," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.3B. ISBN: 9798244538229.
  15. Ch6.5: T. Nguyen, "Production RAG Systems," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.5. ISBN: 9798244538229.
  16. Ch7.2: T. Nguyen, "Performance Optimization and Production Monitoring," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.2. ISBN: 9798244538229.
  17. Ch7.6: T. Nguyen, "Multi-Instance GPU," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.6. ISBN: 9798244538229.
  18. Ch8.1: T. Nguyen, "Latency Metrics," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.1. ISBN: 9798244538229.
  19. Ch8.2A: T. Nguyen, "Error Rates and Reliability," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.2A. ISBN: 9798244538229.
  20. Ch8.2B: T. Nguyen, "NeMo Guardrails Integration," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.2B. ISBN: 9798244538229.
  21. Ch10.1: T. Nguyen, "Conversational UI," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 10.1. ISBN: 9798244538229.
  22. Ref7.16: "Production Monitoring and Operations for Agentic AI," unpublished reference note (16-Production-Monitoring-Operations.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  23. Ref8.04: "Data Quality and Drift Detection for Agent Systems," unpublished reference note (04-Data-Quality-Drift-Detection.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  24. Ref8.06: "Error Troubleshooting and Incident Response for Agent Systems," unpublished reference note (06-Error-Troubleshooting-Incident-Response.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  25. Ref8.08: "Model Updates and Maintenance Procedures," unpublished reference note (08-Model-Updates-Maintenance-Procedures.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
  26. Ref9.09: "Compliance Automation and Tools," unpublished reference note (09-Compliance-Automation-Tools.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note