Observability & Evaluation · Software component

Log Aggregator

Software componentObservability & EvaluationObservability & Evaluationarc:LogAggregator

Relationships

Design guidance

• MUST be in place before operating many horizontally scaled replicas.
• MUST collect structured logs from all services in a unified system with consistent timestamps, tagged with version, commit SHA and pipeline run ID.
• MUST collect structured logs from every replica when requests may traverse different replicas, enabling reconstruction of failure scenarios.
• SHOULD log every query with retrieved chunks and scores, the generated response with citations, and a per-stage latency breakdown in structured (JSON) form.
• SHOULD apply retention periods (e.g., 7-30 days detailed logs, years for aggregated metrics).
• SHOULD tail container logs, parse JSON fields, enrich with cluster/namespace metadata and forward to a central store.

Classification

Technologies

ELK stack (Elasticsearch, Logstash, Kibana)Amazon CloudWatchDatadogElasticsearchELKSplunkLokiFluentdLogstash

Quality attributes

Maintainability (ISO/IEC 25010)Transparency and accountability (NIST AI RMF: accountable and transparent)

Risks mitigated

Fragmented diagnostics across many podsHours-long manual log correlation across containers

Sources

1. Ch1.3: T. Nguyen, "Multi-Agent Systems," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.3. ISBN: 9798244538229.
2. Ch1.8: T. Nguyen, "Scalability and Production Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.8. ISBN: 9798244538229.
3. Ch4.1: T. Nguyen, "Introduction to AI Agent Deployment and Scaling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.1. ISBN: 9798244538229.
4. Ch4.2: T. Nguyen, "Deployment and Scaling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.2. ISBN: 9798244538229.
5. Ch4.7: T. Nguyen, "Scaling Strategies," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.7. ISBN: 9798244538229.
6. Ch6.3B: T. Nguyen, "ETL Worked Example - Load Phase & Pipeline Integration," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.3B. ISBN: 9798244538229.
7. Ch6.5: T. Nguyen, "Production RAG Systems," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.5. ISBN: 9798244538229.
8. Ch7.2: T. Nguyen, "Performance Optimization and Production Monitoring," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.2. ISBN: 9798244538229.
9. Ch10.4: T. Nguyen, "Human-in-the-Loop," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 10.4. ISBN: 9798244538229.
10. Ref4.08: NVIDIA, "Agentic AI in the Factory," NVIDIA Enterprise AI Factory Design Guide White Paper. Accessed: Sep. 27, 2026. [Online]. Available: https://docs.nvidia.com/ai-enterprise/planning-resource/ai-factory-white-paper/latest/agentic-ai-in-the-factory.html
11. Ref7.16: "Production Monitoring and Operations for Agentic AI," unpublished reference note (16-Production-Monitoring-Operations.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
12. Ref9.09: "Compliance Automation and Tools," unpublished reference note (09-Compliance-Automation-Tools.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note