Agentic AI Reference Architecture

Observability & Evaluation · Software component

Attribution Analyzer

Software componentObservability & EvaluationObservability & Evaluationarc:AttributionAnalyzer

An interpretability component that computes how much each input token or evidence component causally influenced a model output, producing attribution maps.

Responsibility. Attributes model outputs to the input tokens that influenced them.

Also known as: Attribution analysis, Post-hoc feature attribution, Interpretability analyzer, Feature importance, Saliency map generator, SHAP explainer, LIME explainer

is invoked bysends data toinvokesevaluatesis routed to byis evaluated byEvaluation Harness: is invoked byEvaluation HarnessExplanation Presenter: sends data toExplanation PresenterDecision Engine: invokesDecision EngineFoundation LLM: evaluatesFoundation LLMExplanation Method Selector: is routed to byExplanation Method Selec…Reasoning Faithfulness Tester: is evaluated byReasoning Faithfulness T…
Direct neighbourhood (hover for relationship types)

Relationships

invokes dependency

is invoked by dependency

is routed to by dynamic

sends data to dynamic

evaluates assurance

is evaluated by assurance

Design guidance

Classification

Patterns
Integrated gradientsAttention-based attributionPost-hoc model-agnostic explanationSHAP (Shapley-value attribution)LIME (local surrogate model)Saliency maps
Technologies
LIMESHAP
Quality attributes
Explainability (NIST AI RMF: explainable and interpretable)Functional suitability: correctness and validity (ISO/IEC 25010 | NIST AI RMF: valid)
Risks mitigated
Unfaithful post-hoc rationalizationsReliance on misleading evidenceSpurious learned patterns (e.g., attention on image borders or metadata)

Sources

  1. Ch3.6: T. Nguyen, "Trace Analysis and Execution Debugging," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.6. ISBN: 9798244538229.
  2. Ch9.3: T. Nguyen, "Sandboxing and Transparency Foundations," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 9.3. ISBN: 9798244538229.
  3. Ch10.5: T. Nguyen, "Human-over-the-Loop," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 10.5. ISBN: 9798244538229.
  4. Ref10.07: "Chapter 10 Summary: Human-AI Interaction and Oversight," unpublished reference note (07-Chapter-10-Summary.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note