Agentic AI Reference Architecture

Observability & Evaluation · Software component

Trajectory Scorer

Software componentObservability & EvaluationObservability & Evaluationarc:TrajectoryScorer

A response scorer that evaluates how an agent reached its result, assessing reasoning-chain soundness, tool-selection sequence, strategy adaptation and decision transparency from execution traces.

Responsibility. Scores the quality of an agent's reasoning and action process rather than only its outcome.

Also known as: Process evaluator, Trajectory evaluation, Trajectory Evaluator

Variant of Response Scorer abstract

When to choose. Choose when deployment requires auditability, interpretability or trust calibration, not just correct outcomes.

is invoked byreadsinvokesspecializesevaluatesEvaluation Harness: is invoked byEvaluation HarnessTrace Store: readsTrace StoreLLM Judge: invokesLLM JudgeResponse Scorer: specializesResponse ScorerTool Selector: evaluatesTool Selector
Direct neighbourhood (hover for relationship types)

Relationships

invokes dependency

is invoked by dependency

reads dependency

evaluates assurance

Design guidance

Classification

Patterns
Process evaluation in addition to outcome evaluationLLM-as-a-judge over intermediate steps
Technologies
NVIDIA Agent Intelligence Toolkit
Quality attributes
Transparency and accountability (NIST AI RMF: accountable and transparent)Explainability (NIST AI RMF: explainable and interpretable)Maintainability (ISO/IEC 25010)
Risks mitigated
Poor tool selection hidden behind acceptable final answers

Sources

  1. Ch3.2: T. Nguyen, "Compare Agent Performance Across Tasks and Datasets," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.2. ISBN: 9798244538229.
  2. Ref3.01: NVIDIA, "Agent Evaluation in NVIDIA NeMo Agent Toolkit," NVIDIA NeMo Agent Toolkit Documentation, v1.8. Accessed: Sep. 27, 2026. [Online]. Available: https://docs.nvidia.com/nemo/agent-toolkit/latest/improve-workflows/evaluate.html