Observability & Evaluation · Data artifact

Evaluation Configuration

Data artifactObservability & EvaluationObservability & Evaluationarc:EvaluationConfig

A declarative configuration specifying an evaluation's dataset source, evaluators with their metric names and judge LLMs, and output location, overridable at runtime.

Responsibility. Defines what an evaluation run measures, with which judges and data.

Also known as: eval_config.yml

Direct neighbourhood (hover for relationship types)

Relationships

configures structural

Evaluation Harness Ref3.01

Design guidance

SHOULD declare dataset, evaluators (type, metric, judge LLM) and output sections.

Classification

Patterns: Configuration over customization
Technologies: NVIDIA Agent Intelligence Toolkit
Quality attributes: Reliability (ISO/IEC 25010 | NIST AI RMF: valid and reliable)Maintainability (ISO/IEC 25010)

Sources

Ref3.01: NVIDIA, "Agent Evaluation in NVIDIA NeMo Agent Toolkit," NVIDIA NeMo Agent Toolkit Documentation, v1.8. Accessed: Sep. 27, 2026. [Online]. Available: https://docs.nvidia.com/nemo/agent-toolkit/latest/improve-workflows/evaluate.html