Observability & Evaluation · Human role
Ground Truth Annotator
Human roleObservability & EvaluationObservability & Evaluationarc:GroundTruthAnnotator
A domain expert who labels test queries with known correct answers and authors edge-case and adversarial examples for evaluation datasets.
Responsibility. Supplies expert ground-truth labels and curated edge cases for offline evaluation.
Also known as: Dataset curator, Expert labeler
Relationships
sends data to dynamic
Quantitative guidance
As stated by the sources; verify before use.
- Example: ground-truth labels for 1,000 queries needed before offline testing (Ch3.1A).
Sources
- Ch3.1A: T. Nguyen, "Implement Evaluation Pipelines and Task Benchmarks," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.1A. ISBN: 9798244538229.
- Ch7.3: T. Nguyen, "NeMo Agent Toolkit Profiling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.3. ISBN: 9798244538229.