Observability & Evaluation · Data artifact
Regression Threshold Policy
Data artifactObservability & EvaluationObservability & Evaluationarc:RegressionThresholdPolicy
A version-controlled configuration of per-metric absolute minimums and maximum allowed regressions that a regression gate enforces.
Responsibility. Declares the acceptable metric bounds for promoting an agent change.
Also known as: Evaluation thresholds, Rollback trigger thresholds
Relationships
configures structural
receives data from dynamic
constrains control
Design guidance
- SHOULD live in a configuration file rather than being hard-coded in CI workflows.
- SHOULD be tightened as the system matures; weakening MUST require documented justification.
Quantitative guidance
As stated by the sources; verify before use.
- Offline gate: accuracy drop <=2%, P95 latency increase <=20%, error rate <5% (Ch3.1A).
- Example file: accuracy min 0.80 / max -2%; P95 max 3,000 ms / +20%; empathy min 0.50 / -0.10; compliance min 0.95 / -5% (Ch3.1B).
- Two-tier example: > 0.80 proceeds automatically, 0.75-0.80 requires manual review, < 0.75 blocks deployment (Ch4.2).
- Registry flags versions whose correctness drops > 5% below the previous version (Ch4.2).
Sources
- Ch3.1A: T. Nguyen, "Implement Evaluation Pipelines and Task Benchmarks," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.1A. ISBN: 9798244538229.
- Ch3.1B: T. Nguyen, "Implement Evaluation Pipelines and Task Benchmarks - Guided Practice," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.1B. ISBN: 9798244538229.
- Ch4.1: T. Nguyen, "Introduction to AI Agent Deployment and Scaling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.1. ISBN: 9798244538229.
- Ch4.2: T. Nguyen, "Deployment and Scaling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.2. ISBN: 9798244538229.
- Ch4.4: T. Nguyen, "Performance Profiling and Optimization," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.4. ISBN: 9798244538229.
- Ch7.3: T. Nguyen, "NeMo Agent Toolkit Profiling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.3. ISBN: 9798244538229.