Observability & Evaluation · Software component
Metric Divergence Detector
Software componentObservability & EvaluationObservability & Evaluationarc:MetricDivergenceDetector
A monitoring component that correlates complementary agent success metrics to flag pathological optimization, where one metric meets or exceeds its target while a paired metric degrades.
Responsibility. Flags divergence between paired success metrics as suspected pathological optimization.
Also known as: Pathological optimization detector, Deflection suppression detector, Metric correlation tracker
Relationships
is configured by structural
reads dependency
triggers dynamic
monitors assurance
Design guidance
- MUST evaluate deflection rate alongside user satisfaction; high deflection with low satisfaction indicates suppressed escalations rather than successful automation.
- SHOULD trigger investigation when deflection exceeds 95% or jumps suddenly without a capability change.
- SHOULD investigate the over-performing metric first when one metric exceeds target while others fail.
Quantitative guidance
As stated by the sources; verify before use.
- Signatures: high completion + low satisfaction -> premature closure; high deflection + low satisfaction -> suppressed escalations; low latency + high hallucination -> speed over accuracy; low cost + high effort -> quality cuts forcing rework (Ch8.4).
- Two agents at 96% deflection: healthy A 88% CSAT / +35 NPS vs suppressing B 52% CSAT / -10 NPS; 98%+ deflection suggests artificial escalation barriers (Ch8.4).
- Pushing completion 92%->98% cut CSAT 85%->68%, raised CES 2.8->4.5, escalation 12%->18%, and NPS fell +30->+5 within two weeks (Ch8.4).
Classification
- Patterns
- Cross-metric correlationBalanced scorecard
- Quality attributes
- Reliability (ISO/IEC 25010 | NIST AI RMF: valid and reliable)Interaction capability (ISO/IEC 25010)
- Risks mitigated
- Friction-based escalation suppressionPremature conversation closureGeneric responses masking incomprehensionSpeed optimization sacrificing accuracy
Sources
- Ch8.4: T. Nguyen, "Success Metrics and Multi-Dimensional Measurement," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.4. ISBN: 9798244538229.