Cognition · Software component
Token Uncertainty Scorer
Software componentCognitionCognition & Memoryarc:TokenUncertaintyScorer
A statistical hallucination detector that flags generated outputs as uncertain using token log-probabilities, response perplexity and response-length anomalies relative to expected patterns.
Responsibility. Flags likely hallucinated generations from model-internal uncertainty signals.
Also known as: Statistical evaluator, Confidence filtering, Perplexity scoring, Response length anomaly detection
Relationships
is invoked by dependency
receives data from dynamic
sends data to dynamic
triggers dynamic
guards control
- Answer Synthesizer abstract Ch3.10
is evaluated by assurance
Design guidance
- SHOULD monitor token probabilities and perplexity during generation and flag low-confidence outputs for further review.
- SHOULD calibrate perplexity thresholds, since novel but factual content also yields high perplexity (false positives).
Classification
- Patterns
- Defense-in-depth stage 2 (real-time)Token probability thresholds
- Quality attributes
- Functional suitability: correctness and validity (ISO/IEC 25010 | NIST AI RMF: valid)Performance efficiency (ISO/IEC 25010)
- Risks mitigated
- Confident fabrication by guessing
Sources
- Ch3.10: T. Nguyen, "Efficiency Metrics," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.10. ISBN: 9798244538229.