Observability & Evaluation · Software component
Reward Hacking Monitor
Software componentObservability & EvaluationObservability & Evaluationarc:RewardHackingMonitor
A training-time monitoring component that compares policy outputs across checkpoints to detect emerging reward-exploitation patterns such as excessive verbosity, sycophancy, confidence inflation and formulaic phrasing.
Responsibility. Detects reward hacking during and after RL optimization so training can be stopped or corrected.
Also known as: Behavioral monitoring, Reward exploitation detector
Relationships
sends data to dynamic
evaluates assurance
monitors assurance
Design guidance
- SHOULD run continuously during RL training, since reward hacking persists despite KL regularization and ensembles (Goodhart's Law).
- SHOULD surface high-reward-scoring responses for manual human review.
Classification
- Patterns
- Cross-version behavioural comparisonEarly stopping signal
- Risks mitigated
- Reward hackingExcessive verbositySycophantic agreementConfidence hackingExploitation of annotation artifacts
Sources
- Ch10.3: T. Nguyen, "RLHF Methodology," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 10.3. ISBN: 9798244538229.