Agentic AI Reference Architecture

Governance & Compliance · Software component

Counterfactual Fairness Tester

Software componentGovernance & ComplianceSafety, Security & Governancearc:CounterfactualFairnessTester

An audit component that submits paired test cases differing only in a sensitive attribute (audit-study style) to a decision model and compares the resulting decisions.

Responsibility. Tests whether decisions change when only a protected attribute changes.

Also known as: Audit study tester, Counterfactual fairness assessment

invokesevaluatesproducesreadsInference Server: invokesInference ServerPredictive Decision Model: evaluatesPredictive Decision ModelFairness Audit Report: producesFairness Audit ReportDemographic Audit Dataset: readsDemographic Audit Dataset
Direct neighbourhood (hover for relationship types)

Relationships

invokes dependency

reads dependency

evaluates assurance

produces lifecycle

Quantitative guidance

As stated by the sources; verify before use.

Classification

Patterns
Counterfactual fairnessAudit study methodology
Quality attributes
Fairness (NIST AI RMF: fair, harmful bias managed)
Risks mitigated
Hidden disparity missed by metrics on organic data due to confounding

Sources

  1. Ch9.4: T. Nguyen, "Fairness and Bias Mitigation," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 9.4. ISBN: 9798244538229.