Agentic AI Reference Architecture

Cognition · Software component

Thought State Evaluator

Software componentCognitionCognition & MemoryVariation point (abstract)arc:ThoughtStateEvaluator

An abstract cognition component that uses a language model as a heuristic to assess how promising intermediate reasoning states are, producing signals that guide pruning and selection.

Responsibility. Scores the promise of candidate reasoning states for search guidance.

Also known as: State evaluator, Heuristic evaluator, Thought scorer, Scoring and evaluation mechanism

invokesreadsis orchestrated bywritesis cached byis specialized byis configured byis specialized byis configured byLLM Inference Service: invokesLLM Inference ServiceSemantic Memory Store: readsSemantic Memory StoreThought Exploration Controller: is orchestrated byThought Exploration Cont…Graph Reasoning State Store: writesGraph Reasoning State St…Reasoning Chain Cache: is cached byReasoning Chain CacheValue Thought Evaluator: is specialized byValue Thought EvaluatorThought Decomposition Specification: is configured byThought Decomposition Sp…Vote Thought Evaluator: is specialized byVote Thought EvaluatorThought Evaluation Prompt Template: is configured byThought Evaluation Promp…
Direct neighbourhood (hover for relationship types)

Variants

VariantWhen to choose
Value Thought EvaluatorChoose when intermediate states can be checked for objective feasibility (e.g., mathematical reachability, constraint satisfaction) and absolute thresholds are needed for pruning.
Vote Thought EvaluatorChoose when evaluation criteria are subjective or rating scales poorly defined (e.g., narrative coherence), or when a single best candidate must be selected.

Relationships

is configured by structural

invokes dependency

is cached by dependency

reads dependency

writes dependency

is orchestrated by control

Design guidance

Quantitative guidance

As stated by the sources; verify before use.

Classification

Patterns
LLM self-evaluation as search heuristicHybrid value-then-vote evaluationEnsemble-voted value estimation
Quality attributes
Performance efficiency (ISO/IEC 25010)Functional suitability: correctness and validity (ISO/IEC 25010 | NIST AI RMF: valid)
Risks mitigated
Wasted exploration of unproductive branches

Sources

  1. Ch5.2: T. Nguyen, "Tree-of-Thought (ToT) Fundamentals," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 5.2. ISBN: 9798244538229.