Agentic AI Reference Architecture

Cognition · Software component

Thought Generator

Software componentCognitionCognition & MemoryVariation point (abstract)arc:ThoughtGenerator

An abstract cognition component that prompts a language model to produce k candidate next thoughts from the current reasoning state, conditioned on existing thoughts.

Responsibility. Produces multiple candidate next thoughts for a given reasoning state.

Also known as: Thought proposer, Generation transformation, Candidate thought generator

invokesreadsis orchestrated byis cached byis specialized byis specialized byis configured byLLM Inference Service: invokesLLM Inference ServiceEpisodic Memory Store: readsEpisodic Memory StoreThought Exploration Controller: is orchestrated byThought Exploration Cont…Reasoning Chain Cache: is cached byReasoning Chain CacheIndependent Sampling Thought Generator: is specialized byIndependent Sampling Tho…Sequential Proposal Thought Generator: is specialized bySequential Proposal Thou…Thought Decomposition Specification: is configured byThought Decomposition Sp…
Direct neighbourhood (hover for relationship types)

Variants

VariantWhen to choose
Independent Sampling Thought GeneratorChoose for open-ended exploration of vast, unstructured solution spaces where large conceptual leaps are needed (e.g., creative writing plans).
Sequential Proposal Thought GeneratorChoose for highly constrained, structured solution spaces where candidates should collectively and systematically cover the space (e.g., arithmetic operations).

Relationships

is configured by structural

invokes dependency

is cached by dependency

reads dependency

is orchestrated by control

Design guidance

Classification

Patterns
Tree-of-ThoughtGraph-of-Thought generation transformation
Quality attributes
Functional suitability: correctness and validity (ISO/IEC 25010 | NIST AI RMF: valid)

Sources

  1. Ch5.2: T. Nguyen, "Tree-of-Thought (ToT) Fundamentals," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 5.2. ISBN: 9798244538229.