Cognition · Software component
Reasoning Path Sampler
Software componentCognitionCognition & Memoryarc:ReasoningPathSampler
A cognition component that generates k independent complete chain-of-thought reasoning paths for one problem using stochastic decoding (temperature, top-k, nucleus sampling) instead of greedy decoding.
Responsibility. Generates diverse independent reasoning paths for the same problem.
Also known as: Self-Consistency sampler, Stochastic CoT sampler, Diverse reasoning path generator
Relationships
is configured by structural
invokes dependency
writes dependency
- Working Memory Buffer abstract Ch5.3
is routed to by dynamic
is triggered by dynamic
sends data to dynamic
is monitored by assurance
Design guidance
- MUST use stochastic decoding at moderate temperature (about 0.7-1.0) so samples differ in strategy, not merely wording; greedy or very low temperature yields near-identical paths with no error-correction benefit.
- SHOULD tune temperature, top-k, top-p and k empirically on representative problems from the target domain rather than copying defaults.
- SHOULD generate samples in parallel through batched inference to bound latency in interactive applications.
- SHOULD start with the strongest base model the budget permits and add sampling only if its accuracy is insufficient, rather than compensating a weak model with more samples.
Quantitative guidance
As stated by the sources; verify before use.
- k samples cost ~k times the output tokens of single-path CoT; GSM8K example: $0.0042 (CoT) vs ~$0.0120 (k=5) per problem, +186%; $153,300 vs $438,000 per year at 100,000 daily queries (Ch5.3).
- k=5 sequential samples raise a 4 s CoT generation to ~20 s; with batching, a 200% token increase produced ~50% latency increase on GSM8K (Ch5.3).
- Default settings temperature 0.7, top-k 40, k=40; top-p typically 0.9 (Ch5.3).
Classification
- Patterns
- Self-Consistency generation phaseTemperature samplingTop-k samplingNucleus (top-p) samplingParallel batched samplingStreaming generate-and-discard
- Quality attributes
- Functional suitability: correctness and validity (ISO/IEC 25010 | NIST AI RMF: valid)Reliability (ISO/IEC 25010 | NIST AI RMF: valid and reliable)Cost efficiencyPerformance efficiency (ISO/IEC 25010)
- Risks mitigated
- Greedy-decoding commitment to an early erroneous reasoning path
Sources
- Ch5.3: T. Nguyen, "Self-Consistency Fundamentals," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 5.3. ISBN: 9798244538229.