Agentic AI Reference Architecture

Cognition · Software component

Markov State Augmenter

Software componentCognitionCognition & Memoryarc:MarkovStateAugmenter

A state-representation component that enriches the current decision state with sufficient interaction history for the augmented state to satisfy the Markov property.

Responsibility. Augments decision state with relevant history so future outcomes depend only on the augmented state.

Also known as: History-augmented state builder

writesreadssends data toWorld Model State: writesWorld Model StateUser Activity History Store: readsUser Activity History St…MDP Policy Solver: sends data toMDP Policy Solver
Direct neighbourhood (hover for relationship types)

Relationships

reads dependency

writes dependency

sends data to dynamic

Classification

Patterns
State augmentation with history
Risks mitigated
Suboptimal policies from Markov-assumption violations

Sources

  1. Ch5.10: T. Nguyen, "Utility-Based Decision Making Fundamentals," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 5.10. ISBN: 9798244538229.