Agentic AI Reference Architecture

Memory · Data store

Experience Replay Buffer

Data storeMemoryCognition & Memoryarc:ExperienceReplayBuffer

A store of past experience transitions (state, action, reward, next state) from which batches are sampled to retrain learned models, mixing old and new experience.

Responsibility. Retains past experience transitions for repeated sampling during training.

Also known as: Replay buffer, Episodic replay memory, Replay memory

is written by; is read byis read by; is written byis written byis read byis written byReinforcement Learning Policy Learner: is written by; is read byReinforcement Learning P…Hindsight Goal Relabeler: is read by; is written byHindsight Goal RelabelerLearned-Policy Decision Engine: is written byLearned-Policy Decision …Continual Learning Trainer: is read byContinual Learning TrainerEvent-Based Episode Encoder: is written byEvent-Based Episode Enco…
Direct neighbourhood (hover for relationship types)

Relationships

is read by dependency

is written by dependency

Quantitative guidance

As stated by the sources; verify before use.

Classification

Patterns
Experience replayMixed old/new mini-batch samplingUniform experience replayPrioritized experience replay (by TD error)Hindsight experience replay
Quality attributes
Performance efficiency (ISO/IEC 25010)Interaction capability (ISO/IEC 25010)Reliability (ISO/IEC 25010 | NIST AI RMF: valid and reliable)
Risks mitigated
Catastrophic forgettingOverfitting to recent experience patternsTemporally correlated training samplesForgetting earlier learned patterns

Sources

  1. Ch5.7: T. Nguyen, "Episodic Memory," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 5.7. ISBN: 9798244538229.
  2. Ch5.12: T. Nguyen, "Learning-Based Decision Making Fundamentals," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 5.12. ISBN: 9798244538229.