Agentic AI Reference Architecture

Knowledge & Data · Software component

Embedding Service

Software componentKnowledge & DataKnowledge & DataVariation point (abstract)arc:EmbeddingService

A service that converts text or other media into vector embeddings for similarity search.

Responsibility. Converts content to vectors.

Also known as: Embedder, Embedding infrastructure, Embedding pipeline, Embedding generation service

writes; reads; is cached byis invoked byinvokeswritesis orchestrated byis specialized byis invoked byis invoked byis invoked byreceives data fromreceives data fromis invoked byis invoked byis invoked bydeployed onis specialized byis invoked byis specialized byEmbedding Cache: writes; reads; is cached byEmbedding CacheAgent Controller: is invoked byAgent ControllerInference Server: invokesInference ServerVector Index Store: writesVector Index StoreIngestion Pipeline Orchestrator: is orchestrated byIngestion Pipeline Orche…Text Embedding Service: is specialized byText Embedding ServiceVector Retriever: is invoked byVector RetrieverMemory Retriever: is invoked byMemory RetrieverCitation Verifier: is invoked byCitation VerifierSelf-Managed Vector Index Store: receives data fromSelf-Managed Vector Inde…Document Chunker: receives data fromDocument ChunkerMemory Consolidator: is invoked byMemory ConsolidatorKnowledge Retrieval Agent: is invoked byKnowledge Retrieval AgentQuery Complexity Classifier: is invoked byQuery Complexity Classif…CPU Compute Node: deployed onCPU Compute NodeSelf-Hosted GPU Embedding Service: is specialized bySelf-Hosted GPU Embeddin…Alignment-Score Fact Checker: is invoked byAlignment-Score Fact Che…Joint Multimodal Embedding Service: is specialized byJoint Multimodal Embeddi…+8 more (see relationships)
Direct neighbourhood (hover for relationship types)

Variants

VariantWhen to choose
CPU Embedding ServiceChoose only when GPU infrastructure is unavailable and query volume is low (below the few-thousand-queries-per-day GPU break-even); pair with an embedding cache.
Hosted Embedding API ServiceChoose when building initial development or baseline systems, when no data-sovereignty or air-gap constraint applies, and when per-token API cost is acceptable for the query volume.
Joint Multimodal Embedding ServiceChoose for rapid deployment on existing text RAG with mostly general imagery (photos, simple diagrams); avoid for information-dense charts needing precise values or OCR.
Modality-Specific Embedding ServiceChoose when each modality needs its best-suited embedding model (e.g., domain BERT for text, CLIP for images, DePlot for charts) and components must be upgradable independently.
Self-Hosted GPU Embedding ServiceChoose when data sovereignty, zero-trust or air-gapped operation is required, when long documents must be embedded, or when volume is high enough (beyond a few thousand queries daily) that GPU per-query efficiency beats API pricing; requires operating GPU and serving infrastructure.
Text Embedding ServiceChoose when all modalities are grounded to text (captions, linearized tables, transcripts) so a single text embedding model and text vector index suffice.

Relationships

deployed on structural

hosts structural

invokes dependency

is cached by dependency

is invoked by dependency

reads dependency

writes dependency

receives data from dynamic

is orchestrated by control

Design guidance

Quantitative guidance

As stated by the sources; verify before use.

Classification

Patterns
Batched embedding requests (batch size tuned per workload)OpenAI-compatible embedding API for drop-in provider portabilityDevelop on hosted API, deploy on self-hosted service
Technologies
NV EmbedOpenAI Embeddings APINVIDIA NeMo RetrieverNVIDIA NIMCohere Embed API
Quality attributes
Performance efficiency (ISO/IEC 25010)Cost efficiencyFlexibility (ISO/IEC 25010)
Risks mitigated
Per-call API overhead in high-volume ingestionVendor lock-in of embedding pipelines

Sources

  1. Ch1.3: T. Nguyen, "Multi-Agent Systems," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.3. ISBN: 9798244538229.
  2. Ch1.4: T. Nguyen, "Memory and Perception Systems," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.4. ISBN: 9798244538229.
  3. Ch1.6: T. Nguyen, "Stateful Orchestration - Pitfalls, Integration, and Synthesis," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.6. ISBN: 9798244538229.
  4. Ch1.7A: T. Nguyen, "Relational Reasoning with Knowledge Graphs - The Fundamentals, Integration, and Extraction," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.7A. ISBN: 9798244538229.
  5. Ch1.7B: T. Nguyen, "Relational Reasoning with Knowledge Graphs - Hybrid RAG+KG Integration," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.7B. ISBN: 9798244538229.
  6. Ch1.8: T. Nguyen, "Scalability and Production Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.8. ISBN: 9798244538229.
  7. Ch2.1: T. Nguyen, "Framework Landscape and Selection," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 2.1. ISBN: 9798244538229.
  8. Ch2.2: T. Nguyen, "LangGraph," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 2.2. ISBN: 9798244538229.
  9. Ch2.7: T. Nguyen, "Multimodal RAG Approaches," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 2.7. ISBN: 9798244538229.
  10. Ch3.2: T. Nguyen, "Compare Agent Performance Across Tasks and Datasets," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.2. ISBN: 9798244538229.
  11. Ch3.9: T. Nguyen, "Reasoning Quality," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.9. ISBN: 9798244538229.
  12. Ch3.10: T. Nguyen, "Efficiency Metrics," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.10. ISBN: 9798244538229.
  13. Ch4.7: T. Nguyen, "Scaling Strategies," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.7. ISBN: 9798244538229.
  14. Ch6.1: T. Nguyen, "Embeddings and RAG Fundamentals," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.1. ISBN: 9798244538229.
  15. Ch7.1A: T. Nguyen, "Advanced Implementation with Nvidia NEMO Framework and Nvlink," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.1A. ISBN: 9798244538229.
  16. Ch8.3: T. Nguyen, "Token Economics and Architecture," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.3. ISBN: 9798244538229.
  17. Ref2.07: NVIDIA Developer, "Building multimodal AI RAG with LlamaIndex, NVIDIA NIM, and Milvus | LLM app development," YouTube. Accessed: Sep. 26, 2026. [Online Video]. Available: https://www.youtube.com/watch?v=NaT5Eo97_I0
  18. Ref7.15: "Advanced Agentic AI Optimization Techniques," unpublished reference note (15-Advanced-Agentic-Optimization.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note