Agentic AI Reference Architecture

Knowledge & Data · Software component

Modality-Specific Embedding Service

Software componentKnowledge & DataKnowledge & Dataarc:ModalitySpecificEmbeddingService

An embedding service that uses a separately chosen, modality-optimised embedding model for each content type, writing to that modality's own vector store.

Responsibility. Embeds each modality with its own best-suited model.

Also known as: Separate embedders, Per-modality embedding

Variant of Embedding Service abstract

When to choose. Choose when each modality needs its best-suited embedding model (e.g., domain BERT for text, CLIP for images, DePlot for charts) and components must be upgradable independently.

alternative tospecializesis target of alternativeTois invoked bywritesText Embedding Service: alternative toText Embedding ServiceEmbedding Service: specializesEmbedding ServiceJoint Multimodal Embedding Service: is target of alternativeToJoint Multimodal Embeddi…Per-Modality Fan-Out Retriever: is invoked byPer-Modality Fan-Out Ret…Modality-Specific Vector Store: writesModality-Specific Vector…
Direct neighbourhood (hover for relationship types)

Relationships

is invoked by dependency

writes dependency

alternative to variability

Classification

Patterns
Separate stores with cross-modal reranking
Technologies
BERTCLIPDePlot
Quality attributes
Maintainability (ISO/IEC 25010)Functional suitability: correctness and validity (ISO/IEC 25010 | NIST AI RMF: valid)

Sources

  1. Ch2.7: T. Nguyen, "Multimodal RAG Approaches," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 2.7. ISBN: 9798244538229.