Agentic AI Reference Architecture

Model Serving · Model asset

Vision-Language Model

Model assetModel ServingModelsVariation point (abstract)arc:VisionLanguageModel

A generative multimodal model combining a vision encoder with a language model via cross-attention, producing captions and answers about images, including reading visible text.

Responsibility. Generates natural-language descriptions and answers grounded in image content.

Also known as: Multimodal LLM, VLM, Neva

deployed onis optimized byis optimized byis specialized byis specialized byis specialized byInference Server: deployed onInference ServerEngine Builder: is optimized byEngine BuilderModel Quantizer: is optimized byModel QuantizerBalanced Vision-Language Model: is specialized byBalanced Vision-Language…Edge Vision-Language Model: is specialized byEdge Vision-Language ModelHigh-Accuracy Vision-Language Model: is specialized byHigh-Accuracy Vision-Lan…
Direct neighbourhood (hover for relationship types)

Variants

VariantWhen to choose
Balanced Vision-Language ModelChoose for cloud deployment needing a balance of accuracy and compute.
Edge Vision-Language ModelChoose for edge deployment on embedded GPU devices where memory and power are constrained.
High-Accuracy Vision-Language ModelChoose for demanding applications requiring the highest visual reasoning accuracy.

Relationships

deployed on structural

is optimized by lifecycle

Quantitative guidance

As stated by the sources; verify before use.

Classification

Patterns
Cross-attention vision-language fusionVision encoder -> projection layer -> LLM decoder
Technologies
NVIDIA NeVA 22BGPT-4VNVIDIA Neva

Sources

  1. Ch2.7: T. Nguyen, "Multimodal RAG Approaches," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 2.7. ISBN: 9798244538229.
  2. Ch7.5: T. Nguyen, "NeMo Curator, Riva Speech AI & Multimodal Integration," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.5. ISBN: 9798244538229.
  3. Ch10.1: T. Nguyen, "Conversational UI," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 10.1. ISBN: 9798244538229.
  4. Ref2.07: NVIDIA Developer, "Building multimodal AI RAG with LlamaIndex, NVIDIA NIM, and Milvus | LLM app development," YouTube. Accessed: Sep. 26, 2026. [Online Video]. Available: https://www.youtube.com/watch?v=NaT5Eo97_I0