Agentic AI Reference Architecture

Model Serving · Model asset

Edge-Optimized Model

Model assetModel ServingModelsarc:EdgeOptimizedModel

A compact model (quantized, pruned, distilled or efficiency-designed) packaged for a specific class of edge hardware within its memory and latency budget.

Responsibility. Delivers acceptable accuracy within edge device constraints.

Also known as: Compressed model, Student model

is optimized byis optimized bydeployed ondeployed onis trained byis trained byis evaluated byis optimized byis produced byEngine Builder: is optimized byEngine BuilderModel Quantizer: is optimized byModel QuantizerEdge GPU Device: deployed onEdge GPU DeviceEdge Inference Runtime: deployed onEdge Inference RuntimeKnowledge Distiller: is trained byKnowledge DistillerFederated Aggregator: is trained byFederated AggregatorEdge Validation Test Bench: is evaluated byEdge Validation Test BenchModel Pruner: is optimized byModel PrunerNeural Architecture Searcher: is produced byNeural Architecture Sear…
Direct neighbourhood (hover for relationship types)

Relationships

deployed on structural

is evaluated by assurance

is optimized by lifecycle

is produced by lifecycle

is trained by lifecycle

Design guidance

Quantitative guidance

As stated by the sources; verify before use.

Classification

Patterns
Progressive compressionHardware-targeted variants
Technologies
TensorRT engineCore ML packageTFLite modelNemotron 4B / 8B
Quality attributes
Performance efficiency (ISO/IEC 25010)

Sources

  1. Ch4.3: T. Nguyen, "Container Orchestration and Edge Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.3. ISBN: 9798244538229.
  2. Ref7.12: "Advanced Nemotron Deployment Patterns," unpublished reference note (12-Nemotron-Advanced-Deployment.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note