Agentic AI Reference Architecture

Model Serving · Software component

Attention Kernel

Software componentModel ServingModelsVariation point (abstract)arc:AttentionKernel

An abstract accelerator kernel implementation that computes transformer self-attention over the current sequence and cached key-value projections.

Responsibility. Computes self-attention during model inference.

is invoked bydeployed onis monitored byreadsis configured byis monitored byis specialized byis specialized byis specialized byInference Server: is invoked byInference ServerGPU Node: deployed onGPU NodeInference Engine Profiler: is monitored byInference Engine ProfilerKV Cache Store: readsKV Cache StoreEngine Build Configuration: is configured byEngine Build ConfigurationKernel Profiler: is monitored byKernel ProfilerFused Block-wise Attention Kernel: is specialized byFused Block-wise Attenti…Standard Attention Kernel: is specialized byStandard Attention KernelSparse Attention Kernel: is specialized bySparse Attention Kernel
Direct neighbourhood (hover for relationship types)

Variants

VariantWhen to choose
Fused Block-wise Attention KernelChoose by default for any transformer deployment, especially long contexts on memory-bandwidth-limited GPUs; benchmark against standard attention for very short sequences.
Sparse Attention KernelChoose when much larger effective contexts are needed and occasionally missing long-range dependencies that full attention would capture is acceptable.
Standard Attention KernelChoose only when benchmarks show fused kernels are slower, e.g., very short sequences (under 256 tokens) with small batches.

Relationships

deployed on structural

is configured by structural

is invoked by dependency

reads dependency

is monitored by assurance

Quantitative guidance

As stated by the sources; verify before use.

Classification

Quality attributes
Performance efficiency (ISO/IEC 25010)

Sources

  1. Ch4.4: T. Nguyen, "Performance Profiling and Optimization," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.4. ISBN: 9798244538229.