Agentic AI Reference Architecture

Infrastructure · Software component

Layer-4 Load Balancer

Software componentInfrastructureInfrastructurearc:L4LoadBalancer

A transport-layer load balancer that forwards TCP/UDP connections to replicas using only IP/port information, without inspecting application payloads.

Responsibility. Forwards connections to replicas based solely on transport-layer information.

Also known as: Transport-layer load balancer, Cluster service load balancer, Standard cluster service load balancing, Kubernetes ClusterIP Service, kube-proxy (iptables/IPVS), ClusterIP Service

Variant of Load Balancer abstract

When to choose. Choose when all instances are identical and all requests equivalent, so raw throughput and simplicity outweigh request-aware routing.

alternative to; is routed to byroutes todeployed onroutes tospecializesreceives data fromalternative tois target of alternativeToreadsLayer-7 Load Balancer: alternative to; is routed to byLayer-7 Load BalancerLLM Inference Service: routes toLLM Inference ServiceContainer Orchestrator: deployed onContainer OrchestratorSupervisor Agent: routes toSupervisor AgentLoad Balancer: specializesLoad BalancerAgent API Gateway: receives data fromAgent API GatewaySession Affinity Load Balancer: alternative toSession Affinity Load Ba…Cache-Aware Inference Router: is target of alternativeToCache-Aware Inference Ro…Service Discovery Registry: readsService Discovery Registry
Direct neighbourhood (hover for relationship types)

Relationships

deployed on structural

reads dependency

is routed to by dynamic

receives data from dynamic

routes to dynamic

alternative to variability

Design guidance

Quantitative guidance

As stated by the sources; verify before use.

Classification

Patterns
Round-robinRandom selectionClient-IP session affinityRandom pod selection (approximate round robin)
Technologies
Kubernetes Service (type LoadBalancer / ClusterIP)Kubernetes Service (type LoadBalancer)Kubernetes kube-proxyiptablesIPVS
Quality attributes
Performance efficiency (ISO/IEC 25010)Maintainability (ISO/IEC 25010)

Sources

  1. Ch1.8: T. Nguyen, "Scalability and Production Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.8. ISBN: 9798244538229.
  2. Ch4.3: T. Nguyen, "Container Orchestration and Edge Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.3. ISBN: 9798244538229.
  3. Ch4.5: T. Nguyen, "NVIDIA NIM and Triton Inference Server," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.5. ISBN: 9798244538229.
  4. Ch4.7: T. Nguyen, "Scaling Strategies," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.7. ISBN: 9798244538229.