Agentic AI Reference Architecture

Infrastructure · Software component

Session Affinity Load Balancer

Software componentInfrastructureInfrastructurearc:SessionAffinityLoadBalancer

A load balancer that pins all requests from the same client to the same replica so conversation context cached there is reused.

Responsibility. Keeps a client's follow-up requests on the replica holding its context.

Also known as: Sticky sessions, Client-IP affinity, IP hash, Consistent routing

Variant of Load Balancer abstract

When to choose. Choose as the simplest cache-locality option for conversational agents whose follow-up messages come from the same client.

routes tospecializesis target of alternativeTois target of alternativeTois target of alternativeTois target of alternativeToalternative tois target of alternativeToAgent Controller: routes toAgent ControllerLoad Balancer: specializesLoad BalancerLayer-4 Load Balancer: is target of alternativeToLayer-4 Load BalancerMetric-Aware Load Balancer: is target of alternativeToMetric-Aware Load BalancerCache-Aware Inference Router: is target of alternativeToCache-Aware Inference Ro…Least-Connections Load Balancer: is target of alternativeToLeast-Connections Load B…Weighted Round-Robin Load Balancer: alternative toWeighted Round-Robin Loa…Round-Robin Load Balancer: is target of alternativeToRound-Robin Load Balancer
Direct neighbourhood (hover for relationship types)

Relationships

routes to dynamic

alternative to variability

Design guidance

Quantitative guidance

As stated by the sources; verify before use.

Classification

Patterns
Session affinityIP-hash routingSticky sessions
Technologies
Kubernetes Service sessionAffinity: ClientIP
Quality attributes
Performance efficiency (ISO/IEC 25010)Maintainability (ISO/IEC 25010)
Risks mitigated
Context rebuild on follow-up messagesLocal cache misses for partially stateful agents

Sources

  1. Ch4.3: T. Nguyen, "Container Orchestration and Edge Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.3. ISBN: 9798244538229.
  2. Ch4.7: T. Nguyen, "Scaling Strategies," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.7. ISBN: 9798244538229.