Infrastructure · Software component

Weighted Round-Robin Load Balancer

Software componentInfrastructureInfrastructurearc:WeightedRoundRobinLoadBalancer

A load balancer that distributes requests in proportion to per-replica weights reflecting capacity, cost or rollout stage.

Responsibility. Routes traffic proportionally to configured replica weights.

Also known as: Weighted distribution, Capacity-aware routing

Variant of Load Balancer abstract

When to choose. Choose when replicas run on heterogeneous infrastructure with measurable capacity differences, or for weighted gradual rollouts.

Direct neighbourhood (hover for relationship types)

Relationships

SHOULD assign weights automatically (default or metadata-based) to replicas launched by autoscaling.
SHOULD monitor actual traffic share against configured weights to detect misconfiguration.
MAY be combined with least-connections routing.

As stated by the sources; verify before use.

Weights 4:1:1 send 67% of traffic to a GPU replica and 17% to each CPU replica (Ch4.7).
Document case: GPU replicas 3x faster; weight 3 vs 1 sent 60% to two GPU replicas and ~13% to each of three CPU replicas; with least connections, P95 26 s and 85% average GPU utilization (Ch4.7).

Patterns: Capacity-proportional routingCost-weighted routingWeighted gradual rollout (e.g., 5% new version)Overflow tier routing
Quality attributes: Performance efficiency (ISO/IEC 25010)Cost efficiency
Risks mitigated: Under-utilized expensive GPU replicasOverloaded low-capacity replicas

Ch4.7: T. Nguyen, "Scaling Strategies," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.7. ISBN: 9798244538229.