Infrastructure · Software component
Resource-Utilization Autoscaler
Software componentInfrastructureInfrastructurearc:ResourceUtilizationAutoscaler
An autoscaler that adjusts replica counts to keep average CPU and/or memory utilization of a workload near declared targets, scaling on whichever resource approaches saturation.
Responsibility. Scales replicas to hold per-replica CPU or memory utilization at target levels.
Also known as: CPU-based autoscaling, Memory-based autoscaling, Dual-metric CPU/memory HPA
Variant of Autoscaler abstract
When to choose. Choose CPU for compute-bound agents dominated by inference or reasoning loops, and memory for agents holding large context windows or document embeddings.
Relationships
is configured by structural
invokes dependency
scales control
- Agent Controller abstract Ref8.05
alternative to variability
Design guidance
- SHOULD NOT be used alone for agents that mostly wait on external APIs; CPU may sit near 20% while the agent is at full capacity.
- SHOULD serve as baseline scaling on simple built-in CPU/memory metrics.
Quantitative guidance
As stated by the sources; verify before use.
- Worked example targets 70% CPU and 75% memory utilization (Ch4.7).
- Dynamic sizing (40 GPUs peak, 4 average, 1 off-peak; weighted 12) vs static 40 GPUs: $1,920/day -> $240/day (87.5%); typical savings 50-70% (Ref8.05).
Classification
- Patterns
- Target-utilization scalingDual-metric (max of CPU, memory) scaling
- Technologies
- Kubernetes Horizontal Pod Autoscaler (Resource metrics)Kubernetes HorizontalPodAutoscaler
- Quality attributes
- Performance efficiency (ISO/IEC 25010)Maintainability (ISO/IEC 25010)
- Risks mitigated
- Resource-specific bottlenecks (CPU or memory saturation)
Sources
- Ch4.7: T. Nguyen, "Scaling Strategies," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.7. ISBN: 9798244538229.
- Ref7.17: "Scaling Agentic AI Systems: Patterns and Strategies," unpublished reference note (17-Scalability-Patterns.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
- Ref8.05: "Cost Optimization and Resource Monitoring for Agent Systems," unpublished reference note (05-Cost-Optimization-Resource-Monitoring.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note