Agentic AI Reference Architecture

Infrastructure · Software component

Policy Plugin Gateway

Software componentInfrastructureInfrastructurearc:PolicyPluginGateway

An API gateway proxy that runs an ordered, configurable plugin chain (authentication, rate limiting, transformation, logging) on each request before routing it to backend agents.

Responsibility. Enforces composable API-management policies on every request.

Also known as: API management gateway, Plugin-based API gateway, AI gateway, Action constraint gateway, Infrastructure-level enforcement gateway

Variant of API Gateway Proxy abstract

When to choose. Choose when centralised policy management, varied authentication schemes (OAuth2, JWT, API keys), traffic transformation or complex routing justify added latency and configuration complexity.

guards; routes toinvokes; hostsis invoked byemits telemetry tohostsinvokeshostscontrols access tospecializesinvokesis target of alternativeToExternal Service API: guards; routes toExternal Service APIRate Limiter: invokes; hostsRate LimiterAgent Controller: is invoked byAgent ControllerAudit Log Store: emits telemetry toAudit Log StoreRetry Handler: hostsRetry HandlerAuthorization Policy Decision Point: invokesAuthorization Policy Dec…Circuit Breaker: hostsCircuit BreakerAgent Service API: controls access toAgent Service APIAPI Gateway Proxy: specializesAPI Gateway ProxyParameter Security Validator: invokesParameter Security Valid…High-Performance Reverse Proxy: is target of alternativeToHigh-Performance Reverse…
Direct neighbourhood (hover for relationship types)

Relationships

hosts structural

invokes dependency

is invoked by dependency

emits telemetry to dynamic

routes to dynamic

controls access to control

guards control

alternative to variability

Design guidance

Quantitative guidance

As stated by the sources; verify before use.

Classification

Patterns
Plugin pipelineDeclarative configuration as code (GitOps)Weighted load balancingPolicy decision/enforcement separationCentralized policy enforcement pointSequential validation chain (authorization, rate limiting, parameter validation)Defense-in-depth Layer 1
Technologies
Kong GatewayNGINX (base)Open Policy Agent
Quality attributes
Maintainability (ISO/IEC 25010)Security (ISO/IEC 25010 | NIST AI RMF: secure and resilient)
Risks mitigated
Abuse via unthrottled consumersCascade failuresExcessive agencyMisconfigured or malicious agent requests reaching backend systems

Sources

  1. Ch4.1: T. Nguyen, "Introduction to AI Agent Deployment and Scaling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.1. ISBN: 9798244538229.
  2. Ch9.2: T. Nguyen, "Action Constraints and Permission Models," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 9.2. ISBN: 9798244538229.