Observability & Evaluation · Data artifact

Cost Rate Card

Data artifactObservability & EvaluationObservability & Evaluationarc:CostRateCard

A versioned table of unit prices (per-million input, output and cached input tokens per model, plus GPU-hour, memory, storage and network rates) used to convert metered usage into monetary cost.

Responsibility. Supplies the unit prices applied to metered usage when computing cost.

Also known as: Token pricing table, Cost model, Asymmetric token price list

Direct neighbourhood (hover for relationship types)

Relationships

configures structural

Token Cost Meter Ch8.3 Ref8.05

is read by dependency

Cost Chargeback Service Ref8.05

Design guidance

MUST price input, output and cached input tokens separately, because output tokens typically cost 4-5x input tokens.
SHOULD include infrastructure rates (GPU-hour, host memory, storage, network) so self-hosted and API inference share one cost model.

Quantitative guidance

As stated by the sources; verify before use.

Output tokens cost ~4:1 to 5:1 relative to input tokens, reflecting one forward pass per generated token versus single-pass input encoding (a 200-token output needs 200 forward passes) (Ch8.3).
GPT-4o: $2.50 per 1M input, $10.00 per 1M output, $1.25 per 1M cached input tokens; GPT-4o-mini: $0.15 input / $0.60 output per 1M (~17x cheaper input, ~16x cheaper output) (Ch8.3).
Customer-service example (67 input, 267 output tokens, $0.0028375/request): output is 94% of cost at 80% of tokens; a 20% output cut saves 18.7% of cost versus 1.1% for a 20% input cut (~17x more) (Ch8.3).
GPU rates: H100 $2-3/h, A100 $1.50-2/h, L40S $1.50-2/h; host RAM $0.05-0.10/GB/month; storage $0.10-0.25/GB/month; network $0.01-0.15/GB (Ref8.05).
Cost composition: infrastructure 60-70%, API and third-party 20-30%, operational 10-15% of total (Ref8.05).

Classification

Patterns: Asymmetric input/output token pricing
Technologies: OpenAI APIGPT-4oGPT-4o-mini
Quality attributes: Cost efficiency
Risks mitigated: Naive uniform-cost-per-token assumptions misdirecting optimization

Sources

Ch8.3: T. Nguyen, "Token Economics and Architecture," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.3. ISBN: 9798244538229.
Ref8.05: "Cost Optimization and Resource Monitoring for Agent Systems," unpublished reference note (05-Cost-Optimization-Resource-Monitoring.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note