Latency-Optimized Deployment Profile

A model deployment profile that minimises time-to-first-token, e.g., FP16 precision on a single GPU, sacrificing throughput.

Responsibility. Configures an inference microservice for fastest initial response.

Also known as: vllm-fp16-latency-optimized, Low-latency profile, Single-GPU profile

Variant of Model Deployment Profile abstract

When to choose. Choose for latency-critical interactive applications where time-to-first-token SLOs dominate.