Memory-Optimized Deployment Profile

A quantized model deployment profile that reduces memory footprint so a model fits smaller GPUs or edge devices, possibly at some cost in throughput and latency.

Responsibility. Configures an inference microservice to fit constrained GPU memory.

Also known as: Quantized profile, Edge profile

Variant of Model Deployment Profile abstract

When to choose. Choose when available GPU memory is insufficient for full-precision weights plus activations, such as smaller GPUs or edge devices.