Model Serving · Software component
Quantization Calibrator
Software componentModel ServingModelsarc:QuantizationCalibrator
A model optimisation component that runs representative inputs through a network to collect per-tensor activation statistics and computes quantization thresholds (scaling factors) minimising information loss.
Responsibility. Derives the scaling factors used to map full-precision activations to low-precision integers.
Also known as: Entropy calibrator, INT8 calibration, Entropy calibration, Calibration step
Relationships
is invoked by dependency
reads dependency
Design guidance
- MUST calibrate with data representative of the production input distribution, not generic web text.
- SHOULD use entropy (KL-divergence) calibration when accuracy is the priority; min-max trades extra accuracy loss for faster calibration; percentile clipping sits between them.
Quantitative guidance
As stated by the sources; verify before use.
- Entropy calibration of a 7B model takes 10-20 minutes; min-max takes 2-3 minutes but typically loses an additional 0.2-0.5% accuracy (Ch7.4).
- FP8 calibration example uses 512 calibration batches (512 forward passes) (Ch7.4).
Classification
- Patterns
- Entropy calibration (KL-divergence minimisation)Entropy (KL-divergence) calibrationMin-max calibrationPercentile calibrationE4M3 FP8 calibration
- Technologies
- TensorRT-LLMTensorRTNVIDIA TensorRT Model Optimizer (ModelOpt)
- Quality attributes
- Functional suitability: correctness and validity (ISO/IEC 25010 | NIST AI RMF: valid)
- Risks mitigated
- Accuracy loss from mis-set quantization thresholdsINT8 slower than FP16 from poor quantize-dequantize placement'No scaling factors detected' calibration failuresSuboptimal scaling factors amplifying quantization error
Sources
- Ch4.6: T. Nguyen, "TensorRT-LLM and NVIDIA Fleet Command," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.6. ISBN: 9798244538229.
- Ch7.4: T. Nguyen, "TensorRT-LLM Fundamentals and Quantization," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.4. ISBN: 9798244538229.