Model Serving · Software component

Model Pruner

Software componentModel ServingModelsarc:ModelPruner

A model optimisation component that removes low-contribution weights, neurons or layers and briefly fine-tunes to recover accuracy until a target sparsity is reached.

Responsibility. Reduces model size and compute by eliminating redundant parameters.

Also known as: Pruning stage

Direct neighbourhood (hover for relationship types)

Relationships

optimizes lifecycle

Edge-Optimized Model Ch4.3
Foundation LLM abstract Ref7.05 Ref7.15 +1

Design guidance

SHOULD prefer structured pruning when speedups are needed without specialised sparse kernels.
SHOULD prefer structured 2:4 sparsity where hardware acceleration is required; unstructured sparsity needs specialised kernels with limited hardware support (Ref7.05).

Quantitative guidance

As stated by the sources; verify before use.

Typical target sparsity 50-90%; 70% pruned + INT8 model ~10x smaller at ~98% accuracy (Ch4.3).

Classification

Patterns: Structured pruningUnstructured pruningIterative magnitude pruning2:4 structured sparsityUnstructured sparsity
Quality attributes: Performance efficiency (ISO/IEC 25010)
Risks mitigated: Model exceeding edge memory budget

Sources

Ch4.3: T. Nguyen, "Container Orchestration and Edge Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.3. ISBN: 9798244538229.
Ref7.05: S. Verma and N. Vaidya, "Mastering LLM Techniques: Inference Optimization," NVIDIA Technical Blog, Nov. 17, 2023. [Online]. Available: https://developer.nvidia.com/blog/mastering-llm-techniques-inference-optimization/
Ref7.15: "Advanced Agentic AI Optimization Techniques," unpublished reference note (15-Advanced-Agentic-Optimization.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
Ref7.18: "Chapter 7 Summary: NVIDIA Platform Implementation," unpublished reference note (18-Chapter-7-Summary.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note