Model Serving · Software component
Query Complexity Classifier
Software componentModel ServingModelsarc:QueryComplexityClassifier
A lightweight classifier that predicts a query's complexity from its embedding to inform model-tier routing.
Responsibility. Predicts query complexity before inference to inform routing.
Also known as: Complexity classifier model, Complexity Classifier, SLM pre-screener, Initial triage model, Intent and complexity classifier, Difficulty classifier, Query complexity heuristic, Decomposition bypass check, LLM-based complexity classification, Small classifier model
Variant of Query Complexity Assessor abstract
When to choose. Choose when query phrasing varies widely and higher classification accuracy justifies the added latency and cost of a classifier model.
Relationships
invokes dependency
- Embedding Service abstract Ch1.8
is invoked by dependency
sends data to dynamic
- Retriever abstract Ch3.4
- Tool Selector Ch3.4
alternative to variability
Design guidance
- SHOULD use features such as query length, multiple questions, reference to prior context, and domain keywords.
- SHOULD run on a small language model for initial intent classification and complexity identification.
Quantitative guidance
As stated by the sources; verify before use.
- Costs pennies per thousand queries (Ch1.8).
- Effective complexity classifiers achieve 85-90% accuracy (Ch3.4).
- Adaptive retrieval: 1-2 documents for simple queries vs 10-15 for complex ones (Ch3.4).
- Small models used for routing cost ~90% less than frontier models while keeping accuracy on classification and intent recognition (Ch3.10).
- 100-200 labeled easy/hard examples suffice to train a lightweight difficulty classifier from problem features (Ch5.3).
Classification
- Patterns
- Embedding-based classificationFeature-based complexity classificationRule-based heuristics
- Quality attributes
- Cost efficiency
- Risks mitigated
- Misrouting complex queries to weak modelsOver-decomposition of simple queries
Sources
- Ch1.8: T. Nguyen, "Scalability and Production Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.8. ISBN: 9798244538229.
- Ch3.4: T. Nguyen, "Tuning Model Parameters for Production Performance," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.4. ISBN: 9798244538229.
- Ch3.10: T. Nguyen, "Efficiency Metrics," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.10. ISBN: 9798244538229.
- Ch5.3: T. Nguyen, "Self-Consistency Fundamentals," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 5.3. ISBN: 9798244538229.
- Ch6.6: T. Nguyen, "Query Decomposition and Adaptive Retrieval," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.6. ISBN: 9798244538229.
- Ch8.3: T. Nguyen, "Token Economics and Architecture," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 8.3. ISBN: 9798244538229.