Knowledge & Data · Data artifact
Quality Filter Rule Set
Data artifactKnowledge & DataKnowledge & Dataarc:QualityFilterRuleSet
A configuration of document quality thresholds and reference lists, such as minimum and maximum length, minimum word count and boilerplate phrases, applied by a quality filter.
Responsibility. Defines the acceptance thresholds for document quality validation.
Also known as: Quality filters config, Boilerplate dictionary, Curation filter thresholds
Relationships
configures structural
Design guidance
- MUST be tuned empirically; no universal threshold (e.g., min_words 50 vs. 100) is correct across domains.
- SHOULD tolerate higher perplexity and use higher deduplication similarity thresholds for spoken-language transcripts than for formal web text.
Sources
- Ch6.3A: T. Nguyen, "ETL Pipeline Fundamentals," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.3A. ISBN: 9798244538229.
- Ch7.5: T. Nguyen, "NeMo Curator, Riva Speech AI & Multimodal Integration," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.5. ISBN: 9798244538229.