Foundation LLM

Model assetModel ServingModelsVariation point (abstract)arc:FoundationLLM

General-purpose large language model weights, ranging from frontier models used for planning to smaller models used for execution.

Responsibility. Provides language reasoning and generation capability.

Also known as: Frontier model, Smaller execution model, LLM, Model weights, Llama 3.1 70B Instruct, Llama 3.1 8B Instruct, GPT-2 1.5B, Llama 3.3 70B, Llama 3 8B, Pre-trained helpful-but-not-harmless model, Pre-trained base model, Base model

Direct neighbourhood (hover for relationship types)

Variants

Variant	When to choose
Domain-Adapted Base Model	—
Fine-Tuned Agent Model	—
Large Language Model Tier	Choose for complex queries or premium users where the quality improvement justifies cost and sharding complexity.
Reasoning Language Model	Choose for complex queries where accuracy improvements justify the substantially higher token cost.
Reference Policy Model	—
Small Language Model Tier	Choose for simple or domain-specific queries where benchmarking shows it meets quality thresholds.
Standard Language Model Tier	Choose for queries classified as moderately complex.

Relationships

deployed on structural

is evaluated by assurance

is monitored by assurance

is optimized by lifecycle

is trained by lifecycle

Fine-Tuning Pipeline abstract Ch10.3 Ref7.14 +1
Fully Sharded Data Parallel Executor Ch7.1A
RLHF Policy Optimizer Ch7.1B Ch9.1

Design guidance

SHOULD be selected by benchmarking accuracy, relevance and user satisfaction across sizes and choosing the smallest model meeting thresholds.
SHOULD NOT expect MQA/GQA benefits unless the model was trained with them; retrofitting typically costs 1-3% accuracy.
SHOULD NOT rely systematically on length extrapolation beyond training length; it extends working memory opportunistically but unreliably.
SHOULD NOT assume a larger context window improves performance without curation of its contents.
SHOULD NOT be deployed as an assistant without alignment: a base model trained only on next-token prediction generates plausible continuations without regard to helpfulness, truthfulness or appropriateness.

Quantitative guidance

As stated by the sources; verify before use.

A 70B-parameter model needs ~140 GB of memory and does not fit an 80 GB GPU (Ch1.8).
MQA cuts KV-cache memory ~64x (64-head model), enabling batch 4 -> 32 and 30-70% higher generation throughput; GQA (8 groups) gives 8x KV reduction with minimal accuracy impact (Ch4.4).
Stated context windows (late 2025): Claude 3 200k, GPT-4 Turbo 128k, Gemini Pro 1M, Jamba 1.5 256k; research models beyond 2M tokens (Ch5.9).
Length extrapolation example: model trained on ~4,000-token sequences handling 8,000 tokens (2x) (Ch5.9).
Jamba-1.5-Large: 256K context, ~2.5x faster than comparable Transformers on long contexts; Jamba-1.5-Mini up to 140K context on a single GPU (Ref5.03).

Classification

Patterns: Right-sizingDomain fine-tuning of smaller modelsMulti-Query Attention (MQA)Grouped-Query Attention (GQA)Length extrapolation beyond training contextALiBi relative position biasesHybrid Transformer-Mamba mixture-of-experts architecture (Ref5.03)Grouped-query attention (Ref5.03)Long-sequence training via sequence and context parallelism (Ref5.02)
Technologies: Llama 3.1 70BLlama-3-8BLlama-3-70BLlama-3-405BGPT-3.5-TurboGPT-4GPT-4-TurboClaude 3.5 SonnetGeminiLlama-3MistralLlama 3 8BLlama 3 70BMixtral 8x7BMixtral 8x22BMistral 7BMeta LlamaNVIDIA AI Foundation modelsClaude 3GPT-4 TurboGemini ProJamba 1.5NVIDIA NemotronNVIDIA NeMoLlama 2NemotronLlama Nemotron
Quality attributes: Functional suitability: correctness and validity (ISO/IEC 25010 | NIST AI RMF: valid)Cost efficiency

Sources

Ch1.2: T. Nguyen, "Core Agent Patterns," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.2. ISBN: 9798244538229.
Ch1.5B: T. Nguyen, "Stateful Orchestration - Worked Examples," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.5B. ISBN: 9798244538229.
Ch1.7A: T. Nguyen, "Relational Reasoning with Knowledge Graphs - The Fundamentals, Integration, and Extraction," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.7A. ISBN: 9798244538229.
Ch1.8: T. Nguyen, "Scalability and Production Deployment," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 1.8. ISBN: 9798244538229.
Ch2.6: T. Nguyen, "Tool Integration and Function Calling," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 2.6. ISBN: 9798244538229.
Ch2.7: T. Nguyen, "Multimodal RAG Approaches," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 2.7. ISBN: 9798244538229.
Ch3.5: T. Nguyen, "Prompt Optimization, Few-Shot Learning, Fine-Tuning," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.5. ISBN: 9798244538229.
Ch3.6: T. Nguyen, "Trace Analysis and Execution Debugging," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.6. ISBN: 9798244538229.
Ch3.10: T. Nguyen, "Efficiency Metrics," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 3.10. ISBN: 9798244538229.
Ch4.4: T. Nguyen, "Performance Profiling and Optimization," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.4. ISBN: 9798244538229.
Ch4.5: T. Nguyen, "NVIDIA NIM and Triton Inference Server," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.5. ISBN: 9798244538229.
Ch4.6: T. Nguyen, "TensorRT-LLM and NVIDIA Fleet Command," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 4.6. ISBN: 9798244538229.
Ch5.2: T. Nguyen, "Tree-of-Thought (ToT) Fundamentals," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 5.2. ISBN: 9798244538229.
Ch5.9: T. Nguyen, "Working Memory," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 5.9. ISBN: 9798244538229.
Ch7.1A: T. Nguyen, "Advanced Implementation with Nvidia NEMO Framework and Nvlink," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.1A. ISBN: 9798244538229.
Ch7.1B: T. Nguyen, "Nvidia NIM and Colang," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 7.1B. ISBN: 9798244538229.
Ch9.1: T. Nguyen, "Output Filtering and Content Moderation," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 9.1. ISBN: 9798244538229.
Ch9.5: T. Nguyen, "Constitutional AI," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 9.5. ISBN: 9798244538229.
Ch10.3: T. Nguyen, "RLHF Methodology," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 10.3. ISBN: 9798244538229.
Ref2.07: NVIDIA Developer, "Building multimodal AI RAG with LlamaIndex, NVIDIA NIM, and Milvus | LLM app development," YouTube. Accessed: Sep. 26, 2026. [Online Video]. Available: https://www.youtube.com/watch?v=NaT5Eo97_I0
Ref4.01: NVIDIA, "TensorRT-LLM," GitHub repository. Accessed: Sep. 27, 2026. [Online]. Available: https://github.com/NVIDIA/TensorRT-LLM
Ref5.01: NVIDIA, "NeMo (NVIDIA-NeMo/Speech)," GitHub repository. Accessed: Sep. 27, 2026. [Online]. Available: https://github.com/NVIDIA-NeMo/Speech
Ref5.02: NVIDIA, "NeMo RL: A Scalable and Efficient Post-Training Library," GitHub repository. Accessed: Sep. 27, 2026. [Online]. Available: https://github.com/NVIDIA-NeMo/RL
Ref5.03: Jamba Team et al., "Jamba-1.5: Hybrid Transformer-Mamba Models at Scale," arXiv:2408.12570, 2024.
Ref7.12: "Advanced Nemotron Deployment Patterns," unpublished reference note (12-Nemotron-Advanced-Deployment.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note
Ref7.13: NVIDIA, "Llama Nemotron," NVIDIA NeMo Framework User Guide, v25.09. Accessed: Sep. 27, 2026. [Online]. Available: https://docs.nvidia.com/nemo-framework/user-guide/25.09/llms/llama_nemotron.html
Ref7.14: "NVIDIA Agentic AI Platform Ecosystem Integration," unpublished reference note (14-NVIDIA-Ecosystem-Integration.md), Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam supplementary materials, 2026. unpublished note