Agentic AI Reference Architecture

Infrastructure · Software component

Dataframe Compute Engine

Software componentInfrastructureInfrastructureVariation point (abstract)arc:DataframeComputeEngine

An abstract data-processing engine that executes dataframe operations such as filtering, deduplication and text normalization for ETL transformation stages.

Responsibility. Executes bulk transformation operations over document collections.

Also known as: ETL processing engine

hostshostsis specialized byhostsis specialized byContent Deduplicator: hostsContent DeduplicatorDocument Quality Filter: hostsDocument Quality FilterGPU-Accelerated Dataframe Engine: is specialized byGPU-Accelerated Datafram…Text Normalizer: hostsText NormalizerCPU Dataframe Engine: is specialized byCPU Dataframe Engine
Direct neighbourhood (hover for relationship types)

Variants

VariantWhen to choose
CPU Dataframe EngineChoose for datasets under ~10 million documents, infrequent (monthly or rarer) processing, simple quality filters without deduplication, or limited capacity to manage GPU infrastructure.
GPU-Accelerated Dataframe EngineChoose for billions of documents where CPU pipelines take multiple days, regular full reprocessing of large corpora, fuzzy deduplication at scale, or when GPU infrastructure already exists.

Relationships

hosts structural

Sources

  1. Ch6.3B: T. Nguyen, "ETL Worked Example - Load Phase & Pipeline Integration," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 6.3B. ISBN: 9798244538229.