How DPF Works

You describe your data in plain language; DPF does the engineering. This page covers what happens behind that prompt — where your data lands, how you query it, and how AI agents and applications drive the same platform through its API.

From File to Queryable Table

1

Describe Your Data

Upload a sample file or connect a source, then say what you want in plain language. No schema definition, no mapping spreadsheets.

2

AI Builds the Pipeline

The AI infers the schema and generates production-ready parsing and transformation code. You review it once and save it as a reusable spec.

3

Run It On Demand or On Schedule

Run the job manually, on a schedule, or on a trigger — a new file landing in SFTP or S3. Every run reuses the same spec.

4

Query the Results

Data lands in open Apache Iceberg tables. Query it in the portal, from your BI tool, or from your own compute engine.

Under the Hood

đŸ”ī¸ Open Storage, No Lock-In

Data lands in open Apache Iceberg tables with schema evolution, time-travel, and rollback built in. Query it instantly with our built-in engine, or bring your own compute: Spark, Trino, Redshift, Databricks, and more.

🔌 API-First, Agent-Ready

Everything is API-first: every action in the portal is also a REST call exposed through a remote MCP server. AI agents can onboard sources, run transformations, and query results with or without a human in the loop.

The Technical Foundation

đŸ”ī¸

Open Iceberg Platform, Built-In Query Engine

An open data management platform built on Apache Iceberg — open tables with schema evolution, time-travel, and rollback, with no proprietary lock-in. Load and query instantly with the built-in engine, or follow our integration guides to bring your own compute: Spark, Trino, Databricks, or Redshift.

â†Šī¸

One-Click Rollback

Every data load captures Iceberg snapshots. Made a mistake? Roll back to the exact pre-load state with a single API call. Full audit trail included.

🤖

Built for Humans & AI Agents

Every action in DPF is API-driven — the web portal itself is just one client of the same REST API. Automate your entire data integration pipeline programmatically, drive it from our MCP server for AI-powered data analysis, or use the web portal.

Platform Capabilities

đŸŽ¯ Job Design Flow

One-time setup process to define parsing rules, transformation logic, and mapping specifications that are reused for every execution.

⚡ Job Execution Flow

Automated execution pipeline that parses input files, validates data, applies transformations, and loads results to open table formats.

🔌 REST API

Comprehensive, API-driven platform — every capability (integration management, job execution, credential handling, and data consumption) is exposed as a REST API call, not just bolted on after the fact.

🤖 AI Agent Integration

DPF is built to integrate with AI from the ground up. Our Model Context Protocol (MCP) server lets AI agents autonomously set up data integrations, manage transformations, and make data available — using the same API-driven actions a human would take in the portal.

đŸ’ģ Web Portal

User-friendly interface for integration setup, file uploads, job monitoring, and results visualization with team collaboration features.

🤝 Team Collaboration

Share integrations, collaborate on transformations, manage permissions, and enable data collaboration across teams without stepping on each other.

Want the Details?

The documentation covers connecting your own query engine, end-to-end API examples, and the full REST API reference.