Build Analytic Models

Add file and query sources, build models with the AI Data Builder, preview, document, publish, roll back and refresh

An analytic model is a SQL query over your sources and other models. You build it in the Data Builder: a chat with a builder agent on one side and the model’s SQL, results and documentation on the other. Publishing freezes the SQL as a version; a materialized model also gets a Parquet snapshot that agents read.

Set up the Data Builder (once per workspace)

The builder agent runs in a sandbox like any other agent, so it needs a runtime. An Admin sets this once under Data -> Settings (Builder configuration); it applies to every analytic model in the workspace from the next builder turn.

Setting	What it is for
Harnesses	The agent runtime for builder turns. See Harness.
Sandbox environment	The image the builder runs in. See Sandbox Environments.
Vault	Credentials for the builder’s own tools. Connection credentials are not taken from here. See Vault.
Tools	An optional tool bundle for the builder.
Reference contexts (published)	Knowledge bases the builder may read, for example your metric definitions.
Skills (approved)	Skills installed on the builder.
Custom prompt	Extra instructions for every builder in the workspace, up to 20,000 characters.

Harness, sandbox environment and vault are required: until they are set, the builder refuses to run.

Add more sources

Database tables become sources through Manage tables (see Connect a data source). Two more kinds of source are added by Admins.

File sources

A CSV, TSV or Parquet file from Files can be a source. No copy is made: every query reads the file as it is now, so the source is always current.

1. Start

   In Data -> Connections, click Connect source and choose Workspace files as the Source type.

2. Pick one file

   Choose a .csv, .tsv or .parquet file from the Workspace tab of Files or from an agent’s outputs. Folders and other formats cannot be sources.

3. Set parse options

   Enter a Source name and, for delimited files, the Encoding (UTF-8, UTF-16, Big5, CP950, Latin-1 or Detect automatically), Delimiter and whether the First row is a header.

4. Add

   Tick Also create an analytic model to build and publish a materialized snapshot of the file right away (agents and apps can only read models, not sources). Click Add source.

The source gets the SQL name data.files.raw.<file name> (or data.files.outputs.<file name>). Open it and use Parse options to change the options or override a column’s detected type (VARCHAR, BIGINT, DOUBLE, DECIMAL(38,9), BOOLEAN, DATE, TIMESTAMP), for example to keep IDs with leading zeros as VARCHAR. Deleting the source does not touch the file.

Query sources (Oracle)

On an Oracle connection card, Add query saves one SELECT that the database runs in its own SQL dialect. Use it to let Oracle do large joins and filters before the result reaches DataGen.

• Every table must be written as schema.table and be in the connection’s schemas.
• Only the result is read, up to 100,000 rows. The query text can be up to 20,000 characters.
• Check and save asks the database for the result’s columns, which also validates the query.

The source’s SQL name is data.<connection alias>.query.<name>.

Create a model

There are three ways to start a model. Each opens the Data Builder on a new draft.

Start from	How
One source, as is	Open the source and click Build model. The draft is SELECT * FROM <source>, materialized.
A blank model	Datasets -> Analytic models -> New analytic dataset, enter a name, choose a Mode, then Create analytic dataset.
A file	Tick Also create an analytic model when adding a file source. This one is published immediately.

Choose a mode

Mode	Runs where	Use it when
Analytic SQL (default)	DuckDB in DataGen, across any sources and models	You combine several sources, or sources from different databases.
Native SQL	Unchanged on one PostgreSQL, MySQL or Oracle Connection	You want the database to join, filter and aggregate large tables first.

A Native SQL model names tables the way the database does (no data.* names), needs no registered tables, and is always materialized. Analytic SQL models read sources and other models by their SQL names.

Build with the AI Data Builder

The builder has three areas:

• A sidebar with Chats (your builder conversations), Setup (the model’s files and settings) and Versions.
• The builder chat.
• The model viewer with the SQL, Data, Schema and Guide tabs.

Describe the model you want in the chat. The builder agent explores the catalog, writes model.sql, checks it against the publish rules, can start a preview, and writes the model’s guide. You can edit model.sql yourself at any point. Click Save draft to keep your changes; saving never publishes.

Example request

Build a model with one row per customer per month: order count, revenue,
and first order date. Use only completed orders. Document the grain and
which columns can be summed.

Under Setup -> Source assets, limit which sources and models the builder sees. The default is All available assets. A narrower selection gives the builder less to search and keeps the model from reading tables it should not.

Preview

On the SQL tab, Execute SQL runs the current draft in the background. You can keep editing or leave; the preview never changes published data. Results appear on the Data tab and stay available for 24 hours. Run details shows the executed SQL and the Next preview timeout (minutes). The builder agent can start previews too; they are labeled Started by Agent.

Execution mode

In the model’s settings, choose the Execution mode:

• Live query runs the saved SQL each time the model is read.
• Materialized stores the result as a snapshot. This is what agents and apps read, and what refresh schedules update.

Document the model

Documentation is what makes a model safe for agents to use.

• Schema tab: a description of the dataset and of each column, including units and business rules. Click Save descriptions.
• Guide tab: model.md, covering the grain and keys, measures, joins, lineage and caveats. It is saved and published with the SQL, and agents read it with datagen-data describe. Ask the builder to write it, or write it yourself.

Publish

1. Review

   Click Publish dataset. The Review publication dialog shows the name, execution mode and status. For a materialized model, set the Build timeout (1 to 60 minutes).

2. Build

   Confirm with Publish dataset. A materialized model is queued and built into a snapshot (Queued, Building snapshot, Snapshot ready). If the build fails, Retry build; the previous snapshot stays available meanwhile.

3. Live

   When the build succeeds, the new version becomes Live. Connected agents read it from their first query after the build.

Versions and rollback

Versions lists every published version, newest first, with the snapshots each one has kept and their row counts.

Action	What it does
Inspect version	Opens that version’s definition read-only.
Restore to draft	Replaces the current draft with that version’s SQL and settings. Nothing is published until you publish again.
Roll back to this version	Makes that version and one of its retained snapshots the live data immediately. Your working draft is unchanged.

Restoring a definition does not restore historical source rows. To roll back a materialized version, it must still have a retained snapshot; otherwise restore it to the draft and publish it again to rebuild from current data.

Snapshots kept per model: the active one, the last 3 refreshes, and each published version’s snapshot. Older ones are removed automatically.

Refresh

Refresh reruns the published SQL and stores a new snapshot. It never changes the SQL, never publishes, and never touches the draft. Open the model’s Automations tab.

• Refresh published data runs a refresh now.
• New automation creates a schedule:

Frequency	Notes
Every hour, Daily, Weekly	Pick the minute, time or weekday, and a Timezone.
Custom (cron)	Five fields: minute hour day month weekday. Runs at most once an hour.
When upstream data refreshes	Runs after a materialized model this one reads is refreshed, never on a clock.

A schedule runs with the access of the person who last saved or resumed it (Runs as that person). If a run fails, the model keeps serving its last successful data; after 3 failures in a row the schedule pauses itself. Fix the cause and click Resume.

Delete a model

Delete analytic model removes the model with its versions, schedules and agent connections. Other models that read it must be updated.

Next

Use data in agents and apps

Connect a published model to an agent and query it from the sandbox.

datagen-data CLI

Every command the builder and your agents use to query data.