> ## Documentation Index > Fetch the complete documentation index at: https://datagen.dev/llms.txt > Use this file to discover all available pages before exploring further. # Build Analytic Models > Add file and query sources, build models with the AI Data Builder, preview, document, publish, roll back and refresh An analytic model is a SQL query over your sources and other models. You build it in the **Data Builder**: a chat with a builder agent on one side and the model's SQL, results and documentation on the other. Publishing freezes the SQL as a version; a materialized model also gets a Parquet snapshot that agents read. ## Set up the Data Builder (once per workspace) The builder agent runs in a sandbox like any other agent, so it needs a runtime. An **Admin** sets this once under **Data -> Settings** (**Builder configuration**); it applies to every analytic model in the workspace from the next builder turn. | Setting | What it is for | | - | - | | **Harnesses** | The agent runtime for builder turns. See [Harness](/foundation/harness). | | **Sandbox environment** | The image the builder runs in. See [Sandbox Environments](/foundation/sandbox-environments). | | **Vault** | Credentials for the builder's own tools. Connection credentials are not taken from here. See [Vault](/foundation/vault). | | **Tools** | An optional tool bundle for the builder. | | **Reference contexts (published)** | Knowledge bases the builder may read, for example your metric definitions. | | **Skills (approved)** | Skills installed on the builder. | | **Custom prompt** | Extra instructions for every builder in the workspace, up to 20,000 characters. | Harness, sandbox environment and vault are required: until they are set, the builder refuses to run. ## Add more sources Database tables become sources through **Manage tables** (see [Connect a data source](/data/connect#choose-tables)). Two more kinds of source are added by Admins. ### File sources A CSV, TSV or Parquet file from [Files](/files/overview) can be a source. No copy is made: every query reads the file as it is now, so the source is always current. In **Data -> Connections**, click **Connect source** and choose **Workspace files** as the **Source type**. Choose a `.csv`, `.tsv` or `.parquet` file from the **Workspace** tab of Files or from an agent's outputs. Folders and other formats cannot be sources. Enter a **Source name** and, for delimited files, the **Encoding** (UTF-8, UTF-16, Big5, CP950, Latin-1 or **Detect automatically**), **Delimiter** and whether the **First row is a header**. Tick **Also create an analytic model** to build and publish a materialized snapshot of the file right away (agents and apps can only read models, not sources). Click **Add source**. The source gets the SQL name `data.files.raw.` (or `data.files.outputs.`). Open it and use **Parse options** to change the options or override a column's detected type (`VARCHAR`, `BIGINT`, `DOUBLE`, `DECIMAL(38,9)`, `BOOLEAN`, `DATE`, `TIMESTAMP`), for example to keep IDs with leading zeros as `VARCHAR`. Deleting the source does not touch the file. ### Query sources (Oracle) On an Oracle connection card, **Add query** saves one `SELECT` that the database runs in its own SQL dialect. Use it to let Oracle do large joins and filters before the result reaches DataGen. * Every table must be written as `schema.table` and be in the connection's schemas. * Only the result is read, up to 100,000 rows. The query text can be up to 20,000 characters. * **Check and save** asks the database for the result's columns, which also validates the query. The source's SQL name is `data..query.`. ## Create a model There are three ways to start a model. Each opens the Data Builder on a new draft. | Start from | How | | - | - | | One source, as is | Open the source and click **Build model**. The draft is `SELECT * FROM `, materialized. | | A blank model | **Datasets -> Analytic models -> New analytic dataset**, enter a name, choose a **Mode**, then **Create analytic dataset**. | | A file | Tick **Also create an analytic model** when adding a file source. This one is published immediately. | ### Choose a mode | Mode | Runs where | Use it when | | - | - | - | | **Analytic SQL** (default) | DuckDB in DataGen, across any sources and models | You combine several sources, or sources from different databases. | | **Native SQL** | Unchanged on one PostgreSQL, MySQL or Oracle **Connection** | You want the database to join, filter and aggregate large tables first. | A Native SQL model names tables the way the database does (no `data.*` names), needs no registered tables, and is always materialized. Analytic SQL models read sources and other models by their SQL names. ## Build with the AI Data Builder The builder has three areas: * A sidebar with **Chats** (your builder conversations), **Setup** (the model's files and settings) and **Versions**. * The builder chat. * The model viewer with the **SQL**, **Data**, **Schema** and **Guide** tabs. Describe the model you want in the chat. The builder agent explores the catalog, writes `model.sql`, checks it against the publish rules, can start a preview, and writes the model's guide. You can edit `model.sql` yourself at any point. Click **Save draft** to keep your changes; saving never publishes. ```text Example request theme={null} Build a model with one row per customer per month: order count, revenue, and first order date. Use only completed orders. Document the grain and which columns can be summed. ``` Under **Setup -> Source assets**, limit which sources and models the builder sees. The default is **All available assets**. A narrower selection gives the builder less to search and keeps the model from reading tables it should not. ### Preview On the **SQL** tab, **Execute SQL** runs the current draft in the background. You can keep editing or leave; the preview never changes published data. Results appear on the **Data** tab and stay available for 24 hours. **Run details** shows the executed SQL and the **Next preview timeout (minutes)**. The builder agent can start previews too; they are labeled **Started by Agent**. ### Execution mode In the model's settings, choose the **Execution mode**: * **Live query** runs the saved SQL each time the model is read. * **Materialized** stores the result as a snapshot. This is what agents and apps read, and what refresh schedules update. ### Document the model Documentation is what makes a model safe for agents to use. * **Schema** tab: a description of the dataset and of each column, including units and business rules. Click **Save descriptions**. * **Guide** tab: `model.md`, covering the grain and keys, measures, joins, lineage and caveats. It is saved and published with the SQL, and agents read it with `datagen-data describe`. Ask the builder to write it, or write it yourself. ## Publish Click **Publish dataset**. The **Review publication** dialog shows the name, execution mode and status. For a materialized model, set the **Build timeout** (1 to 60 minutes). Confirm with **Publish dataset**. A materialized model is queued and built into a snapshot (**Queued**, **Building snapshot**, **Snapshot ready**). If the build fails, **Retry build**; the previous snapshot stays available meanwhile. When the build succeeds, the new version becomes **Live**. Connected agents read it from their first query after the build. ## Versions and rollback **Versions** lists every published version, newest first, with the snapshots each one has kept and their row counts. | Action | What it does | | - | - | | **Inspect version** | Opens that version's definition read-only. | | **Restore to draft** | Replaces the current draft with that version's SQL and settings. Nothing is published until you publish again. | | **Roll back to this version** | Makes that version and one of its retained snapshots the live data immediately. Your working draft is unchanged. | Restoring a definition does not restore historical source rows. To roll back a materialized version, it must still have a retained snapshot; otherwise restore it to the draft and publish it again to rebuild from current data. Snapshots kept per model: the active one, the last 3 refreshes, and each published version's snapshot. Older ones are removed automatically. ## Refresh Refresh reruns the **published** SQL and stores a new snapshot. It never changes the SQL, never publishes, and never touches the draft. Open the model's **Automations** tab. * **Refresh published data** runs a refresh now. * **New automation** creates a schedule: | Frequency | Notes | | - | - | | **Every hour**, **Daily**, **Weekly** | Pick the minute, time or weekday, and a **Timezone**. | | **Custom (cron)** | Five fields: minute hour day month weekday. Runs at most once an hour. | | **When upstream data refreshes** | Runs after a materialized model this one reads is refreshed, never on a clock. | A schedule runs with the access of the person who last saved or resumed it (**Runs as** that person). If a run fails, the model keeps serving its last successful data; after 3 failures in a row the schedule pauses itself. Fix the cause and click **Resume**. ## Delete a model **Delete analytic model** removes the model with its versions, schedules and agent connections. Other models that read it must be updated. ## Next Connect a published model to an agent and query it from the sandbox. Every command the builder and your agents use to query data. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.