Core concepts

Workspaces, building blocks and foundation, how blocks attach to each other, and how versioning and publishing work

This page is the mental model behind DataGen AI Cloud. Read it once; every other page builds on these ideas.

Workspace

Everything you build lives in a workspace: connections, files, knowledge bases, agents, apps and the foundation they run on. Each user has a personal workspace, and teams create shared workspaces and invite members.

Each member has a workspace role — Admin, Builder, Operator or Viewer — and an individual resource can override a member’s role for that resource only.

Building blocks and foundation

DataGen separates what you build from what it runs on.

• Building blocks are the things your team creates and uses: Data, Files, Context, Agents and Apps.
• Foundation components are shared settings that building blocks pin: a Vault for credentials, a Sandbox Environment for the image and resources, and a Harness for the agent runtime.

Every agent, app and AI builder runs inside a sandbox. When it starts, the sandbox uses the pinned Sandbox Environment, Harness and Vault. Anything left unset uses the workspace default:

Foundation	Default when nothing is pinned
Vault	The workspace’s default vault, named Default, created on first use
Sandbox Environment	The standard image, shown as Default (standard image)
Harness	The built-in Claude Agent SDK against Anthropic

That is why you can build a first agent without configuring any foundation. See Vault, Sandbox Environments and Harness when you need more control.

How blocks attach

Building blocks feed into each other. Files flow into Context and into Data. Agents use knowledge bases and published Data models; apps use published Data models and workspace files.

Files ──┬──> Context ─────────────┬──> Agents  (READ or READ_WRITE)
        │                          └──> App builder (read-only)
        └──> Data ──> published model ─┬──> Agents
             (CSV, TSV, Parquet)        └──> Apps

Block	What you build	What it attaches to	Foundation it uses
Data	Connections, sources and analytic models you publish	Agents and apps (published models)	Sandbox Environment and Vault for the AI Data Builder
Files	Uploaded files and folders	Data (CSV, TSV and Parquet file sources); Context (as a source)	None
Context	A versioned knowledge base built from selected files	Agents (READ or READ_WRITE); the AI App Builder of custom apps (read-only, build time)	Sandbox Environment, Harness and Vault for the AI Context Builder
Agents	An agent with instructions, tools, skills, automations and channels	Consumes Context, Data, Skills and tools	Vault, Sandbox Environment and Harness
Apps	A marimo notebook or a custom web app	A running app reads the workspace files selected on its Sources tab and credentials from its vault; custom apps can also connect published models	Sandbox Environment, Harness and Vault

Context is a snapshot

A knowledge base is a named, versioned file tree built from the sources you select: workspace files, personal files, agent outputs, an agent’s own files, or an external source. Each publish stores a new snapshot, and attached agents read its files read-only at /mnt/context/contexts/<slug>/ in their sandbox. It is a snapshot, not a live link: when a source file changes, the knowledge base changes only when you publish again.

Access mode

When you attach a knowledge base to an agent, you choose what the agent may do with it. You set the access mode on the knowledge base’s Agents tab:

Access mode	What the agent can do
READ	Read the knowledge base (Read in the UI). The tree mounts read-only in the agent’s sandbox. This is the default.
READ_WRITE	Read it, and write back into it — for example to record what it learned. Choose Read & write to let the agent write on its own, or Read & write (ask first) to have it ask for approval before each write.

Context exposure

An agent’s context exposure decides which knowledge bases it sees at all:

• ATTACHED — the agent sees only the knowledge bases attached to it. Every agent created today uses this mode.
• ALL — the agent sees every knowledge base in the workspace, read-only. This mode exists only on older agents and cannot be turned on for an agent that does not already have it.

Exposure never grants write access. Only an explicit READ_WRITE attachment does.

Access modes apply to agents only. A custom app connects knowledge bases to its AI App Builder: the builder chat reads them read-only while you build, and the published app does not see them.

An agent’s memory is a knowledge base attached with Read & write: the agent records what it learns there. See Memory.

Data attaches as published models

Agents read published analytic models, not raw connections or sources. An agent connects published, materialized models and queries them with the datagen-data CLI; each query reads the live version’s current data. Published models are not a knowledge-base source — you connect them to an agent or app directly. See Use data.

Versioning and publishing

Context, Data, Agents and Apps share one lifecycle: you work on a draft, then publish a version that everything else uses.

Block	Work on	Publish	Go back
Data	A draft model in the AI Data Builder	Publish a version (live or materialized)	Roll back to an earlier version, or restore it to the draft
Context	Sources and builder output in the AI Context Builder	Publish a new snapshot	Roll back to an earlier version (published as a new version)
Agents	Configuration and files in the Agent Builder	Publish, which also deploys the agent	Roll back to an earlier version
Apps (custom)	Files and settings in the AI App Builder	Save a version, then publish it from Versions	Publish an earlier version

Publishing never changes what you are editing, and editing never changes what is published. Consumers — an agent reading a knowledge base or querying a model — always read the published version. Automations follow the same rule: a scheduled run on a knowledge base or an app writes to the draft and never publishes, and a scheduled data refresh reruns the published SQL without changing it.

Marimo notebooks work differently: they have no builder, and marimo saves the notebook file in place in the workspace’s shared notebooks folder.

Next

Quick start

Put these concepts to work in about 10 minutes.

Vault

Start with the foundation: store the credentials your blocks use.