/databricks-genie-agents

name: databricks-genie-agents
description: "Create, manage, and query Databricks Genie Agents — curated, per-data natural-language agents (formerly Genie Spaces): build, export/import, migrate across workspaces, and ask questions of a *specific* Agent via the Conversation API. For general data questions or finding data across your workspace, use databricks-data-discovery (Genie One) instead."
compatibility: Requires databricks CLI (>= v1.0.0)
metadata:
  version: "0.1.0"

Databricks Genie Agents

Create, manage, and query Genie Agents (formerly Genie Spaces) - natural language interfaces for SQL-based data exploration.

Overview

Genie Agents allow users to ask natural language questions about structured data in Unity Catalog. The system translates questions into SQL queries, executes them on a SQL warehouse, and presents results conversationally.

A Genie Agent is a **curated agent scoped to specific data** — its tables, sample questions, and instructions are authored for a particular business area. This is distinct from **Genie One** / the general "ask Genie" data-discovery path (see the `databricks-data-discovery` skill), which answers questions across your data without a curated, per-scope agent.

Creating a Genie Agent

Step 1: Understand the Data

Before creating a Genie Agent, explore the available tables to:

• **Select relevant tables** — typically gold layer (aggregated KPIs) and sometimes silver layer (cleaned facts) or metric views
• **Understand the story** — what business questions can this data answer? What insights can users discover?
• **Design meaningful sample questions** — questions should reflect real use cases and lead to actionable insights in the data

Use `discover-schema` as the default — one call returns columns, types, sample rows, null counts, and row count. If you only know the schema, list tables first with `query "SHOW TABLES IN ..."`.

`databricks experimental aitools tools discover-schema catalog.schema.gold_sales catalog.schema.gold_customers`

For Genie, knowing column distribution shapes the sample questions and text instructions. If you don't already know the data, probe cardinality, ranges, and top categorical values with aggregate SQL through `databricks experimental aitools tools query --warehouse <WH> "..."` so your sample questions reflect what's actually in the data. Both commands auto-pick the default warehouse; set `DATABRICKS_WAREHOUSE_ID` or pass `--warehouse <ID>` to override.

Fan out independent probes with `databricks experimental aitools tools statement submit` (returns a statement_id immediately) + `... get` (blocks until terminal: `SUCCEEDED|FAILED|CANCELED|CLOSED`):

SIDS=()
for q in "$@"; do
  SIDS+=( "$(databricks experimental aitools tools statement submit --warehouse "$WH" "$q" | jq -r .statement_id)" )
done
for s in "${SIDS[@]}"; do databricks experimental aitools tools statement get "$s"; done
# Use `status` for non-blocking peek; `cancel` to terminate.

Step 2: Create the Genie Agent

Define your Genie Agent in a local JSON file (e.g., `genie_agent.json`) for version control and easy iteration. See "serialized_space Format" below for the full structure.

# List all Genie Agents
databricks genie list-spaces

# Create a Genie Agent from a local file
# IMPORTANT: sample_questions require a 32-char hex "id" and "question" must be an array
# IMPORTANT: parent_path must ALREADY EXIST — create it first, or create fails with
#   "Tree node with path ... does not exist":
databricks workspace mkdirs /Workspace/Users/you@company.com/genie_spaces
databricks genie create-space --json "{
  \"warehouse_id\": \"WAREHOUSE_ID\",
  \"title\": \"Sales Analytics\",
  \"description\": \"Explore sales data\",
  \"parent_path\": \"/Workspace/Users/you@company.com/genie_spaces\",
  \"serialized_space\": $(cat genie_agent.json | jq -c '.' | jq -Rs '.')
}"

# Get agent details (with full config)
databricks genie get-space SPACE_ID --include-serialized-space

# Tag the Genie Agent for resource tracking — use any tag the user indicated for their
# project; otherwise default to `ai_generated_source=databricks-agent-skills`.
# (Beta CLI surface — ignore if the command fails.)
databricks workspace-entity-tag-assignments create-tag-assignment \
  geniespaces SPACE_ID ai_generated_source --tag-value databricks-agent-skills || true

# Delete a Genie Agent
databricks genie trash-space SPACE_ID

Step 3: Test and Iterate

Use the Conversation API (section below) to ask questions and verify answers. If answers are inaccurate or incomplete, improve the agent — see "Improving a Genie Agent" below.

Export & Import

**Convention:** `genie_agent.json` always holds the **parsed** agent object (not a JSON-string-encoded blob), so it's readable and editable. At each use site we stringify it with `jq -c '.' | jq -Rs '.'` — same pattern as Step 2 Create and "Improving a Genie Agent" below. `jq -r '.serialized_space | fromjson'` on export strips the outer quoting so the file is already a parsed object.

# Export: extract serialized_space AND unwrap it to a parsed object on disk
databricks genie get-space SPACE_ID --include-serialized-space -o json \
  | jq '.serialized_space | fromjson' > genie_agent.json

# Import: same stringify pattern as Step 2 (Create)
databricks genie create-space --json "{
  \"warehouse_id\": \"WAREHOUSE_ID\",
  \"title\": \"Sales Analytics\",
  \"description\": \"Migrated agent\",
  \"parent_path\": \"/Workspace/Users/you@company.com/genie_spaces\",
  \"serialized_space\": $(cat genie_agent.json | jq -c '.' | jq -Rs '.')
}"

Improving a Genie Agent

**Recommendation-first:** when asked to optimize, tune, or fix an Agent (or its queries/tables), start by diagnosing and presenting a recommended change — do not run mutating actions (`update-space`, `ALTER`, `OPTIMIZE`, liquid clustering, warehouse changes) until the user approves. Diagnose with read-only queries only.

**Wrong filte