/rudder-data-graphs

name: rudder-data-graphs
description: Produces Data Graph YAML from RETL sources for Audiences. Use when designing Data Graphs, mapping RETL to entities/events, or assessing customer fit for Audiences.
allowed-tools: "Bash(rudder-cli *), Read, Write, Edit"

RETL Connection Analysis & Data Graph Design

What this skill produces

Given a customer name (or workspace / org identifier), this skill produces five deliverables, in order:

1. **Source Inventory** — every relevant RETL source categorized as entity / event / audience / supporting-only. 2. **Untapped Segment List** — realistic, filterable business dimensions the customer has not yet expressed as audiences. 3. **Data Graph YAML** — one per workspace / domain, ready for validation in `rudder-cli` or the visual builder. 4. **Demo Warehouse Spec** — tables, key columns, joins, and known / inferred schema details needed to mock the graph. 5. **Action Items** — priority-ordered next steps, including the upgrade narrative and any assumptions to confirm.

The end-to-end flow is: find workspaces → list RETL sources → shortlist relevant sources → fetch full configs for the shortlist → categorize → design graph → validate → hand off. Each step below says what to do and *why*, so you can adapt when the customer doesn't match the common shapes.

---

Step 1 — Find the customer's workspaces

1. `admin_search_organizations(company_name=...)` to get the `org_id`. 2. `admin_search_workspaces(search_by=organizationId)` using that id. 3. Keep only `status=ACTIVE` workspaces. *Why:* inactive workspaces often have stale or disconnected configs and will produce misleading source lists. 4. Note the likely warehouse-backed workspaces, but **do not assume you already have the correct `accountId` at this stage**. *Why:* the Data Graph YAML embeds `account_id`, and that value is the warehouse account the graph runs against. The most authoritative source for it is `rudder-cli workspace accounts list --json` (see Step 4a); a RETL source config is a convenient secondary hint when one already exists, but it is not a prerequisite.

---

Step 2 — Discover sources, anchored on audiences

Run `list_sources(workspace_id=..., retl_type=all, includeConfig=false)` per workspace. *Why `includeConfig=false`:* full configs blow the context budget on large workspaces; start broad, then fetch detail only for sources you actually need.

**Important:** `includeConfig=false` is a discovery pass only. You cannot extract `filterSpec`, SQL, join hints, or reliable `accountId` values from that response alone. After shortlisting sources, fetch full config for the shortlisted source ids before making graph decisions.

**Audience sources are the anchor signal.** *Why:* this skill exists to build Audiences. Audience RETL sources are the customer's own pre-validated statements about what they segment on. A workspace can have hundreds of table sources but only a handful of audiences, and the audiences point directly at the entities that matter for activation.

**Discovery order:**

1. **List every audience source first** from the discovery pass. 2. **Fetch full config for those audience sources** and extract:

• `config.filterSpec.filterGroups[].filters[].fieldName` — the fields the customer actually filters on.
• The underlying table/model reference — this is the anchor entity for that audience.
• `primaryKey` and any source-level `accountId` if present.

3. **Pull in the tables/models that back those audiences.** Fetch full config for those backing sources too. These are non-negotiable: you need their schema shape to size the entity properly and to compute untapped segments in Step 6. 4. **Scan remaining tables/models for supporting roles:**

• **Event candidates** — tables/models with a real business-time timestamp (`ordered_at`, `sent_at`, `opened_at`, etc.) and a clear relationship back to an entity. Pull these in; events are a major Data Graph unlock that audiences alone can't surface.
• **FK targets** — tables/models referenced by foreign keys from anchor entities (e.g., if the anchor `contact` has a `branch_id`, pull in `branch`). Pull these in.
• **Everything else** — keep in the Source Inventory for completeness, but do not force into the Data Graph unless it materially improves segmentation or the customer asks. Noise suppression matters on large workspaces.

5. **Resolve `accountId` only after the shortlist exists.** Cross-check the warehouse account id from the specific RETL source configs you are actually using in the graph against `rudder-cli workspace accounts list` (see Step 4a). If shortlisted sources disagree on `accountId`, stop and flag it; one graph should not span multiple warehouse accounts.

**What to extract, by sourceType:**

| `sourceType` | Fields to read | What you can trust from it | |---|---|---| | `audience` | `config.filterSpec.filterGroups[].filters[].fieldName`, underlying table/model ref, `primaryKey`, `accountId` if present | fields already used for segmentation, anchor entity, graph account hint | | `table` | `config.table`, `config.schema`, `primaryKey`, `accountId` if present | physical table identity and PK | | `model` | `config.sql`, `primaryKey`, `accountId` if present | model intent and projected columns/aliases; not guaranteed types |

Fallback — customer has no audience sources yet

This is common. RETL Audiences is recent; many customers have extensive table/model sources but zero audiences. *Don't stop* — this is the exact customer who most needs the Data Graph pitch.

When audiences are absent, switch the anchor:

1. **Ask the customer (or infer from destination mappings) which entities they currently activate on.** If `dim_contact` is synced to HubSpot, contacts are an activation entity. Treat those tables/models as anchor entities. 2. **Apply the same "pull in FK targets and event candidates" rules** around those anchors. 3. **Frame Step 6 differently:** this is not "unused audience fields" anymore. I