/rudder-profiles-update

name: rudder-profiles-update
description: "Modifies an existing RudderStack Profiles project: features, inputs, id_types, cohorts, feature views, SQL models, optimizations, macros, propensity/attribution, or incremental migration. Use when updating Profiles YAML, adding features/audiences/models or changing incremental behavior."
allowed-tools: "Bash(pb *), Read, Write, Edit"

RudderStack Profiles Project Update

Modify an existing Profiles project carefully. Always understand the current state first, classify risk, and validate after each change.

Workflow

1. **Read current state** — Read all YAML files, run `pb show models`, run `pb compile`. 2. **Fix first** — If compile already fails, fix the existing project before introducing new changes. 3. **Classify the change** — Determine risk level before making edits (see table below). 4. **Make one change at a time** — Edit, then `pb compile`. Do not batch multiple changes before validating. 5. **Offer a run** — Only after compile is green and user confirms.

Change Risk Classification

| Change Type | Risk | Key Concern | |-------------|------|-------------| | Add new entity_var | Safe | Must aggregate if has `from` | | Add new input source | Safe | Verify table/columns exist via `describe_table()` | | Add new id_type | Moderate | Affects identity resolution graph | | Add feature view (`using_ids`) | Safe | Re-keys existing output; needs an id_type already on the entity | | Add entity cohort | Moderate | Filter features must exist in the parent var_group; re-run to materialize | | Add/modify sql_template model | Moderate | Pongo2 + `DeRef` dependencies; `single_sql` can't take a top-level `WITH`; materialization choice affects cost | | Add optimizations.yaml flag | Safe | Enable one at a time and confirm output is unchanged | | Modify existing entity_var | Moderate | Downstream refs may break; invalidates incremental checkpoints | | Add propensity model | Moderate | Date handling rules change completely (see below) | | Add attribution model | Moderate | Needs a separate campaign entity + campaign id_stitcher first (see below) | | Remove entity_var or input | Breaking | Must scan all refs first; warn about downstream consumers | | Change entity or id_stitcher | Breaking | Full re-run required; all checkpoints invalidated |

Before writing any change: 1. Name the risk class. 2. Name the expected blast radius. 3. Tell the user what validation will prove the change is safe.

Standard Update Rules

• Verify every new table or column with `describe_table()` before using it.
• If an entity var has a `from` key, its `select` MUST use an aggregation: `count`, `sum`, `max`, `min`, `avg`, `first_value`, or `last_value` (order-dependent ones need a `window:` with `order_by`).
• Entity var reference syntax: dot form `'{{<entity_name>.var_name}}'` — e.g. `'{{user.order_count}}'` (the first segment is the entity's name, not the literal word `entity`; the `.Var("...")` function form also works but dot is preferred).
• Model paths in `from:` are a path to a model — `inputs/<name>`, `models/<name>`, or `packages/<pkg>/...` — pb has no dbt-style `ref('...')`.
• For removals or renames, scan all files for downstream references first and warn the user explicitly before proceeding.

Propensity Models

Before writing `models/profiles-ml.yaml`, confirm all four items with the user:

1. Label definition (what binary outcome to predict). 2. Prediction window (how far ahead to predict). 3. Eligible-user filter (which users qualify for scoring). 4. Output model names.

Then:

• Use `model_type: propensity` in the model definition.
• Convert ALL date-based entity vars to macros.
• Add `python_requirements: - profiles_mlcorelib>=0.8.1` to `pb_project.yaml` if not already present.
• Validate with `validate_propensity_model_config()` and `evaluate_eligible_user_filters()` before compile.

Banned date functions for propensity features

Never use these in entity vars that feed propensity models:

• `current_date()`, `current_timestamp()`, `datediff()`, `sysdate`, `getdate()`, `now()`

Always use the conventional project-defined macros instead (confirm they exist in `models/macros.yaml` — they are NOT pb built-ins):

• `{{macro_datediff('column')}}` — days between `column` and the project's `end_time`
• `{{macro_datediff_n('column', N)}}` — boolean predicate "within N days"; the second arg is an **integer day count**, not a unit string like `'months'`

Why: macros expand to use the project's `end_time` so they stay anchored to each training snapshot's reference date. Direct date functions evaluate at query time and corrupt historical training data (label leak).

See `references/propensity-yaml-template.md`.

Attribution Models

Attribution computes first-touch and last-touch credit for conversions across user→campaign journeys (`model_type: attribution`, a PyNative model in `profiles_mlcorelib`). It has real prerequisites — do not start the model until they exist:

• a separate **campaign** entity in `pb_project.yaml` with its own id_types,
• a **`campaign_id_graph`** id_stitcher for that entity,
• campaign entity_vars for start/end dates,
• `python_requirements: - profiles_mlcorelib>=0.8.1`.

Confirm conversions, conversion timing/value, attribution window, touchpoints, and campaign data with the user first. Note the schema is `additionalProperties: False`; `conversion_vars[*].timestamp` is a `user.Var('...')` reference (not a column); `campaign_start_date`/`campaign_end_date` are campaign entity_var names (not literals); `conversion_window` is a string like `"30d"`. See `references/attribution-yaml-template.md`.

SQL Template Models & Performance

When a transform can't be an entity_var (multi-step SQL, joins, a filtered/reshaped source), add a `sql_template` model (`single_sql`/`multi_sql`, `this.DeRef(...)`, materialization, optional `ids`/`contract`/`features`). Reference it from entity_vars with `from: models/<name>`. Keep it `run_type