/tuning-incremental-sync-config

name: tuning-incremental-sync-config
description: >
  Change the sync configuration of an existing data warehouse schema — switch sync_type, pick a different
  incremental_field, set primary_key_columns, choose cdc_table_mode, or change sync_frequency. Use when the user
  asks "switch my orders table from full refresh to incremental", "this table is syncing too slowly / too
  frequently", "I need to pick a different incremental column", "set up CDC for this Postgres table", or when
  diagnosis of a failing sync pointed to an incremental-field or PK misconfiguration.

Tuning incremental sync config

A sync's configuration lives on the `ExternalDataSchema` and can be changed any time via `external-data-schemas-partial-update`. Most changes are non-destructive (take effect on the next sync), but a few (switching sync_type, changing primary keys) require careful handling to avoid corrupting the synced data.

When to use this skill

• The user wants to change how an already-connected table is synced
• A diagnosis flagged the incremental field or primary key as wrong
• The table is syncing too often / not often enough
• Switching an incremental table to CDC (or vice versa)
• The source table was changed on the other side (new columns, dropped columns) and the sync config needs to catch up

If the user is setting up a brand-new source, use `setting-up-a-data-warehouse-source` instead — configuration is chosen at creation time there.

Available tools

| Tool | Purpose | | ------------------------------------------------------ | ------------------------------------------------------------------------- | | `external-data-schemas-retrieve` | Current sync_type, incremental_field, PKs, sync_frequency | | `external-data-schemas-incremental-fields-create` | Refresh candidate incremental fields from the live source | | `external-data-schemas-partial-update` | Apply the config change | | `external-data-schemas-reload` | Trigger a sync with the new config | | `external-data-schemas-resync` | Wipe and re-import from scratch when the change invalidates existing data | | `external-data-schemas-delete-data` | Drop the synced table while keeping the schema entry | | `external-data-sources-check-cdc-prerequisites-create` | Pre-flight Postgres CDC (only when switching to/from CDC) | | `external-data-sources-webhook-info-retrieve` | Current webhook state (when switching to/from sync_type=webhook) | | `external-data-sources-create-webhook-create` | Register a webhook after switching a schema to sync_type=webhook | | `external-data-sources-update-webhook-inputs-create` | Rotate a webhook signing secret | | `external-data-sources-delete-webhook-create` | Unregister webhook when switching schemas off sync_type=webhook |

The fields you can tune

From the partial-update endpoint:

| Field | Values | Notes | | ------------------------ | ------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------ | | `sync_type` | `full_refresh`, `incremental`, `append`, `cdc`, `webhook` | Source must support the target type — check via incremental-fields | | `incremental_field` | Column name from the source | Must appear in `incremental_fields` list for the schema | | `incremental_field_type` | `datetime`, `date`, `timestamp`, `integer`, `numeric`, `objectid` | Must match the column's real type | | `primary_key_columns` | Array of column names | Required for CDC. Used for upsert dedup on incremental | | `cdc_table_mode` | `consolidated`, `cdc_only`, `both` | Only meaningful when sync_type=cdc | | `sync_frequency` | `1min`, `5min`, `15min`, `30min`, `1hour`, `6hour`, `12hour`, `24hour`, `7day`, `30day`, `never` | Applies to all non-CDC types | | `sync_time_of_day` | `HH:MM:SS` | When sync_frequency is daily/weekly-scale | | `should_sync` | `true` / `false` | Pause the schema without deleting it |

Workflow

Step 1 — Read the current config

Always start with `external-data-schemas-retrieve({id})`. Understanding the current state prevents mistakes like "fixing" an incremental_field that's actually correct.

Note:

• Current `sync_type`, `incremental_field`, `incremental_field_type`, `primary_key_columns`
• Current `status` (don't tune a schema that's currently `Running` — wait or cancel first)
• `last_synced_at` (so you can tell if the next sync worked)
• `latest_error` if present (the error often tells you exactly what to change)

Step 2 — If changing sync_type or incremental_field, refresh candidates

Call `external-data-schemas-incremental-fields-create({id})`. Even though the operation name says