/mfs-ingest

name: mfs-ingest
version: 0.4.0
mfs_compat: ">=0.4,<0.5"
description: >-
  Register, update, or re-sync data sources for MFS so they become searchable —
  postgres / mysql / mongo / snowflake / bigquery, github / jira / linear /
  notion / hubspot / zendesk, slack / discord / gmail / feishu, s3 / gdrive /
  web / file. Use whenever the user wants to ADD a new data source to MFS,
  change an existing connector's config, re-ingest / re-index a source, list
  registered connectors, or troubleshoot a sync that's not picking up data.
  Trigger phrases include "add X to MFS", "ingest my [postgres/slack/github/etc]",
  "register this repo / database / workspace", "make X searchable", "re-sync Y",
  "update the slack token", "what connectors do I have". Do NOT use for:
  searching / finding / reading content (use `mfs-find`); raw mutation of the
  source itself (MFS only reads).

MFS — register / update / re-sync data sources

1. What this skill does

Walks the user through getting a data source into MFS so it's searchable. The work splits into:

1. Picking the right connector scheme. 2. Collecting credentials (preferring `env:VAR` / `file:/path` indirection over plaintext). 3. Writing a connector TOML. 4. Calling `mfs add <uri> --config <toml>` and monitoring the returned job.

Each connector has its own field set, credential acquisition story, and gotchas. Per-connector details live in `reference/connectors/<scheme>.md` — **read the matching one before collecting fields** for any scheme.

Step 0: Pre-flight (always run first)

mfs --version            # missing? `cargo install mfs-cli` (see install row below)
mfs status               # server reachable? connectors/jobs visible?
mfs config show          # endpoint/profile/client id/server-info debugging
mfs connector list       # what's already configured?

Branch on the result:

| Signal | Action | |---|---| | `mfs` not found | install the CLI (Rust): `cargo install mfs-cli`, or the shell installer from the project's GitHub releases page. | | `mfs status` connection refused | the configured server is down. Tell the user how to bring it up — pre-release, the server runs from source: `git clone https://github.com/zilliztech/mfs.git && cd mfs/server/python && uv sync && uv run mfs-server setup && uv run mfs-server run` — and wait. Work only through the configured endpoint rather than pointing the CLI at a different server. | | `mfs status` returns 401 unauthorized | the user's `MFS_API_TOKEN` is missing/wrong. Use `mfs config show` to confirm the endpoint/profile, then set the intended token source and retry. | | server up + `connector list` empty | first-ever connector; jump to **§B (greenfield walk-through)** when intent matches | | server up + N connectors registered | proceed to Step 1 intent classification |

Step 1: Classify intent (the central decision)

Read the user's most recent message. Pick exactly one row:

| User said... | Intent | Jump to | |---|---|---| | "add postgres prod-db to MFS" + credentials available (env / file / about to paste) | **A. Zero-friction add** | §A | | "I want to add postgres / slack / X" (no specifics, vague) | **B. Greenfield walk-through** | §B | | "re-sync github", "re-index slack", "pull latest from jira" | **C. Force re-ingest** | §C | | "update my slack token", "change postgres host", "switch to new DSN" | **D. Edit existing config** | §D | | "what connectors do I have", "list registered sources" | **E. List** | §E | | "find X" / "search Y" / "grep Z" / "cat W" | **wrong skill** | redirect to `mfs-find`, stop | | "is X indexed yet" / "did the sync finish" / "search returns nothing" | **wrong skill or boundary** | suggest `mfs-find` for query-side diagnosis; if user says it's an ingest issue, jump to §C or §F | | Truly unclear after a re-read | **F. Clarify** | §F |

Mid-flow redirect

If at any point the user changes intent ("wait, just list what I have" / "actually let me just re-sync the existing one"), abandon the current § and jump to the new one. Don't insist on finishing the original branch.

---

§A. Zero-friction add

User knows what to add and has credentials handy. Aim for: ≤3 questions to the user, then write toml + run `mfs add`.

1. **Parse the URI** from the user's message. Shape: `<scheme>://<alias>`. Scheme is required and is the connector type (`postgres`, `slack`, …). Alias is the human-readable instance ID — gets used as the toml filename and the connector's row in metadata.

• If only `<scheme>` was given (no alias), ASK: "What should I call

this instance? (free-form; appears as the URI host part, e.g. `postgres://**prod-db**`)"

• **`file` takes a bare path, not an alias.** The target is a local

path: `mfs add /abs/path` (the URI is derived as `file://local/abs/path`). The path is client-side: on the same host the server reads it directly; on a different host the CLI bundles and uploads the tree (`--upload` / `--no-upload` to force).

2. **Read the matching `reference/connectors/<scheme>.md`** for the required field set, and `reference/credentials.md` for how credentials work. For each credential field, put a **reference** in the toml — `env:VAR_NAME` or `file:/abs/path` — which the server resolves against its own environment / filesystem at ingest time. Make sure that value is present where the server runs (client and server share a machine on a loopback endpoint; otherwise it lives on the server — ask the user if unsure).

3. **Write the toml** to a temp path:

   # mfs-server connector config — <scheme>
   # URI: <uri>
   <field1> = "<value or env:VAR>"
   <field2> = "<value>"
   ...

Use a path like `/tmp/mfs-<alias>.toml` so it doesn't pollute the user's cwd.

4. **Run `mfs add` in estimate-confirm mode** for external sources where cost matters (databases >100k rows, GitHub repos with many issues, large Slack workspaces, full website crawls):