/sync-finance-data

name: sync-finance-data
description: Use when the user says "sync finance", "daily sync", "run the morning sync", "fetch my finances", "ingest new docs", "run everything", or any full financial-data update. Runs all configured sources — bank/card fetch, IBKR, Drive/manual staging — then ingests and reconciles SQLite, refreshes deterministic market caches, sorts Drive drops, and performs the gated backup. It writes SQLite only; the live dashboard is run separately.

sync-finance-data

Run the complete data update into local SQLite: collect every configured source, ingest and reconcile it, refresh cached reference data, and back it up when due. It writes the database only; `./run_dashboard.sh` is the separate, user-run way to look at the result.

Use judgment, not pattern-matching. Categorization, deduplication, and reconciliation across sources are your job. Run every foreground command to completion before moving on; a scheduled task cannot receive deferred-task notifications.

**External collection is best-effort; local ingest is the reliable core.** A bank, IBKR, market, or Drive failure becomes a count-only `⚠️` summary bullet and the run continues. Fail only when migration or local ingestion cannot complete safely.

Where things live

• Drive folder ID: the `[drive]` section of `.secrets/findash` (key `root_folder_id=…`, chmod 600). Folder structure: [`docs/drive-layout.md`](../../docs/drive-layout.md).
• SQLite schema + conventions: [`docs/sqlite-schema.md`](../../docs/sqlite-schema.md)
• Safe query/write gateway: [`docs/database-operations.md`](../../docs/database-operations.md)
• How each archetype maps to tables: [`docs/doc-types/`](../../docs/doc-types/README.md)
• Password for protected payslip PDFs: the `[pdf-passwords]` section of `.secrets/findash` (one `pattern=password` line per file pattern)
• Drive access: the official **Google Drive connector** — a claude.ai connector the user adds in Claude, not a findash MCP server. Its tool names are connector-specific, so discover them by function with ToolSearch (search files · get file metadata · download file content · create file · update file). Every call is scoped to the vault root id, which `python3 scripts/drive_root.py` prints (and nothing else); connector downloads land in staging through `python3 scripts/staging_files.py save-download`. Any failure means "Drive degraded" (warn + continue). The weekly database backup is local (`scripts/backup_database.py`) and never leaves the machine.
• Local DB: `data/finance.db`
• Staging — the ingest queue: `inbox/staging/fetched/` (pairs staged by `fetch-bank-data`) and `inbox/staging/drive/` (Drive pulls from step 3, filenames prefixed `<driveId>__` so identity survives a crashed session)

Flow

1. Fetch configured bank/card sources (best-effort)

Run the [`fetch-bank-data`](../fetch-bank-data/SKILL.md) skill in the foreground. Each configured source stages one private JSON + notes pair under `inbox/staging/fetched/`; missing credentials skip that source. OTP, CAPTCHA, browser, login, and provider failures add one count-only warning and do not stop the sync. Never copy raw provider errors or private financial fields into stdout.

2. Fetch IBKR investments (best-effort, gated)

Run `python3 scripts/migrate_db.py`, then run the [`fetch-investments`](../fetch-investments/SKILL.md) skill when an active `account_feeds.provider='ibkr'` row exists or the connector is visibly connected. Never onboarded means skip silently. Connected interactive sessions ingest trades and reconciliation snapshots directly into SQLite. Unattended or unavailable connector sessions add one warning and continue; never request trading or money-movement tools.

3. Gather new Drive drops into staging (best-effort)

Pull anything new from the vault into local staging through the Google Drive connector. Every call here is best-effort: on failure, add a `⚠️` bullet (step 10) and continue — never abort.

• **Find the connector tools** with ToolSearch, matching by function. If they are absent — the connector was never added, or this session cannot reach claude.ai connectors — add `⚠️ Drive connector unavailable — manual drops skipped` and go straight to step 4.
• **Resolve the vault root:** `python3 scripts/drive_root.py` prints the configured `root_folder_id`. Probe it with the metadata tool; a failure here is the same warning as above, plus the fix if the cause is visible (connector not connected → add it via Claude → Connectors; root folder not found → fix `[drive] root_folder_id`).
• **Stay inside the vault.** Only ever search with `parentId = '<folder id>'` and `excludeContentSnippets: true`. Never call recent-files, whole-Drive `title`/`fullText` searches, sharing, or trash tools: the connector sees the user's entire Drive, and the vault is the only part that is findash's business.
• **Prepare the fixed layout:** list the root once and create any category folder from [`docs/drive-layout.md`](../../docs/drive-layout.md) that is missing (create-file with the folder MIME type, `title` = folder name, `parentId` = root). Idempotent; never rename or move an existing folder.
• **List the inbox and the vault:** search `dump/` by its folder id, then each category folder (`payslips`, `investments`, `long-term-savings`, `full-statements`, `fx-conversions`, `other`) recursively — follow folder entries by id and pass `pageToken` until a response is empty. Collect `(id, path, modifiedTime, fileSize)`.
• **If layout preparation or listing fails** (permissions, a signed-out connector, or network trouble): one warning bullet — `⚠️ Drive unavailable or vault layout incomplete — manual drops skipped` — then go straight to step 4 and ingest whatever is already staged. A missing fixed folder by itself is not an outage: layout preparation creates it before listing.
• **Dedup before download:** query `documents.drive_id`; skip anything already present. Exception: a `dump/`-listed file whose `drive_id` is alrea