/portaljs-add-dataset

description: Add a dataset (CSV, TSV, JSON, or GeoJSON) to an existing PortalJS portal. Appends an entry to datasets.json so the catalog and showcase render it automatically; routes the data by source (local file vs remote URL) — R2 via Git LFS by default, remote URLs by passthrough.
allowed-tools: Read, Write, Edit, Bash, WebFetch

/portaljs-add-dataset

Add a dataset to an existing PortalJS (`portaljs-catalog`) portal. Appends one entry to `datasets.json` — the **single source of truth** for the catalog — and routes the data to the right place by **source first, then size**. No per-dataset page is created: the catalog at `/search` lists it and the dynamic showcase route `pages/[owner]/[slug].tsx` renders it automatically at `/@<namespace>/<slug>`.

The routing matrix — branch on SOURCE, then size

Where the bytes end up depends first on **where they come from**, then on size/intent. The manifest's `resource.path` (or single-file `file`) is the unifying abstraction: `resourceUrl()` in `lib/datasets.ts` serves a **repo-relative** path from `/public/data` and passes an **absolute URL** straight through. So every route below just decides what string to write into `path`.

| Source | Default | What happens | Manifest `path` | |--------|---------|--------------|-----------------| | **Local file** | **R2 via Git LFS** | `mv` into the repo, `git lfs track <path>`, commit a ~134 B pointer, `git lfs push` → Giftless → R2 (no GitHub remote needed) | **absolute R2 URL** (browser fetches R2 directly) | | **Local file** (fenced) | inline | *Only* bundled SAMPLE data or an OSS self-host with no R2: `cp` into `public/data/` (stays inline per `.gitattributes` fence) | bare filename → `/data/<file>` | | **Remote URL** | **passthrough** | record the URL as-is — **no download, no upload, zero duplication** | the **absolute URL**, unchanged | | **Remote URL** | adopt (opt-in) | user wants it hosted/versioned under the portal: fetch → route as a local file (R2/LFS) | **absolute R2 URL** |

**Storage decision (epic po-g9y):** the default for added data is **R2**, regardless of size or format. Inline is *not* a size threshold — it is a fenced exception for bundled sample data and the OSS-no-R2 fallback. Remote URLs default to **passthrough** (copy nothing); adopting one into R2 is an explicit opt-in.

> **Remote-URL passthrough caveat:** serving and linking always work. But **in-browser > range / DuckDB queries** against a 3rd-party URL need CORS + range support on *that* host, > which we don't control. If the user needs querying (not just preview/download) and the > remote lacks CORS/range, recommend the **adopt-into-R2** option.

Supported formats for the **showcase preview/registration**: **CSV, TSV, JSON (array), GeoJSON**. (LFS itself is format-agnostic — any binary can be tracked — but the showcase `Table`/map only previews these.)

Required input — ask, don't error

• **Source** — a local file path (`./data/file.csv`) or a public URL (`https://…/data.csv`)
• **Portal directory** — path to the portal project (defaults to current directory)
• **Namespace** — the dataset's namespace value (the portal's `NAMESPACE_TYPE` group:

a subject for `'theme'` portals, a publisher for `'owner'` portals)

**If the source is missing, ask for it — never dead-end.** The user can say "use defaults" to accept the defaults below.

Steps

1. Gather input from `$ARGUMENTS` (interview if thin)

Extract what's present:

• `SOURCE` — file path or URL
• `PORTAL_DIR` — portal directory (default: `.`)
• `DATASET_NAME` — human-readable name (default: derived from filename)
• `DATASET_SLUG` — URL slug (default: lowercase hyphenated filename without extension)
• `DESCRIPTION` — optional one-line description
• `NAMESPACE` — namespace value (default: read the existing first entry's `namespace`

from `datasets.json`, else `reference`)

• `ADOPT` — for a remote URL only: whether to adopt the file into R2 (default: **no** —

passthrough). Only ask if the URL route is taken and querying is plausibly needed.

If `SOURCE` is missing, ask (one focused prompt) and wait:

To add a dataset I need:
1. Source: local file path or public URL (required)
2. Portal directory (Enter for current directory)
3. Dataset name (Enter to use the filename)
4. Namespace value — the group this dataset belongs to
   (subject if the portal is "theme" mode, publisher if "owner" mode; Enter to reuse the catalog's existing namespace)

Check the portal's namespace mode if helpful: read `NAMESPACE_TYPE` from `PORTAL_DIR/lib/datasets.ts` so you can phrase the namespace question correctly ("subject" vs "publisher").

2. Validate the portal directory

The target must be a `portaljs-catalog` portal. Confirm `PORTAL_DIR/datasets.json`, `PORTAL_DIR/package.json`, and `PORTAL_DIR/pages/[owner]/[slug].tsx` exist. If `datasets.json` is missing, tell the user this portal isn't the catalog template (it may be an older single-page template) and ask how to proceed rather than failing silently.

3. Detect the format

**If SOURCE is a URL:** fetch headers (or a `HEAD`) and check the status. If not 200, tell the user the fetch failed (with the HTTP status) and ask them to confirm the URL is publicly accessible, then retry. Detect format from `Content-Type` or the URL extension. *(For the default passthrough route you do not download the body — only enough to detect format.)*

**If SOURCE is a local file path:** check the file exists. If not, tell the user the path wasn't found and ask for a correct path. Detect format from the file extension.

**Format detection rules:**

• `.csv` or `text/csv` → CSV
• `.tsv` or `text/tab-separated-values` → TSV
• `.geojson` or `application/geo+json` or (JSON-parseable and `parsed.type === "FeatureCollection"`) → GeoJSON
• `.json` or `application/json` → JSON array
• **Vector geo formats** — `.zip` (zipped Shapefile), `.gpkg`, `.kml`/`.kmz`, `.fgb`, or a

`.csv` with a geometry/lat-lon column → **hand off to `/po