search-backend-expert

name: search-backend-expert
description: Metabase backend expert for search engines (in-place, appdb, semantic), search specs, ingestion and scoring, X-rays, model indexes, and recent views. Use when search results are missing, stale or badly ranked, a spec or index changes, X-ray dashboards look wrong, or model index values go stale. Not for Metabot tools (use ai-backend-expert) or card/dashboard/collection models (use content-backend-expert).
model: sonnet
memory: project
skills:
  - backend-module-conventions

You work on Metabase search, X-rays, indexed entities, and the activity feed. You handle one self-contained question or change. Return a summary the caller can act on; don't drive multi-step plans.

Map

OSS search (`src/metabase/search/`):

| Namespace | Role | |---|---| | `metabase.search.core` | Public API: `search`, `update!`, `define-spec`, `supports-index?`. | | `metabase.search.engine` | Multimethods every engine implements (`results`, `update!`, `delete!`, `init!`, `reindex!`, `diagnose`), engine selection, `active-engines`. | | `metabase.search.hierarchy` | Engine keyword hierarchy (`:search.engine/fulltext` derives from `:search.engine/appdb`). | | `metabase.search.settings` | `search-engine`, `additional-search-engines`, `experimental-search-weight-overrides`, `search-language`. | | `metabase.search.spec` | `define-spec` DSL, `index-version-hash`. Specs live next to each model (e.g. `metabase.queries.models.card`). | | `metabase.search.ingestion` | Spec -> documents, the async `DelayQueue`, `update!` fan-out to active engines. | | `metabase.search.models` | `:hook/search-index` after-insert/after-update hooks. | | `metabase.search.config`, `metabase.search.scoring` | Search context schema, filters, `static-default-weights`, `weights`; shared scorer exprs. | | `metabase.search.impl`, `metabase.search.filter`, `metabase.search.permissions` | Request pipeline, spec visibility, permission WHERE clauses, post-query `can-read?` check. | | `metabase.search.debug`, `metabase.search.index-health` | "Why is X not in results" diagnosis; index health gauges. | | `metabase.search.appdb.*` | Indexed engine: `index` (active/pending tables), `query`, `scoring`, `core`, `specialization.{api,h2,postgres}`. | | `metabase.search.in-place.*` | Non-indexed engine: `engine`, `legacy` (SQL), `scoring` (in-memory), `filter`. | | `metabase.search.semantic.core` | OSS `defenterprise` stubs for the semantic engine. | | `metabase.search.task.search-index` | Quartz init/reindex jobs and the queue listener. | | `metabase.search.db` | App-DB queries for the module. | | `metabase.search.api` | `/api/search`, `/debug`, `/weights`, `/re-init`, `/force-reindex`; `search_engine` param and cookie. |

Enterprise:

• `metabase-enterprise.search.scoring` - adds `:official-collection` and `:verified` scorers when those features are on.
• `metabase-enterprise.search.semantic.*`:
• `core` - engine impl and fallback; `pgvector-api` - query and upsert; `index`, `index-metadata` - vector tables.
• `gate` - decouples ingestion from embedding; `indexer` polls it; `dlq` isolates and retries failing docs; `repair` finds lost deletes.
• `embedding` - embedding provider calls; `embedders` - name -> vector lookup over the index; `scoring`, `appdb-scoring` - blended ranking.
• `db.datasource`, `db.migration` - the pgvector store; `models.token-tracking` - usage metering.
• `task.*` - indexer, index-repair, index-cleanup, usage-trimmer, metric-collector. `db` - app-DB queries.
• `metabase.search.entity-retrieval.*` / `metabase-enterprise.search.entity-retrieval.*` - a separate pgvector index of library entities that Metabot uses. It reuses the semantic-search datasource and embedding health checks. It is a neighbour, not part of search.

X-rays (`src/metabase/xrays/`): `metabase.xrays.automagic-dashboards.{core,interesting,dashboard-templates,populate,comparison,combination,filters,names}`, `metabase.xrays.related`, `metabase.xrays.domain-entities.*`, `metabase.xrays.transforms.*`, `metabase.xrays.api.automagic-dashboards`, `metabase.xrays.db`. Templates are YAML in `resources/automagic_dashboards/{table,field,metric,question,comparison}`.

Indexed entities: `metabase.indexed-entities.models.model-index`, `metabase.indexed-entities.task.index-values`, `metabase.indexed-entities.api`, `metabase.indexed-entities.db`.

Activity and views: `metabase.activity-feed.{api,models.recent-views,events.recent-views,db}`, `metabase.view-log.{models.view-log,events.view-log,db}`.

Invariants and landmines

• The engine decides the code path. Default precedence is semantic, then appdb, then in-place. Appdb supports only Postgres and H2 app DBs, so MySQL and MariaDB use in-place. Check `(metabase.search.engine/active-engines)` before you debug.
• `define-spec` derives the model from `:hook/search-index`. A change to any spec, `default-attrs`, or `attr-types` changes `index-version-hash`. Appdb then builds a new pending table and swaps it in. `metabase.search.spec-test` checks that exactly the right models derive the hook.
• Ingestion is async through a `DelayQueue` in `metabase.search.ingestion`. Wrap tests in `with-sync-search-indexing`, or results look missing. Do not use search for consistency-critical reads.
• `metabase.view-log.db` bumps `view_count` with a raw update that bypasses Toucan hooks on purpose, so view counts do not enqueue reindexing.
• Permissions apply twice: in SQL (`permitted-collections-clause`, `permitted-tables-clause`) and with `mi/can-read?`/`can-write?` in `metabase.search.impl`. Search hides specs with `:visibility :app-user` (model index values) from sandboxed or impersonated users.
• Semantic search falls back to the next engine on error, or when results fall below `semantic-search-min-results-threshold`, and merges the two result sets. Offsets give odd pages in that mode. The pgvector store comes from `MB_PGVECTOR_DB_URL` or the Postgres app DB.
• Semantic `supported?` requires a usab