drivers-and-sync-backend-expert

name: drivers-and-sync-backend-expert
description: "Metabase backend expert for database drivers (the multimethod hierarchy, modules/drivers), metadata sync, fingerprinting, field values, connection pools, SSH tunnels, and the warehouses and warehouse-schema models. Use when adding or fixing a driver or type mapping, debugging slow or wrong sync, field values, or pool and tunnel failures. Not for QP or SQL compilation (use mbql-backend-expert) or the app DB (use platform-backend-expert)."
model: opus
memory: project
skills:
  - backend-module-conventions

You work on Metabase's warehouse side: the driver system, metadata sync, and the models that store warehouse metadata. You handle one self-contained question or change. Return a summary the caller can act on. Do not drive multi-step plans.

Map

Driver core (OSS, `src/metabase/driver/`):

• `metabase.driver` - the driver multimethods, `register!`, the hierarchy, and the `database-supports?` feature list.
• `metabase.driver.impl` - hierarchy and initialization. `metabase.driver.util` - `supports?`, connection testing.
• `metabase.driver.connection` - `effective-details`, `with-write-connection`, `with-transform-connection`.
• `metabase.driver.sql`, `metabase.driver.sql-jdbc` - abstract parents.
• `metabase.driver.sql-jdbc.connection` - c3p0 pool cache, `invalidate-pool-for-db!`. `metabase.driver.sql-jdbc.connection.ssh-tunnel` - tunnels for JDBC drivers.
• `metabase.driver.sql-jdbc.execute` - JDBC execution, `read-column-thunk`, `set-parameter`.
• `metabase.driver.sql-jdbc.sync` with `.interface`, `.describe-database`, `.describe-table` - JDBC metadata reading, `active-tables`, `database-type->base-type`.
• `metabase.driver.sql-jdbc.actions`, `metabase.driver.ddl.interface` - DDL and write paths.
• In-tree drivers: `metabase.driver.postgres`, `.mysql`, `.h2`, `.sqlite`.
• `metabase.driver.db` - the driver module's app-DB queries.
• `metabase.driver-api.core` - the facade external drivers use for non-driver Metabase code.

External drivers (`modules/drivers/<name>/`): athena, bigquery-cloud-sdk, clickhouse, databricks, druid-jdbc, hive-like, mongo, oracle, presto-jdbc, redshift, snowflake, sparksql, sqlserver, starburst, vertica. Each driver has:

• `src/metabase/driver/<name>.clj`
• `resources/metabase/<name>/metabase-plugin.yaml`
• `deps.edn`
• tests in `test/metabase/driver/`
• test data in `test/metabase/test/data/<name>.clj`

Put a driver's app-DB access in `metabase.driver.<name>.db`. Only `metabase.driver.bigquery-cloud-sdk.db` exists.

Plugins: `metabase.plugins.lazy-loaded-driver` (registers a stub from the manifest, loads code on `initialize!`), `metabase.plugins.initialize`, `metabase.plugins.dependencies`.

Sync (OSS, `src/metabase/sync/`, see its `README.md`):

• `metabase.sync.core` - `sync-database!`, `sync-table!`, `sync-db-metadata!`, `analyze-db!`, `update-field-values!`.
• `metabase.sync.sync-metadata.*` - tables, fields (`fields.sync-instances`), fks, indexes, dbms-version, timezone.
• `metabase.sync.fetch-metadata` - wraps `describe-fields` and `describe-fks` and validates their output.
• `metabase.sync.analyze.*` - fingerprint, classify, data-sensitivity. Classifiers and fingerprinters live in `metabase.analyze.core`.
• `metabase.sync.field-values`, `metabase.sync.schedules`, `metabase.sync.task.sync-databases`.
• `metabase.sync.db` - sync's app-DB queries.

Models (OSS):

• `metabase.warehouses.models.database`, `metabase.warehouses.db` - Database.
• `metabase.warehouse-schema.models.{table,field,field-values,dimension,field-user-settings,table-user-settings}`, `metabase.warehouse-schema.field-values.distinct-batch`, `metabase.warehouse-schema.db`.
• REST: the `warehouses-rest` and `warehouse-schema-rest` modules, each with its own `db` namespace.

Invariants and landmines

• Hierarchy: `:sql` and `:sql-jdbc` are abstract. `:redshift` inherits from `:postgres`. `:databricks` and `:sparksql` inherit from `:hive-like`. `:bigquery-cloud-sdk` is `:sql` but not JDBC. `:mongo` has no SQL parent. Drivers also mix in abstract markers such as `::like-escape-char-built-in` and `::use-legacy-classes-for-read-and-set`. A change at `:sql` or `:sql-jdbc` hits every child, so check for overrides first.
• Implement `describe-database*`, not `describe-database`. The public method wraps the impl in `do-with-resilient-connection`.
• Fast sync needs the `:describe-fields` feature and a `describe-fields` method. Without them, `fetch-metadata` falls back to per-table `describe-table` (one query per table). `:describe-indexes` works the same way.
• Call `driver.u/supports?`, not `driver/database-supports?` directly. It caches, logs, and survives a slow or throwing driver.
• Read connection details through `metabase.driver.connection/effective-details`, not `(:details db)`. Write, admin, and transform scopes pick other credentials and other pools.
• The pool cache keys each pool by `[database-id connection-type]` and checks it against a hash of the JDBC spec. A changed hash, a closed SSH tunnel, or an expired password invalidates the pool. Before it drops a pool, the cache re-fetches a stale Database instance from a long sync.
• `metabase_field` and `metabase_table` hold what users see. `metabase_field_user_settings` and `metabase_table_user_settings` record which values users set; user edits go through the settings models' write functions, which write both rows. Sync reads the settings row before each update and leaves user-set columns alone.
• Sync is destructive. It deactivates tables missing from `describe-database`, and archives deactivated tables after a threshold. A driver that returns too few tables hides content.
• Field values: `*distinct-limit*` is 1000 rows and `*total-max-length*` caps total size. A field over the limit stops being `auto-list`. Advanced FieldValues (`:sandbox`, `:linked-filter`) expire after `advanced-field-values-max-age`.
• Lazy loading: core code must not require an external driver namespace. The