/walkeros-understanding-stores

name: walkeros-understanding-stores
description:
  Use when working with walkerOS stores, understanding key-value storage in
  flows, or learning about store injection via env. Covers interface, lifecycle,
  $store. wiring, and available store packages.

Understanding walkerOS Stores

Overview

Stores provide key-value storage that other components (sources, transformers, destinations) consume via environment injection. They are the 4th component type in Flow.Json alongside sources, transformers, and destinations.

**Core principle:** Stores are passive infrastructure. They don't process events or participate in chains — they provide state that other components read and write.

Store interface

See [packages/core/src/types/store.ts](../../packages/core/src/types/store.ts) for the canonical interface.

Instance

| Property | Type | Purpose | Required | | --------- | ---------------------------------- | ---------------------------- | ------------ | | `type` | `string` | Store type identifier | **Required** | | `config` | `Store.Config` | Settings and env | **Required** | | `get` | `(key) => StoreValue \| undefined` | Read a value | **Required** | | `set` | `(key, value, ttl?) => void` | Write a value (optional TTL) | **Required** | | `delete` | `(key) => void` | Remove a value | **Required** | | `destroy` | `DestroyFn` | Cleanup on shutdown | Optional |

All methods can be sync or async (return `Promise`).

Value type and `file` mode

Stores hold one canonical value type: structured data (`StoreValue`), with binary (`Uint8Array`) as a first-class leaf. `StoreValue` is `string | number | boolean | null | Uint8Array | StoreValue[] | { [key: string]: StoreValue }` (`undefined` is reserved as the "miss" sentinel and is never a stored value). A shared core codec (`serializeStoreValue` / `deserializeStoreValue`) round-trips that value to and from each backing.

`Store.Config.file?: boolean` (default `false`) picks the mode, decided once at init:

• **Structured (default):** values are `StoreValue` data, serialized by the

shared codec.

• **File (`file: true`):** a byte-native backend (fs, S3, GCS) persists raw

bytes byte-exact. `set()` accepts a `Uint8Array` or `string` and stores it untouched; `get()` hands the exact bytes back. Use for serving assets such as walker.js. The Sheets store is structured-only and rejects `file: true` at init.

TTL is owned by the cache layer, not the store. The store persists values; a `cache` wrapper manages expiry. `flow_validate` warns when a store sets both `file: true` and `cache`, and when a `@walkeros/server-transformer-file` is wired to a byte-native store that does not set `file: true`.

Init function (context pattern)

Stores use the same context-based init pattern as other components:

import type { Store } from '@walkeros/core';

export const storeMyStore: Store.Init = (context) => {
  const { config, env, logger, id } = context;
  const settings = config.settings || {};

  return {
    type: 'my-store',
    config: context.config as Store.Config,
    get(key) {
      /* ... */
    },
    set(key, value, ttl) {
      /* ... */
    },
    delete(key) {
      /* ... */
    },
    destroy() {
      /* cleanup */
    },
  };
};

**Context contains:**

| Property | Type | Purpose | | ----------- | -------------------- | ------------------------- | | `config` | `Store.Config` | Settings from flow config | | `env` | `Store.Env` | Environment dependencies | | `logger` | `Logger.Instance` | Scoped logger | | `id` | `string` | Store identifier | | `collector` | `Collector.Instance` | Reference to collector |

Lifecycle

Stores have the simplest lifecycle of all component types:

Startup:  Stores → Destinations → Transformers → Sources
Shutdown: Sources → Destinations → Transformers → Stores

• **Init first:** Stores init before all other components so they're available

when sources/transformers/destinations start

• **Destroy last:** Stores destroy after all other components so transformers

can flush during their own destroy

• **No lazy init:** Unlike destinations, stores don't support `require` or

deferred activation — they are always eager

Wiring stores via `$store.`

Bundled mode (flow.json)

Use `$store.storeId` in a component's `env` to inject a store instance:

{
  "stores": {
    "data": {
      "package": "@walkeros/server-store-fs",
      "config": { "settings": { "basePath": "./data" } },
      "cache": { "rules": [{ "ttl": 60 }] }
    }
  },
  "transformers": {
    "fingerprint": {
      "package": "@walkeros/server-transformer-fingerprint",
      "env": { "store": "$store.data" }
    }
  }
}

The bundler resolves `$store.data` to a runtime reference. Invalid references are caught at build time. `walkeros validate` also catches typos at validation time, including unknown store names and the colon-instead-of-dot mistake (e.g. `$store:data` is flagged with the suggested form `$store.data`).

The `cache` field enables the built-in in-memory cache tier (`__cache`) on top of any backing store. No separate memory store declaration is needed.

Integrated mode (TypeScript)

Pass the store definition object itself, no `$store.` prefix:

import { startFlow } from '@walkeros/collector';
import { storeFsInit } from '@walkeros/server-store-fs';
import { transformerFingerprint } from '@walkeros/server-transformer-fingerprint';

const data = {
  code: storeFsInit,
  config: { settings: { basePath: './data' } },
  cache: { rules: [{ ttl: 60 }] },
};

await startFlow({
  stores: { data },
  transformers: {
    fingerprint: {
      c