/night-market-research-frontier

name: night-market-research-frontier
description: Map open problems where this repo can advance SOTA. Use when scoping research. Do not use to run the campaign; use night-market-completion-integrity-campaign.

Night Market Research Frontier

This file lists the open problems where this repository holds assets that the published state of the art does not. SOTA (state of the art) here means the best result shipped or published anywhere, not the best result in this repo. Every entry is a candidate. Nothing below is a claimed capability, and citing this file as evidence that a capability exists is an error.

Read this skill when choosing what research bet to place next, when framing an experiment, or when someone asks "what could this project contribute beyond itself?"

Ground rules

Follow these before starting any problem below.

• Everything here stays labeled open or candidate until it clears the

repo evidence bar: one mechanism must explain all observations including negatives, the hypothesis must predict numbers before the run, and the generator is never its own judge. The pipeline from hunch to accepted result is night-market-research-methodology.

• Experiments are changes. They go through the same gates as any other

change (night-market-change-control). This skill authorizes no shortcuts.

• When a problem produces an accepted result, write the dated synthesis

in `docs/research/`, update the changelog, and remove or re-scope the entry here. A frontier list that never shrinks is a wish list.

Problem index

| # | Problem | Primary repo asset | Status | |---|---------|--------------------|--------| | 1 | Completion integrity in autonomous loops | egregore gate + herald judge + imbue verifier-integrity | Open, active campaign | | 2 | Skill-graph governance at scale | ratchets + skill graph (activation harness removed) | Open | | 3 | Collective memory across context resets | ADR-0007 Discussions + memory-palace | Open, partially blocked | | 4 | Insight-palace bridge under a hook budget | Draft spec v0.1.0 + hook infrastructure | Open, spec drafted | | 5 | Behavioral contract attestation | ADR-0008 SLSA path + trust workflow | Open | | 6 | Continuation for autonomous loops | egregore night-run driver + watchdog + bounded Stop hook | Open, cost baseline measured |

1. Completion integrity in autonomous loops

Why current SOTA fails

Autonomous agents self-report "done." The evidence is inlined here and in the two docs of record it was folded into (`.claude/rules/prefer-invariants-over-fallbacks.md` for the harness-loop findings, `plugins/imbue/skills/proof-of-work/modules/verifier-integrity.md` for the verifier findings): the METR randomized trial (arXiv 2507.09089) found experienced developers 19% slower with AI while believing they were faster, so self-assessment of completion is miscalibrated even for humans in the loop. On the verifier side, a green check proves spec-satisfaction, not correctness: the spec can be wrong, or the check can be hollow (a test that passes no matter what the code does). An agent that judges its own work optimizes the judge, not the work. No published harness binds "done" to gates the agent cannot fake.

This repo's specific asset

Three shipped, tested mechanisms that most agent frameworks lack:

• egregore's opt-in completion-integrity gate:

`completion_integrity: bool = False` in `plugins/egregore/scripts/config.py` (commit 83281337, default off). When true, a "fix-required" quality verdict blocks the ship step and merge is held for human review regardless of `auto_merge`. The raw-JSON opt-in path is covered by tests (commit cd903cbf).

• herald's deterministic-first Stop-hook judge:

`plugins/herald/hooks/double_shot_latte.py`. Deterministic verdict by default. An optional LLM second shot fires only on the single ambiguous outcome and is capped at `LLM_TIMEOUT_SECONDS = 8` inside the 10s registered hook budget (commits 3d22f02a, 268cff89).

• imbue's verifier-integrity module:

`plugins/imbue/skills/proof-of-work/modules/verifier-integrity.md` (commit 29081fda): proves the check was worth passing, distinct from proving it passed.

First three steps in this repo

1. Read the executable plan in night-market-completion-integrity-campaign. That skill owns the campaign. This entry only frames the research question. 2. Run egregore on a small manifest with the gate on (raw-JSON opt-in, `completion_integrity: true`) and again with it off, on the same work items. Log every quality verdict. 3. Compare false-done rates: items the ungated loop shipped that the gated loop held as fix-required, adjudicated by a human.

You have a result when

A measured false-done rate delta between gated and ungated runs on the same work items exists, with the human adjudication recorded, and the delta survives a second run. If the delta is zero or the gate holds only items a human calls genuinely done, the gate as designed is falsified: record that too. The promotion question (default-off to default-on) is open until this number exists.

2. Skill-graph governance at scale

Why current SOTA fails

This repo carries 197 registered skills (198 SKILL.md files on disk, find count 2026-07-02) against a finite skill discovery budget of about 16K characters. Skills past the budget are dropped silently (`docs/quality-gates.md`, Follow-on work section). The activation layer does near-keyword matching, so relevant skills can fail to fire. No one, here or elsewhere, has published a principled activation-quality metric: a way to say "this skill library activates the right skill X% of the time, and change Y moved that number."

This repo's specific asset

• A removed activation-lift harness, recoverable from history with

`git checkout 7566cb323^ -- prototypes/forced-eval`. It ran labeled prompts through `claude -p` with and without a forced-eval hook and applied a paired McNemar test. Its one live