llm-wiki-compiler

LLM Wiki Compiler

A Claude Code and Codex-compatible plugin that compiles knowledge into a topic-based wiki — from scattered markdown files or entire codebases. Reduce context costs by ~90% and give your agent a synthesized understanding of any project.

Documentation

What's New in v2.1

• Codex-compatible plugin metadata — install from the same plugin/ package root
• Skill-first Codex workflows — use natural prompts instead of Claude slash commands
• Shared session context helper — one wiki context renderer for Claude hooks and Codex guidance

What's New in v2.0

• Codebase mode — generate wikis from code repositories, not just markdown files
• Auto-detection — /wiki-init detects whether you're in a codebase or knowledge project
• Knowledge graph visualization — interactive canvas-based graph of your wiki

Inspiration

This plugin implements the LLM Knowledge Base pattern described by Andrej Karpathy:

"Raw data from a given number of sources is collected, then compiled by an LLM into a .md wiki, then operated on by various CLIs by the LLM to do Q&A and to incrementally enhance the wiki, and all of it viewable in Obsidian. You rarely ever write or edit the wiki manually, it's the domain of the LLM."

The key insight: instead of re-reading hundreds of raw files every session, have the LLM compile them into topic-based articles once, then query the synthesized wiki. Knowledge compounds instead of fragmenting.

What It Does

You have 100+ files across meetings, strategy docs, codebases, and research. Every Claude session re-reads them. This plugin compiles them into topic-based articles that synthesize everything known about each subject — with backlinks to sources.

Before: Read 13+ raw files (~3,200 lines) per session After: Read INDEX + 2 topic articles (~330 lines) per session

How It Works

flowchart LR
    subgraph Sources["📁 Raw Sources (you own)"]
        M1["meetings/"]
        M2["strategy/"]
        M3["research/"]
        M4["notes/"]
    end

    subgraph Commands["⚡ Commands"]
        INIT["/wiki-init<br/>samples files<br/>proposes structure"]
        CAPTURE["/wiki-capture<br/>link + context"]
        COMPILE["/wiki-compile<br/>batch compilation"]
        INGEST["/wiki-ingest<br/>single file, interactive"]
        SEARCH["/wiki-search<br/>find anything"]
        LINT["/wiki-lint<br/>health check"]
    end

    subgraph Wiki["📖 Compiled Wiki (LLM owns)"]
        INDEX["INDEX.md"]
        T1["topics/retention.md"]
        T2["topics/onboarding.md"]
        T3["topics/..."]
        C1["concepts/..."]
        SCHEMA["schema.md"]
    end

    Sources -->|"383 files · 13 MB"| COMPILE --> Wiki
    Sources -->|"1 link + context"| CAPTURE --> Wiki
    Sources -->|"1 file"| INGEST --> Wiki
    INIT -->|"creates config +<br/>article structure"| COMPILE
    Wiki -->|"13 articles · 161 KB<br/>84% fewer tokens"| LLM["🤖 Your LLM Agent<br/>reads wiki, not raw files"]
    LINT -.->|"checks"| Wiki
    SEARCH -.->|"searches"| Wiki

Install

Clone the repo

git clone https://github.com/ussumant/llm-wiki-compiler.git

Claude Code

# Add as a local marketplace
claude plugin marketplace add /path/to/llm-wiki-compiler

# Install the plugin
claude plugin install llm-wiki-compiler

# Restart Claude Code for hooks to register

For a single Claude session without installing:

claude --plugin-dir /path/to/llm-wiki-compiler/plugin

Codex

This repo includes a Codex plugin manifest at plugin/.codex-plugin/plugin.json and local marketplace metadata at .agents/plugins/marketplace.json. Add this repository as a local Codex plugin marketplace, then install LLM Wiki Compiler from that marketplace.

Codex does not use Claude slash commands. Invoke the same workflows with prompts:

Workflow	Claude Code	Codex prompt
Initialize	/wiki-init	"Initialize a wiki for this repo"
Global setup	/wiki-global-init	"Set up my global wiki"
Capture link	/wiki-capture URL --context "why it matters"	"URL — capture this in my wiki"
Compile	/wiki-compile	"Compile changed sources into the wiki"
Ingest	/wiki-ingest path/to/file.md	"Ingest this source into the wiki: path/to/file.md"
Search	/wiki-search architecture decisions	"Search the compiled wiki for architecture decisions"
Query	/wiki-query what do we know about retention?	"Answer from the compiled wiki: what do we know about retention?"
Lint	/wiki-lint	"Lint the compiled wiki"
Visualize	/wiki-visualize	"Launch the wiki knowledge graph"
Migrate	/wiki-migrate	"Show a wiki-first startup migration report"

The shared session context helper is available at plugin/hooks/wiki-session-context. Claude calls it automatically through the SessionStart hook; Codex users can ask Codex to read the compiled wiki at session start until Codex hook registration is standardized.

Quick Start

Claude Code

# 1. Initialize — auto-detects whether this is a codebase or knowledge project,
#    samples your files, proposes a domain-specific article structure
/wiki-init

# 2. Compile — reads all sources, creates topic articles (5-10 min first run)
/wiki-compile

# 3. Visualize — launch an interactive knowledge graph of your wiki
/wiki-visualize

# 4. Browse in Obsidian — open wiki/INDEX.md to see all topics with backlinks

After setup, Claude reads wiki articles automatically at session start — no special commands needed. The wiki updates incrementally when sources change.

No plugin? Different agent? You can compile a wiki with any LLM agent (Codex, Cursor, Gemini, a plugin-less Claude) using a single self-contained, project-agnostic protocol file. Deploy it with deploy-protocol.ps1 and point the agent at it — no skill or template files required. See EXPORTING.md.

Codex

Ask Codex:

Initialize a wiki for this repo
https://example.com/article — capture this in my wiki
Compile changed sources into the wiki
Launch the wiki knowledge graph

For ongoing sessions, ask Codex to start with wiki/INDEX.md, then read relevant topic articles before raw sources.

Global and Local Wikis

LLM Wiki Compiler supports two layers:

Layer	Default location	Use it for
Global wiki	~/Knowledge	Cross-project links, ideas, research, videos, bookmarks, and reusable patterns
Local wiki	Current repo/folder with .wiki-compiler.json	Deep project-specific architecture, decisions, source files, and operating knowledge

The default global folder is:

~/Knowledge

You can override it:

export LLM_WIKI_GLOBAL_DIR="$HOME/CompanyKnowledge"

Routing defaults:

• "capture this in my wiki" → global wiki
• "capture this in this repo/project wiki" → local wiki
• "capture this in both" → global and local
• no global wiki yet → initialize ~/Knowledge automatically from the global wiki template

Set it up explicitly with:

Set up my global wiki

Codebase Mode (New in v2.0)

Generate a wiki from a code repository — not just markdown files, but the full knowledge embedded in your codebase: architecture, API contracts, decision records, deployment configs, and gotchas.

Quick Start

# One command to set up and compile
/wiki-init --codebase

The compiler auto-detects your project type, discovers modules/services, finds knowledge files (READMEs, ADRs, API specs, Docker configs), and compiles everything into topic articles.

What It Scans

File Type	Examples	What It Captures
Documentation	README.md, CLAUDE.md, ARCHITECTURE.md	Purpose, architecture, conventions
API contracts	*.proto, *.graphql, openapi.yaml	API surface, inter-service communication
Decision records	ADR-*.md, docs/adr/*.md	Key decisions and rationale
Infrastructure	docker-compose.yml, Dockerfile, k8s/*.yaml	Deployment topology, scaling
Operations	docs/runbooks/*.md, CHANGELOG.md	Gotchas, failure modes, version history
Config shape	.env.example, package.json	Environment requirements, dependencies

With deep_scan: true, it also reads entry points, type definitions, and route files for richer articles.

Example Output

# auth-service

## Purpose [coverage: high -- 8 sources]
Handles user authentication and session management via JWT tokens.
All other services call auth-service to validate requests.

## Talks To [coverage: high -- 6 sources]
- **user-service** (REST: /api/users/:id) -- subscription status lookup
- **notification-service** (SQS: auth.password-reset) -- triggers email
- **billing-service** (gRPC) -- validates payment status before premium access

## Key Decisions [coverage: medium -- 3 sources]
- **JWT over sessions** -- stateless scaling, no shared session store (ADR-003)
- **Refresh token rotation** -- security requirement from compliance audit

## Gotchas [coverage: high -- 5 sources]
- Token expiry is 15 minutes, not 1 hour (changed in v2.3)
- Rate limiting on /auth/login is per-IP, not per-user

How It Differs from Google Code Wiki / DeepWiki

Those tools answer "what does this code do?" by parsing functions and generating API docs.

This tool answers "what does this project know?" by synthesizing documentation, decision records, deployment configs, and operational knowledge into articles an agent can navigate.

	Google Code Wiki / DeepWiki	LLM Wiki Compiler
Input	Source code (AST parsing)	Knowledge files + optional code
Output	API docs + architecture diagrams	Synthesized topic articles with coverage indicators
Infrastructure	Hosted platform / server + embeddings	Zero infra — Claude Code plugin
Updates	Full regen on every commit	Incremental — only changed topics
Consumer	Developers reading docs	Your AI agent (and you)

Monorepo / Microservice Support

For monorepos, the compiler detects service boundaries by looking for directories with their own manifest files (package.json, go.mod, etc.). Each service becomes a topic article. Cross-cutting concerns (infrastructure, testing, deployment) get their own articles.

📚 Wiki compiled — 8 topics from 47 files

  Topics created:
  ├── auth-service (12 sources)
  ├── billing-service (9 sources)
  ├── notification-service (6 sources)
  ├── api-gateway (5 sources)
  ├── infrastructure (4 sources)
  ├── testing (3 sources)
  └── deployment (3 sources)

  Concepts discovered:
  ├── error-handling-strategy — shared pattern across 4 services
  └── auth-flow — touches auth, gateway, billing

Codebase Configuration

{
  "version": 2,
  "mode": "codebase",
  "name": "My Project",
  "sources": [{ "path": "./", "exclude": ["node_modules/", "dist/", ".git/", "wiki/"] }],
  "output": "wiki/",
  "service_discovery": "auto",
  "deep_scan": false,
  "knowledge_files": ["README.md", "CLAUDE.md", "*.proto", "openapi.*", "ADR-*.md", "Dockerfile"]
}

Field	Description
mode	"codebase" enables code-aware topic discovery
service_discovery	"auto" detects monorepo vs single project
deep_scan	true to also read source code files for richer articles
knowledge_files	Glob patterns for priority documentation files

Knowledge Graph Visualization