llm-wiki-newsroom

LLM Wiki Newsroom

A multi-agent AI knowledge base run by a five-role "newsroom" — open-source, local-first, no vendor lock-in. Drop articles, documents, and PDFs into the raw/ folder, type a single command, and the newsroom — powered by an agent like Claude Code — reads them, extracts entities, concepts, and relationships, and organizes everything into a fully cross-referenced wiki, a structured and persistent alternative to RAG. Unlike most takes on the idea, the agent that writes a page is never the one that reviews it, and the authoring guidelines evolve themselves over time. Every new document you add also enriches the existing pages. This repo ships with a small example corpus — the debate over what "open source" means for AI — under wiki/, but the framework is domain-agnostic.

Most knowledge tools leave the finding to you. This project makes the AI read and understand your collected documents first, then organizes them into a wiki — with cross-references between pages, automatic flagging of conflicting claims, and per-topic synthesis built in from the start, so later retrieval is fast.

See the output before installing — the example corpus shipped in this repo is published as a browsable GitHub Wiki (no clone needed). It's a rendered static snapshot of the wiki/ folder; the interactive graph below runs locally.

The interactive knowledge graph (graph/graph.html) — every page a node, every wikilink an edge, color-coded by auto-detected cluster, with a live physics layout and filter/search built in. Shown here on a larger private deployment (~2,300 nodes) to convey how it scales; this repo ships a deliberately small 15-node example corpus you can browse the exact same way. (Interface shown in the optional Korean WIKI_LANG=ko mode.)

The concept

This project is one question worked into a running system: how far can you trust knowledge an AI wrote? Two ideas organize everything below.

1. The product is an LLM Wiki — Andrej Karpathy's three-layer pattern. The original documents you collect (Layer 1, raw/), the cross-linked wiki the agent maintains (Layer 2, wiki/), and the operating rules the agent follows (Layer 3, CLAUDE.md + .claude/) are kept strictly separate, so humans and AI don't trespass into each other's territory. As the operator you do exactly two things — feed Layer 1 and tune Layer 3; only the agent writes Layer 2. And ingesting one document doesn't just add a page: it refreshes the ~10–15 existing pages that mention the same entities and concepts, which is what makes the wiki compound instead of just piling up.

2. The factory is a newsroom running four loops. The wiki is produced by five roles modeled on a newspaper staff — and the agent that writes a page is never the one that reviews it:

Four loops turn that division of labor into trust. The first three nest inside one another; only the fourth sits outside, feeding published pages back in:

%%{init: {"flowchart": {"wrappingWidth": 700}}}%%
flowchart BT
    subgraph meta["Meta loop — mistakes become new rules"]
        subgraph outer["Outer loop — two gates before publication"]
            inner["Inner loop — the writer self-checks while drafting"]
        end
    end
    %% the   padding keeps this box as wide as the Meta loop box above
    reground["  Reground loop — pages that go stale or inconsistent come back as input  "]
    reground -.-> meta
    style meta fill:transparent,stroke:#58a6ff,stroke-width:2px
    style outer fill:transparent,stroke:#3fb950,stroke-width:2px
    style inner fill:transparent,stroke:#d29922,stroke-width:2px
    style reground fill:transparent,stroke:#bc8cff,stroke-width:2px,stroke-dasharray:4 4

One piece of the machinery is deliberately not a loop. A loop is feedback — something exists, gets checked, gets corrected. Before any loop runs, a GROUND Ladder governs the input side: how much the writer reads before drafting, widening from the page's declared dependencies toward the whole corpus only on a named signal that the evidence so far is insufficient. The loops correct what came out; the ladder disciplines what goes in — every authoring cycle enters through it. (Details under Key Features.)

The first three loops mirror the "software factory" playbook for AI-assisted coding; the fourth exists because knowledge, unlike code, keeps decaying after you ship it. The full argument for this design is in the companion article: The Knowledge Factory.

Everything else in this README — the commands, the tools, the feature list — hangs off this map.

What makes this different

There are plenty of takes on Karpathy's LLM Wiki idea now. After reading the popular implementations, three things here are genuinely rare — and they are the bet:

• Authoring guidelines that evolve themselves (the meta loop) — something I haven't found in the other implementations. When the same review failure keeps recurring, the system drafts a fix to its own writing rules. The idea is borrowed from Self-Harness and Microsoft SkillOpt. Where the fix touches prose an LLM judge scores, it has to clear a blind A/B against a regression set first; tool code, fact corrections and structural moves are verified deterministically instead, and I can waive the A/B by directing a change myself, with the reason recorded. So it isn't only the wiki that improves over time, but the rules that build it. (Thirteen weeks of that ledger on a private instance: 107 amendments adopted, 72 rejected, 18 deferred. Of the 107, 16 cite a measurement, 40 were adopted on a recorded waiver, and 51 are changes the A/B doesn't cover or that record no measurement. Over the same window, 44 of the 70 defect classes that ever got a fix came back afterwards, and five of them produced 451 of the 823 recurrences (55%); the rule amended most often has been rewritten 19 times and still fires. The loop is still experimental; I'm measuring whether it earns its keep rather than claiming it's solved.)
• A full newsroom, not just "an agent" (the outer loop) — plenty of tools wrap one agent around your notes, and a few add a verifier. Here authoring and review sit in different hands, the review is held to an editorial rubric drawn from real craft (journalism, consulting, encyclopedic forms) so a different person or session reproduces the same bar, and a two-sided gate means the deterministic lint and the qualitative review must both pass.
• Memex-style associative discovery — saved reading trails and "unexpected connection" surfacing that the other implementations don't carry.

The rest — the knowledge graph, contradiction tracking, cascading updates, plain-markdown/Obsidian output — many LLM-wiki tools have in some form.

Installation

git clone https://github.com/alfadur7/llm-wiki-newsroom.git
cd llm-wiki-newsroom
pip install -r requirements.txt

No API keys, no accounts — the dependencies are the whole setup. If igraph/leidenalg have no wheels for your platform (e.g. Termux/Android), delete those two lines from requirements.txt: everything else still installs, and tools/build.py falls back to a pure-Python partition.

Or click "Use this template" to create your own wiki repo from this scaffold. To start from a clean slate, delete the example pages under wiki/ (keep the folders and graph/cluster_labels.json), then ingest your own sources with /wiki-ingest.

This project assumes an environment where the AI agent reads and edits files and invokes tools on its own. Support levels by agent:

Claude Code-only features include cascading updates that refresh related existing pages whenever a new document is added, a backlink index across all pages, automatic contradiction tracking, and associative discovery that follows connected concepts. AGENTS.md and GEMINI.md carry only the basic workflow from the original SamurAIGPT project. The Python tools used to build the wiki run locally with no external API keys.

Slash Commands (Claude Code)

In Claude Code, typing a /-prefixed command like /wiki-ingest runs the matching workflow. The table below gives each command's role in one line; the Key Features section explains how they work. Argument notation follows <required>, [optional], and a | b | c (pick one); invoking a command with no required argument prints usage and stops.

Core workflow

News & sharing

Associative discovery