crawlberg
Turn any website into clean, structured data. Point Crawlberg at a URL and get back Markdown, metadata, and links, from a single page or a whole site, in the language you already use.
Turns PDFs, scanned images, spreadsheets and other documents into clean text, tables and metadata, and splits them into chunks for LLM context windows or search. You can also batch-extract whole folders, pull keywords, and run OCR on scans without a text layer.
Auto-invoked ships a router so the right skill fires automatically as you prompt. No remembering which skill to call.
Normal is the plain upstream plugin, installed as-is. You invoke its skills yourself.
Claude Code
The plugin> /plugin marketplace add xberg-io/xberg> /plugin install xberg@xbergAuto-invocation> /plugin marketplace add flowy-sh/flowy-core> /plugin install flowy-core> /plugin install flowy-xberg
Codex
$ npx skills add xberg-io/xberg -a codex
Cursor
$ npx skills add xberg-io/xberg -a cursor
Gemini CLI
$ npx skills add xberg-io/xberg -a gemini-cli
GitHub Copilot
$ copilot plugin marketplace add xberg-io/xberg$ copilot plugin install xberg@xberg
OpenCode
$ npx skills add xberg-io/xberg -a opencode
Windsurf
$ npx skills add xberg-io/xberg -a windsurf
The fast, precise document-intelligence engine — for every language.
Point Xberg at anything — a PDF, a scanned image, a spreadsheet, an audio file, a URL, a whole archive, or a source tree — and get back clean text, tables, metadata, and structured data. One engine handles format detection, reading, OCR, and extraction, so you never stitch a pipeline together from a dozen libraries.
107 formats · 141 file extensions · 371 code languages · 15 language bindings · 6 output formats · OCR · transcription · embeddings
The fastest, most precise open-source document and PDF-to-Markdown engine — see the benchmarks.
Install · What you get · Capabilities · CLI · Docs
Xberg is the next iteration of Kreuzberg. Same document-intelligence engine, rebuilt and rebranded under a fresh v1 line.
Point Xberg at anything — a PDF, a spreadsheet, a scanned image, an audio file, a URL, an archive, a source tree — and get back clean, structured content you can use right away. One core does the format detection, reading, and extraction, so you don't assemble a pipeline yourself. Call it from Rust, Python, Node.js, Go, Java, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WASM, Kotlin, or C FFI, and run it as a library, CLI tool, REST API, or MCP server.
| Capability | What you get |
|---|---|
| 107 document formats | PDFs, Office, images, HTML, email, e-books, scientific publications, and structured data across 141 file extensions, with intelligent MIME detection and bounded extraction controls. |
| URLs & the web | Point Xberg at an http(s) URL — it fetches and extracts a single document, or crawls and follows links (Auto / Document / Crawl modes via the crawlberg engine). Requires the url-ingestion feature. |
| Audio & video transcription | Speech-to-text from MP3, M4A, WAV, WebM, and MP4 tracks via Whisper ONNX (tiny → large-v3). Requires the transcription feature. |
| Archives, traversed | List and recursively extract nested .zip, .tar, .gz, .7z — documents inside documents — guarded by zip-bomb, compression-ratio, and nesting-depth limits. |
| OCR on demand | Tesseract, PaddleOCR, Candle, or VLM backends — fallback chains, confidence scores, language auto-detection, extensible via plugins. |
| Layout & tables | ML layout models (PP-DocLayout-V3, RT-DETR) and table structure (TATR, SLANet) reconstruct reading order and cell grids for clean Markdown. |
| Code intelligence | Functions, classes, imports, symbols, docstrings from 371 programming languages. Syntax-aware chunking for RAG pipelines. |
| Embeddings & search | Local (ONNX) or provider-hosted embeddings (165 providers via liter-llm), sparse and late-interaction, cross-encoder reranking. |
| Enrichment | NER, keyword extraction (YAKE/RAKE), summarization, translation, redaction, page classification, QR detection, language detection, token reduction (TOON). |
| Structured extraction | Schema-driven JSON straight from any document via local (Ollama, LM Studio, vLLM) or hosted LLMs — no prompt engineering. |
| 6 output formats | Plain text, Markdown, Djot, HTML, JSON tree, or Docling DocTags, plus registered custom renderers. |
| Runs anywhere | Library, CLI (14 commands), REST API (xberg serve), MCP server, Docker, Helm — CPU by default, no GPU required. Content-hash caching, parallel batch, per-file timeouts. |
Capabilities marked requires a feature are Cargo feature flags on the core crate (
url-ingestion,transcription,reranker, layout/ORT). Prebuilt language packages and the Docker image bundle the common set; a from-source build enables only what you select.
Python
pip install xberg
For archived versions and the PyPI storage fallback:
pip install xberg --extra-index-url https://xberg-io.github.io/python-wheels/simple/
PyPI keeps all releases in the current minor and the latest stable patch of each older minor. See the permanent archive and retention policy.
See Python README for full documentation.
Node.js / TypeScript
npm install @xberg-io/xberg
See Node.js README for full documentation.
Rust
cargo add xberg
See Rust README for full documentation.
Go
go get github.com/xberg-io/xberg/packages/go@latest
⚠️ The repository root is not a Go module —
go get github.com/xberg-io/xbergwill fail. Always target the/packages/gosubdirectory as shown above.
See Go README for full documentation.
Java
Available on Maven Central as io.xberg:xberg. See Java README for the dependency snippet.
C#
dotnet add package XbergIo.Xberg
See C# README for full documentation.
Ruby
gem install xberg
See Ruby README for full documentation.
PHP
composer require xberg-io/xberg
See PHP README for full documentation.
Elixir
Add {:xberg, "~> 1.0"} to your mix.exs dependencies. See Elixir README for full documentation.
WebAssembly
npm install @xberg-io/xberg-wasm
See WebAssembly README for full documentation.
Kotlin (Android)
Available on Maven Central as io.xberg:xberg-android. See Kotlin README for the dependency snippet.
Swift
Turn any website into clean, structured data. Point Crawlberg at a URL and get back Markdown, metadata, and links, from a single page or a whole site, in the language you already use.
Reads text from scanned pages, photos and PDFs and returns it as structured Markdown or JSON, so you can pull tables, formulas and headers out of a document without retyping them. Covers many languages and works as input for AI tools.
Gives your AI assistant a memory that persists across sessions, so it remembers your preferences and earlier work in this repository instead of starting fresh. Search, forget, pause and resume memories with simple commands.
Gives AI agents a library of ready-made skills for scientific work such as metabolic flux analysis, protein experiment design, time series modeling, and analytical method validation, so research tasks need less hand-built code. It works with any agent that supports the open Agent Skills standard.
Gives Claude Code skills, agents, and commands for PostHog analytics, session replay, feature flags, error tracking, and LLM observability. Includes MCP server setup and project access checks.
Install it once and the right skill fires as you prompt, with no slash command to remember. It is listed on Flowy in the Data category. It includes xberg-batch-extraction, xberg-chunking, xberg-extracting-keywords. Free and open source.
Is this plugin yours?
Claim it with GitHubSubmit a pluginPromote it