/document-workflows

name: document-workflows
description: >
  Use this skill for building end-to-end document processing workflows and
  pipelines using LandingAI ADE. Trigger when users need to:
  (1) Process batches of documents in parallel or async,
  (2) Build classify-then-extract pipelines for mixed document types,
  (3) Prepare parsed documents for RAG systems with chunking and vector DB ingestion,
  (4) Load extraction results into databases like Snowflake or export to CSV/DataFrames,
  (5) Visualize extraction results: draw bounding box overlays on pages, crop
  chunk images, or highlight/annotate specific words or phrases found in documents,
  (6) Build Streamlit or web UIs for document processing,
  (7) Find and highlight specific terms within document sections using word-level
  grounding (e.g. highlight "L2S" in the Introduction, redact PII, annotate
  extracted values on the original page).
  This skill complements the document-extraction skill which covers ADE SDK basics.
  Use document-extraction to write code that executes parse/extract/split operations with more precision and less cost than adding the document image to the prompt and asking the LLM to find the relevant info.
  Use document-workflows when composing those operations into pipelines,
  or when you need visualization, annotation, or word-level grounding on
  parsed documents.

Document Workflows — ADE Pipeline Patterns

Overview

This skill provides **reusable building blocks** for composing LandingAI ADE primitives (parse, extract, split) into production-ready document processing pipelines. It complements the `document-extraction` skill:

| Concern | `document-extraction` | `document-workflows` | |---------|----------------------|---------------------| | Scope | ADE SDK API: parse, extract, split, grounding | End-to-end pipelines: batch, RAG, DB, classify-route | | When | Need to call a single ADE operation | Need to compose operations into a workflow | | Code | SDK method calls with parameters | Complete functions with error handling, parallelism | | Deps | `landingai-ade` only | + workflow-specific libs (pandas, chromadb, etc.) |

**Philosophy:** Organize by *workflow pattern* (batch, RAG, DB insertion), not by document type. The same pattern applies whether documents are invoices, utility bills, or medical forms.

---

Step 0 (mandatory) — Pre-Flight Document Exploration {#pre-flight}

**Run this before writing any pipeline code** whenever working with documents whose internal structure has not already been inspected in this session.

> **Rule: never write section-detection, heading-matching, or text-search code > without first running Tool 2 (diagnostic parse) on the sample document. > Heading format is document-specific and cannot be inferred from the task > description or document type alone — the only reliable way to know it is to > look at the actual ADE output.** > > Common surprises: a paper's "Introduction" heading may appear as > `1. Introduction` (plain text, no `#`), `## Introduction`, `INTRODUCTION` > (all-caps), or embedded inside a text chunk with body copy. Getting this > wrong means a silent failure (zero chunks matched) that requires a full > re-parse to debug.

Run Tool 1 (visual render) and Tool 2 (diagnostic parse) on 1–3 representative sample documents before writing any code. This takes under a minute and prevents debugging iterations that a pre-flight would have avoided.

Tool 1 — Visual page render

Render 1–2 pages as PNG and read them as visual context. No ADE credits used, but each PNG consumes context tokens. Use when layout is ambiguous or document origin is unknown (handwriting? scan? form?).

.venv/bin/python - << 'EOF'
import pymupdf
from pathlib import Path
from PIL import Image

pdf = Path('path/to/sample.pdf')
out_dir = Path('/tmp/ade_preflight'); out_dir.mkdir(exist_ok=True)
doc = pymupdf.open(pdf)
for pg in range(min(2, len(doc))):   # first 2 pages only
    pix = doc[pg].get_pixmap(matrix=pymupdf.Matrix(1.5, 1.5))   # 108 DPI
    img = Image.frombytes("RGB", [pix.width, pix.height], pix.samples)
    out = out_dir / f"{pdf.stem}_page{pg + 1}.png"
    img.save(out)
    print(out)
doc.close()
EOF

Then read the saved PNGs. Immediately answers:

• Are headings **bold text** (→ ADE may output plain-text heading, not `# Heading`)
• Is the document handwritten or scanned? → Tesseract OCR needed, not PyMuPDF
• Single-column or two-column layout?
• Any noise: running headers, page numbers, watermarks, stamps?

Tool 2 — ADE diagnostic parse

Parses 1 sample and prints markdown structure + chunk inventory. Uses ADE credits — keep to **1–3 samples only**, never the full corpus.

.venv/bin/python - << 'EOF'
import os
from pathlib import Path
from collections import Counter
from dotenv import load_dotenv

# Load API key: prefer existing env var, then .env file lookup
load_dotenv()  # Load API key from .env. Add a path to the .env if needed.

from landingai_ade import LandingAIADE
client = LandingAIADE()
pr = client.parse(document=Path('path/to/sample.pdf'))

print("=== MARKDOWN (first 80 lines) ===")
for i, ln in enumerate(pr.markdown.splitlines()[:80], 1):
    print(f"{i:3}: {ln}")

print("\n=== CHUNKS ===")
for ch in pr.chunks:
    txt = (ch.markdown or '').replace('\n', ' ')[:70]
    b = ch.grounding.box
    print(f"p{ch.grounding.page} {ch.type:12} "
          f"l={b.left:.2f} t={b.top:.2f} r={b.right:.2f} b={b.bottom:.2f} | {txt}")

print(f"\nPages: {pr.metadata.page_count}  "
      f"Chunks: {len(pr.chunks)}  "
      f"Types: {dict(Counter(ch.type for ch in pr.chunks))}")
EOF

> **Cost note:** Save the parse result with `pr.model_dump()` to a JSON file > after the first run. Load it for later development instead of calling > `client.parse()` again. Only re-parse when the document set changes.

What to look for

| Observation | Implication | |-------------|-------------| | Heading is `1. Introduction` (plain text, no `#`) | ADE markdown won't us