/extract

name: extract
description: Extract structured data from websites and produce an executable Playwright script plus extracted data. Use when the user wants to scrape, extract, pull, collect, or harvest data from any website — product listings, tables, search results, feeds, profiles, or any repeating content.

When to Use This Skill

Activate when the user wants to **obtain data** from a website:

• "Extract all product prices from this page"
• "Scrape the table of results from ..."
• "Pull the list of authors and titles from arXiv search results"
• "Collect all job listings from this page"
• "Get the data from this dashboard table"
• "Harvest review scores from ..."
• "Download all the links/images/cards from ..."

The deliverable is always **two artifacts**:

1. **Executable Playwright script** — a standalone `.cjs` file that reproduces the extraction without Actionbook at runtime. 2. **Extracted data** — JSON (default), CSV, or user-specified format written to disk.

Decision Strategy

Use Actionbook as a **conditional accelerator**, not a mandatory step. The goal is reliable selectors in the shortest path.

User request
  │
  ├─► actionbook search "<site> <intent>"
  │     ├─ Results with Health Score ≥ 70%  ──► actionbook get "<ID>" ──► use selectors
  │     └─ No results / low score  ──► Fallback
  │
  └─► Fallback: actionbook browser open <url>
        ├─ actionbook browser snapshot   (accessibility tree → find selectors)
        ├─ actionbook browser screenshot (visual confirmation)
        └─ manual selector discovery via DOM inspection

**Priority order for selector sources:**

| Priority | Source | When | |----------|--------|------| | 1 | `actionbook get` | Site is indexed, health score ≥ 70% | | 2 | `actionbook browser snapshot` | Not indexed or selectors outdated | | 3 | DOM inspection via screenshot + snapshot | Complex SPA / dynamic content |

**Non-negotiable rule:** if `search + get` already provides usable selectors for required fields, start from `get` selectors and do not jump to full fallback (`snapshot`/`screenshot`) by default. Exception: lightweight mechanism probes (for hydration/virtualization/pagination) are allowed when runtime behavior may affect script correctness. Escalate to `snapshot`/`screenshot` only when probes/sample validation indicate selector gaps or instability.

Mechanism-Aware Script Strategy

Websites use patterns that break naive scraping. The generated Playwright script **must** account for these:

Streaming / SSR / RSC hydration

Pages may render a shell first, then stream or hydrate content.

// Wait for hydration to complete — not just DOMContentLoaded
await page.waitForSelector('[data-item]', { state: 'attached' });
await page.waitForFunction(() => {
  const items = document.querySelectorAll('[data-item]');
  return items.length > 0 && !document.querySelector('[data-pending]');
});

**Detection cues:** React root with `data-reactroot`, Next.js `__NEXT_DATA__`, empty containers that fill after JS runs. If `actionbook browser text "<selector>"` returns empty but the screenshot shows content, hydration hasn't completed.

Virtualized lists / virtual DOM

Only visible rows exist in the DOM. Scrolling renders new rows and destroys old ones.

// Scroll-and-collect loop for virtualized lists (scroll container aware)
const allItems = [];
const maxScrolls = 50;
let scrolls = 0;

const container = await page.$('<scroll-container-selector>');
if (!container) throw new Error('Scroll container not found');

let previousTop = await container.evaluate(el => el.scrollTop);
while (scrolls < maxScrolls) {
  const items = await page.$$eval('[data-row]', rows =>
    rows.map(r => ({ text: r.textContent.trim() }))
  );
  for (const item of items) {
    if (!allItems.find(i => i.text === item.text)) allItems.push(item);
  }

  await container.evaluate(el => el.scrollBy(0, 600));
  await page.waitForTimeout(300);

  const currentTop = await container.evaluate(el => el.scrollTop);
  if (currentTop === previousTop) break;

  previousTop = currentTop;
  scrolls += 1;
}

**Detection cues:** Container has fixed height with `overflow: auto/scroll`, row count in DOM is much smaller than stated total, rows have `transform: translateY(...)` or `position: absolute; top: ...px`.

Infinite scroll / lazy loading

New content appends when the user scrolls near the bottom.

// Scroll to bottom until no new content loads (with no-growth tolerance)
let itemCount = 0;
let noGrowthStreak = 0;
const maxScrolls = 80;
let scrolls = 0;

while (scrolls < maxScrolls && noGrowthStreak < 3) {
  await page.evaluate(() => window.scrollTo(0, document.body.scrollHeight));
  await page.waitForTimeout(1200);

  const newCount = await page.$$eval('.item', els => els.length);
  if (newCount > itemCount) {
    itemCount = newCount;
    noGrowthStreak = 0;
  } else {
    noGrowthStreak += 1;
  }

  scrolls += 1;
}

**Detection cues:** Intersection Observer in page JS, "Load more" button, sentinel element at bottom, network requests firing on scroll.

Pagination

Multi-page results behind "Next" buttons or numbered pages.

// Click-through pagination (navigation-aware, SPA-safe)
const allData = [];
const maxPages = 50;
let pageIndex = 0;
while (pageIndex < maxPages) {
  const pageData = await page.$$eval('.result-item', items =>
    items.map(el => ({ title: el.querySelector('h3')?.textContent?.trim() }))
  );
  allData.push(...pageData);

  const nextBtn = await page.$('a.next-page:not([disabled])');
  if (!nextBtn) break;

  const previousUrl = page.url();
  const previousFirstItem = await page
    .$eval('.result-item', el => el.textContent?.trim() || '')
    .catch(() => '');

  await nextBtn.click();

  // Post-click detection only: advance must be caused by this click
  const advanced = await Promise.any([
    page
      .waitForURL(url => url.toString() !== previousUrl, { timeout: 5000