/python-sdk-best-practices

name: brightdata-sdk
description: |
  Web data extraction and discovery using the Bright Data Python SDK.
  Use when user asks to "scrape", "get data from", "extract", "search for",
  or "find" information from websites. Also use when user mentions specific
  platforms like Amazon, LinkedIn, Instagram, Facebook, TikTok, YouTube,
  Reddit, Pinterest, Zillow, Crunchbase, or DigiKey, or asks for "bulk data",
  "historical data", or "dataset". Covers scraping, searching, datasets,
  and browser automation.
metadata:
  author: brightdata
  version: "1.0"

Bright Data SDK

Access web data through a unified Python SDK. One client, eight service categories: platform scraping, platform search, web search (SERP), AI-powered discovery, datasets, web unlocking, browser automation, and scraper studio.

Always use the client as a context manager. In synchronous environments (scripts, notebooks, Claude Code), use `SyncBrightDataClient`. In async environments, use `BrightDataClient`. Both use the same method names — the sync client wraps calls automatically. Note: the sync client currently has limited platform coverage — see the sync compatibility note in `references/scrapers.md` for details. For unsupported platforms or the datasets API, use the async client (`BrightDataClient`).

Service Selection (decide first, then look up the specific method)

Use this decision tree to pick the right service BEFORE reaching for any specific method. Most routing failures come from skipping this step and pattern-matching on user keywords instead.

Have a URL?
  ├── On a supported platform (Amazon, LinkedIn, Facebook, Instagram, YouTube,
  │   TikTok, Reddit, ChatGPT, Perplexity, Pinterest, DigiKey)?
  │     → Platform scraping: client.scrape.<platform>.<method>(url=...)
  │
  ├── Generic page (not on any supported platform)?
  │     → Web unlocker: client.scrape_url(url=...)
  │
  └── Need login / JavaScript / click-scroll-fill / CAPTCHA / multi-step navigation?
        → Browser API: client.browser.get_connect_url() (then connect with Playwright)

No URL?
  ├── Want entities matching natural-language criteria
  │   ("find AI startups in Berlin", "competitors of Acme Corp", "people who worked at X")?
  │     → Discover: client.discover(query=..., intent=...)
  │
  ├── Want web pages / articles / search-result links
  │   ("search Google for X", "find pages about Y")?
  │     → SERP: client.search.google(query=...) [or .bing / .yandex]
  │
  ├── Want to search WITHIN a specific platform
  │   ("find products on Amazon", "TikTok videos by hashtag", "Pinterest pins about recipes")?
  │     → Platform search: client.search.<platform>.<method>(...)
  │
  └── Want bulk historical data at scale
      ("all LinkedIn companies in tech", "historical Amazon prices", "every Zillow listing in Texas")?
        → Datasets: client.datasets.<name>(filter=...) (then .download(snapshot_id))

Edge cases:

• URL on supported platform BUT user explicitly mentions login/click/scroll/JS → Browser API (the interaction trumps the platform).
• URL on supported platform BUT scraper returns 403/blocked → fall back to `client.scrape_url()` (web unlocker).
• "Find/research/who are X" with a URL alongside (e.g., "find competitors of acme.com") → still Discover; the URL is context, not the scrape target.

Method Names: Verify Before Asserting

Before claiming any platform method exists, doesn't exist, or asserting a platform isn't supported, you MUST consult `references/scrapers.md` first. Past evals show the model has hallucinated method names (e.g., `client.scrape.linkedin.people` — does NOT exist; use `.profiles`) and falsely claimed platforms unsupported (e.g., Pinterest is supported via both `client.scrape.pinterest` and `client.search.pinterest`).

**Rule:** load `references/scrapers.md` before naming any specific platform method. The reference file lists every platform, every method signature, and the sync/async availability matrix. Verify, don't assume. If you've already loaded `references/scrapers.md` in this session, consult what's in context — no need to reload.

**Known hallucinations** (these names do NOT exist in the SDK — the model has invented them in past evals):

| Hallucinated | Correct replacement | |---|---| | `client.scrape.linkedin.people(...)` | `client.scrape.linkedin.profiles(url=...)` | | `client.scrape.instagram.users(...)` | `client.scrape.instagram.profiles(url=...)` | | `client.list_datasets()` | `client.datasets.list()` | | `asyncio.gather(*[client.scrape.X.<quick>(...) for ...])` | Trigger pattern — see Batch gotcha |

This list grows as new hallucinations are observed in evals. If you're tempted to write a method name that "feels right" but you haven't seen in `references/scrapers.md`, treat it as a likely hallucination — load the reference and verify.

Useful Standalone Methods

Methods that don't belong to any specific workflow — easy to overlook because they're not tied to a platform or a routing decision. The model has hallucinated some of these (`client.list_datasets()` instead of `client.datasets.list()`); use the canonical names below.

| Method | What it does | |---|---| | `client.datasets.list()` | List all 310+ datasets at runtime. Do NOT use `dir()` or introspection — use this method. | | `client.discover(query=, intent=)` | AI-ranked entity search (companies, people, products). See Service Selection above. | | `client.scrape_url(url=...)` | Web unlocker for any URL. Use for sites without a dedicated platform scraper. | | `client.browser.get_connect_url()` | CDP WebSocket URL for Playwright / Puppeteer / Selenium. | | `client.list_zones()` | List active Bright Data zones. | | `client.delete_zone(name)` | Remove a zone. | | `client.test_connection()` | Verify the API token works. | | `client.get_account_info()` | Usage, quotas, active zones. |

How to Handle Requests

Exploring capabilities

If the user wants to know what's available or asks "what can this do?", describe th