/twscrape

TRANSLATED CONTENT:

twscrape

Python library for scraping Twitter/X data using GraphQL API with account rotation and session management.

When to use this skill

Use this skill when:

• Working with Twitter/X data extraction and scraping
• Need to bypass Twitter API limitations with account rotation
• Building social media monitoring or analytics tools
• Extracting tweets, user profiles, followers, trends from Twitter/X
• Need async/parallel scraping operations for large-scale data collection
• Looking for alternatives to official Twitter API

Quick Reference

Installation

pip install twscrape

Basic Setup

import asyncio
from twscrape import API, gather

async def main():
    api = API()  # Uses accounts.db by default

    # Add accounts (with cookies - more stable)
    cookies = "abc=12; ct0=xyz"
    await api.pool.add_account("user1", "pass1", "email@example.com", "mail_pass", cookies=cookies)

    # Or add accounts (with login/password - less stable)
    await api.pool.add_account("user2", "pass2", "email2@example.com", "mail_pass2")
    await api.pool.login_all()

asyncio.run(main())

Common Operations

# Search tweets
await gather(api.search("elon musk", limit=20))

# Get user info
await api.user_by_login("xdevelopers")
user = await api.user_by_id(2244994945)

# Get user tweets
await gather(api.user_tweets(user_id, limit=20))
await gather(api.user_tweets_and_replies(user_id, limit=20))
await gather(api.user_media(user_id, limit=20))

# Get followers/following
await gather(api.followers(user_id, limit=20))
await gather(api.following(user_id, limit=20))

# Tweet operations
await api.tweet_details(tweet_id)
await gather(api.retweeters(tweet_id, limit=20))
await gather(api.tweet_replies(tweet_id, limit=20))

# Trends
await gather(api.trends("news"))

Key Features

1. Multiple API Support

• **Search API**: Standard Twitter search functionality
• **GraphQL API**: Advanced queries and data extraction
• **Automatic switching**: Based on rate limits and availability

2. Async/Await Architecture

# Parallel scraping
async for tweet in api.search("elon musk"):
    print(tweet.id, tweet.user.username, tweet.rawContent)

3. Account Management

• Add multiple accounts for rotation
• Automatic rate limit handling
• Session persistence across runs
• Email verification support (IMAP or manual)

4. Data Models

• SNScrape-compatible models
• Easy conversion to dict/JSON
• Raw API response access available

Core API Methods

Search Operations

`search(query, limit, kv={})`

Search tweets by query string.

**Parameters:**

• `query` (str): Search query (supports Twitter search syntax)
• `limit` (int): Maximum number of tweets to return
• `kv` (dict): Additional parameters (e.g., `{"product": "Top"}` for Top tweets)

**Returns:** AsyncIterator of Tweet objects

**Example:**

# Latest tweets
async for tweet in api.search("elon musk", limit=20):
    print(tweet.rawContent)

# Top tweets
await gather(api.search("python", limit=20, kv={"product": "Top"}))

User Operations

`user_by_login(username)`

Get user information by username.

**Example:**

user = await api.user_by_login("xdevelopers")
print(user.id, user.displayname, user.followersCount)

`user_by_id(user_id)`

Get user information by user ID.

`followers(user_id, limit)`

Get user's followers.

`following(user_id, limit)`

Get users that the user follows.

`verified_followers(user_id, limit)`

Get only verified followers.

`subscriptions(user_id, limit)`

Get user's Twitter Blue subscriptions.

Tweet Operations

`tweet_details(tweet_id)`

Get detailed information about a specific tweet.

`tweet_replies(tweet_id, limit)`

Get replies to a tweet.

`retweeters(tweet_id, limit)`

Get users who retweeted a specific tweet.

`user_tweets(user_id, limit)`

Get tweets from a user (excludes replies).

`user_tweets_and_replies(user_id, limit)`

Get tweets and replies from a user.

`user_media(user_id, limit)`

Get tweets with media from a user.

Other Operations

`list_timeline(list_id)`

Get tweets from a Twitter list.

`trends(category)`

Get trending topics by category.

**Categories:** "news", "sport", "entertainment", etc.

Account Management

Adding Accounts

**With cookies (recommended):**

cookies = "abc=12; ct0=xyz"  # String or JSON format
await api.pool.add_account("user", "pass", "email@example.com", "mail_pass", cookies=cookies)

**With credentials:**

await api.pool.add_account("user", "pass", "email@example.com", "mail_pass")
await api.pool.login_all()

CLI Account Management

# Add accounts from file
twscrape add_accounts accounts.txt username:password:email:email_password

# Login all accounts
twscrape login_accounts

# Manual email verification
twscrape login_accounts --manual

# List accounts and status
twscrape accounts

# Re-login specific accounts
twscrape relogin user1 user2

# Retry failed logins
twscrape relogin_failed

Proxy Configuration

Per-Account Proxy

proxy = "http://login:pass@example.com:8080"
await api.pool.add_account("user", "pass", "email@example.com", "mail_pass", proxy=proxy)

Global Proxy

api = API(proxy="http://login:pass@example.com:8080")

Environment Variable

export TWS_PROXY=socks5://user:pass@127.0.0.1:1080
twscrape search "elon musk"

Dynamic Proxy Changes

api.proxy = "socks5://user:pass@127.0.0.1:1080"
doc = await api.user_by_login("elonmusk")
api.proxy = None  # Disable proxy

**Priority:** `api.proxy` > `TWS_PROXY` env var > account-specific proxy

CLI Usage

Search Operations

twscrape search "QUERY" --limit=20
twscrape search "elon musk lang:es" --limit=20 > data.txt
twscrape search "python" --limit=20 --raw  # Raw API responses

User Operations

twscrape user_by_logi