{"id":28998618,"url":"https://github.com/luminati-io/twitter-scraper","last_synced_at":"2026-02-10T05:34:06.241Z","repository":{"id":275926470,"uuid":"927625598","full_name":"luminati-io/twitter-scraper","owner":"luminati-io","description":"Twitter (X) Scraper provides a free scraper for small-scale projects and an enterprise-grade API for large-scale Twitter data extraction , with built-in proxy rotation and CAPTCHA solving.","archived":false,"fork":false,"pushed_at":"2025-02-05T09:49:43.000Z","size":1575,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-22T07:02:03.739Z","etag":null,"topics":["captcha-solver","proxy-server","scraper-api","twitter","twitter-scraper","web-scraper","web-scraping","x","x-scraper"],"latest_commit_sha":null,"homepage":"https://brightdata.com/products/web-scraper/twitter","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/luminati-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-05T09:16:35.000Z","updated_at":"2025-02-10T22:33:01.000Z","dependencies_parsed_at":"2025-02-05T10:47:17.474Z","dependency_job_id":null,"html_url":"https://github.com/luminati-io/twitter-scraper","commit_stats":null,"previous_names":["luminati-io/twitter-scraper"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/luminati-io/twitter-scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luminati-io%2Ftwitter-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luminati-io%2Ftwitter-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luminati-io%2Ftwitter-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luminati-io%2Ftwitter-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/luminati-io","download_url":"https://codeload.github.com/luminati-io/twitter-scraper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luminati-io%2Ftwitter-scraper/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261823776,"owners_count":23215150,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["captcha-solver","proxy-server","scraper-api","twitter","twitter-scraper","web-scraper","web-scraping","x","x-scraper"],"created_at":"2025-06-25T07:09:21.746Z","updated_at":"2026-02-10T05:34:06.161Z","avatar_url":"https://github.com/luminati-io.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Twitter (X) Data Scraper by Bright Data\n\n[![Promo](https://github.com/luminati-io/LinkedIn-Scraper/raw/main/Proxies%20and%20scrapers%20GitHub%20bonus%20banner.png)](https://brightdata.com/products/web-scraper/twitter)\n\nThis repository provides two distinct methods for collecting Twitter data:\n1. **Free Twitter Scraper**: For small-scale projects and learning\n2. [**Enterprise Twitter Scraper API**](https://brightdata.com/products/web-scraper/twitter): For production-grade data extraction\n\n## Table of Contents\n1. [Free Twitter Scraper](#free-twitter-scraper)\n    - [Profile Scraping](#1-profile-scraping)\n    - [Post Scraping](#2-post-scraping)\n    - [Limitations](#limitations)\n2. [Twitter Scraper API](#twitter-scraper-api)\n    - [Key Features](#key-features)\n    - [Quick Start Guide](#quick-start-guide)\n    - [Scrape Posts by URL](#1-scrape-posts-by-url)\n    - [Scrape Profile Data](#2-scrape-profile-data)\n    - [Date-Range Tweet Collection](#3-date-range-tweet-collection)\n3. [No-Code Scraper Option](#no-code-scraper-option)\n4. [Data Collection Approaches](#data-collection-approaches)\n5. [Support \u0026 Resources](#support--resources)\n\n\n## Free Twitter Scraper\nIdeal for small-scale projects, experiments, and learning purposes.\n\n### 1. Profile Scraping\nExtracts public profile data from Twitter, including names, followers, tweet counts, and more.\n\n#### Input Requirements:\n| Parameter  | Type  | Required | Description                          |\n|------------|-------|----------|--------------------------------------|\n| usernames  | list  | Yes      | List of Twitter handles to scrape   |\n\n#### Implementation:\n```python\n# free_scraper/twitter_profiles.py\nusernames = [\n    \"satyanadella\",\n    \"BillGates\", \n    \"elonmusk\"\n]\n```\n\n#### Sample Output (CSV):\n\u003cimg width=\"700\" alt=\"twitter_profiles_data\" src=\"https://github.com/luminati-io/twitter-scraper/blob/main/Images/408877618-450d920a-4760-463b-9670-8ac1264b6409.png\" /\u003e\n\n### 2. Post Scraping\nCollect engagement metrics for specific tweets\n\n#### Input Requirements:\n\n| Parameter  | Type  | Required | Description                        |\n|------------|-------|----------|------------------------------------|\n| tweet_ids  | list  | Yes      | List of Twitter post IDs to scrape |\n\n#### Implementation:\n```python\n# free_scraper/twitter_posts.py\nTWEET_IDS = [\n    \"1882412972414214544\",\n    \"1879604491764297992\",\n    \"1869427646368792599\"\n]\n```\n\n#### Sample Output (JSON):\n```json\n{\n  \"tweet_id\": \"1882412972414214544\",\n  \"author\": \"MongoDB\",\n  \"content\": \"We're excited to announce...\",\n  \"engagement\": {\n    \"likes\": 29,\n    \"retweets\": 10,\n    \"views\": 3417\n  }\n}\n```\n\n### Limitations\nThe free method is not recommended for large-scale scraping due to Twitter’s strict anti-bot protections. Some key limitations include:\n- **Rate Limiting:** Twitter blocks requests after a few scrapes.\n- **IP Blocking:** Frequent scraping from the same IP can lead to bans.\n- **Limited Scalability:** Not suitable for high-volume data collection.\n- **Restricted Data Fields:** Only provides basic profile and tweet data, with no advanced filtering options.\n\n## Twitter Scraper API\nA robust, scalable, and reliable solution for large-scale Twitter data extraction. Designed for businesses and developers who need high-quality, real-time data without infrastructure headaches.\n\n### Key Features\n- **Scalable \u0026 Reliable:** Optimized for high-volume and real-time data collection\n- **Anti-Blocking:** Built-in [proxy rotation](https://brightdata.com/solutions/rotating-proxies) and [CAPTCHA solving](https://brightdata.com/products/web-unlocker/captcha-solver)\n- **Legal Compliance:** Fully GDPR and CCPA compliant\n- **Global Coverage:** Access data from any region or language\n- **Real-Time Data:** Fresh data with minimal latency\n- **Advanced Filtering:** Customize data extraction with precise filters\n- **Pay-as-You-Go:** Only pay for successful responses\n- **Free Trial:** Includes 20 free API calls to get started\n- **Dedicated Support:** 24/7 technical assistance\n\n👉 **Learn more:** [Bright Data Twitter Scraper API](https://brightdata.com/products/web-scraper/twitter)\n\n### Quick Start Guide\n- **Sign Up:** Create a [Bright Data account](https://brightdata.com/)\n- **Get API Token:** Obtain your [API key](https://docs.brightdata.com/general/account/api-token) from the dashboard\n- **Choose Endpoint:** Select from the available API endpoints below\n\n### 1. Scrape Posts by URL\nExtract detailed engagement metrics and content for specific tweets using their URLs.\n\n\u003cimg width=\"700\" alt=\"twitter-posts-scraper\" src=\"https://github.com/luminati-io/twitter-scraper/blob/main/Images/409213804-9d07a475-2e3b-45fc-ae8e-ebcd7cef367b.png\" /\u003e\n\n\n#### Request Parameters:\n| Field | Type   | Required | Description            |\n|-------|--------|----------|------------------------|\n| `url`   | string | Yes      | Full Twitter post URL |\n\n#### Example Request:\n```python\n# twitter_api/twitter_posts.py\nposts = [\n    {\"url\": \"https://x.com/OpenAI/status/1885406586136383634\"},\n    {\"url\": \"https://x.com/CNN/status/1796673270344810776\"}\n]\n```\n\n#### Response Schema:\n```json\n{\n    \"post\": {\n        \"author\": {\n            \"name\": \"OpenAI\",\n            \"followers\": 3930255,\n            \"profileImage\": \"https://pbs.twimg.com/profile_images/1885410181409820672/ztsaR0JW_normal.jpg\",\n            \"bio\": \"OpenAI's mission is to ensure that artificial general intelligence benefits all of humanity. We're hiring: https://t.co/dJGr6Lg202\",\n        },\n        \"content\": {\n            \"text\": \"OpenAI o3-mini is now available in ChatGPT and the API.\\n\\nPro users will have unlimited access to o3-mini and Plus \u0026 Team users will have triple the rate limits (vs o1-mini).\\n\\nFree users can try o3-mini in ChatGPT by selecting the Reason button under the message composer.\",\n            \"postedAt\": \"2025-01-31T19:15:33.000Z\",\n            \"id\": \"1885406586136383634\",\n        },\n        \"engagement\": {\n            \"replies\": 1004,\n            \"reposts\": 1997,\n            \"likes\": 13420,\n            \"views\": 2858777,\n            \"quotes\": 684,\n            \"bookmarks\": 1546,\n        },\n    }\n}\n```\n👉 Only key fields are shown here. See the [full JSON response](https://github.com/luminati-io/Twitter-Scraper/blob/main/twitter_data/twitter_posts.json) for all details. Try our dedicated [Twitter Posts Scraper](https://brightdata.com/products/web-scraper/twitter/posts) now.\n\n### 2. Scrape Profile Data\nExtract comprehensive profile information, including recent posts and engagement metrics.\n\n\u003cimg width=\"600\" alt=\"twitter-profile-scraper\" src=\"https://github.com/luminati-io/twitter-scraper/blob/main/Images/409214197-3b3e2f0f-30bc-45d9-b9bc-13358b22a55a.png\" /\u003e\n\n#### Request Parameters:\n\n| Field      | Type   | Required | Description                     |\n|------------|--------|----------|---------------------------------|\n| `url`        | string | Yes      | Twitter profile URL       |\n| `max_number_of_posts`  | number | No       | Number of recent posts to retrieve |\n\n#### Example Request:\n```python\n# twitter_api/twitter_profile_posts.py\nprofiles = [\n    {\"url\": \"https://x.com/satyanadella\", \"max_number_of_posts\": 50},\n    {\"url\": \"https://x.com/BillGates\", \"max_number_of_posts\": 35}\n]\n```\n\n#### Response Schema:\n```json\n{\n    \"profile\": {\n        \"name\": \"Satya Nadella\",\n        \"handle\": \"satyanadella\",\n        \"role\": \"Chairman and CEO at Microsoft\",\n        \"isVerified\": true,\n        \"profileImage\": \"https://pbs.twimg.com/profile_images/1221837516816306177/_Ld4un5A_normal.jpg\",\n        \"website\": \"http://www.microsoft.com/ceo\",\n        \"joinedDate\": \"2009-02-11T04:45:34.000Z\",\n        \"stats\": {\"following\": 286, \"followers\": 3356268, \"postsCount\": 1859},\n    },\n    \"posts\": [\n        {\n            \"id\": \"1807114709499523208\",\n            \"content\": \"What a final!!! Congrats, India, and well played, South Africa. Super World Cup... let us have more cricket in the West Indies and USA!!\",\n            \"postedAt\": \"2024-06-29T18:11:35.000Z\",\n            \"engagement\": {\n                \"replies\": 712,\n                \"reposts\": 10785,\n                \"likes\": 133591,\n                \"views\": 2029775,\n            },\n        },\n        {\n            \"id\": \"1726509045803336122\",\n            \"content\": \"We remain committed to our partnership with OpenAI and have confidence in our product roadmap, our ability to continue to innovate with everything we announced at Microsoft Ignite, and in continuing to support our customers and partners. We look forward to getting to know Emmett\",\n            \"postedAt\": \"2023-11-20T07:53:28.000Z\",\n            \"engagement\": {\n                \"replies\": 4524,\n                \"reposts\": 14480,\n                \"likes\": 89773,\n                \"views\": 41675720,\n            }\n        },\n    ],\n}\n```\n👉 Only key fields are shown here. See the [full JSON response](https://github.com/luminati-io/Twitter-Scraper/blob/main/twitter_data/twitter_profile_posts.json) for all details. Try our dedicated [Twitter Profile Scraper](https://brightdata.com/products/web-scraper/twitter/profile) now. \n\n### 3. Date-Range Tweet Collection\nRetrieve posts within a specific date range.\n\n#### Request Parameters:\n\n| Parameter  | Type   | Required | Description                   |\n|------------|--------|----------|-------------------------------|\n| `url`        | string | Yes      | Twitter Profile URL          |\n| `start_date` | string | Yes      | Start date (ISO format)       |\n| `end_date`   | string | Yes      | End date (ISO format)         |\n\n\n#### Example Request:\n```python\n# twitter_api/twitter_posts_date_range.py\nprofiles = [\n    {\n        \"url\": \"https://x.com/satyanadella\",\n        \"start_date\": \"2025-01-15T09:00:00.000Z\",\n        \"end_date\": \"2025-01-31T23:00:00.000Z\",\n    },\n    {\"url\": \"https://x.com/cnn\", \"start_date\": \"2025-01-01\", \"end_date\": \"2025-01-15\"},\n    {\"url\": \"https://x.com/fabrizioromano\", \"start_date\": \"\", \"end_date\": \"\"},\n]\n```\n\n#### Response Schema:\n```json\n{\n    \"post\": {\n        \"author\": {\n            \"name\": \"Satya Nadella\",\n            \"handle\": \"satyanadella\",\n            \"role\": \"Chairman and CEO at Microsoft\",\n            \"isVerified\": false,\n            \"profileImage\": \"https://pbs.twimg.com/profile_images/1221837516816306177/_Ld4un5A_normal.jpg\",\n            \"followers\": 3356282,\n            \"following\": 286,\n        },\n        \"content\": {\n            \"text\": \"For me, cricket is maybe the only thing that could possibly come close to watching the Excel World Champ on ESPN!\",\n            \"images\": [\"https://pbs.twimg.com/media/GiF4nePaEAAHQg5.jpg\"],\n            \"externalUrl\": \"https://techcommunity.microsoft.com/blog/excelblog/congrats-to-the-winners-of-this-years-mewc--mecc/4355651\",\n            \"postedAt\": \"2025-01-24T22:29:46.000Z\",\n            \"id\": \"1882918743409676719\",\n        },\n        \"engagement\": {\n            \"replies\": 102,\n            \"reposts\": 208,\n            \"likes\": 3416,\n            \"views\": 286901,\n            \"quotes\": 39,\n            \"bookmarks\": 127,\n        },\n    }\n}\n```\n👉 Only key fields are shown here. See the [full JSON response](https://github.com/luminati-io/Twitter-Scraper/blob/main/twitter_data/twitter_date_range_posts.json) for all details.\n\n## No-Code Scraper Option\nFor users preferring a graphical interface, we offer a no-code solution through our control panel:\n\n- Configure scrapers in minutes\n- Automates the entire data collection process\n- Direct result download (multiple formats)\n\nFor detailed instructions on using the no-code scraper, please visit our [Getting Started guide](https://github.com/luminati-io/Twitter-Scraper/blob/main/no-code-scraper.md).\n\n\n## Data Collection Approaches\nYou can use the following parameters to fine-tune your results:\n| **Parameter**       | **Type**   | **Description**                                            | **Example**                  |\n|---------------------|------------|------------------------------------------------------------|------------------------------|\n| `limit`             | `integer`  | Max results per input                                   | `limit=10`                   |\n| `include_errors`    | `boolean`  | Get error reports for troubleshooting                     | `include_errors=true`        |\n| `notify`            | `url`      | Webhook notification URL to be notified upon completion  | `notify=https://notify-me.com/` |\n| `format`            | `enum`     | Output format (e.g., JSON, NDJSON, JSONL, CSV)         | `format=json`                |\n\n💡 **Pro Tip:** You can also select whether to deliver the data to an [external storage](https://docs.brightdata.com/scraping-automation/web-data-apis/web-scraper-api/overview#via-deliver-to-external-storage) or to deliver it to a [webhook](https://docs.brightdata.com/scraping-automation/web-data-apis/web-scraper-api/overview#via-webhook).\n\n## Support \u0026 Resources\n- **API Documentation:** [Bright Data Docs](https://docs.brightdata.com/scraping-automation/web-scraper-api/trigger-a-collection)\n- **Scraping Best Practices:** [Avoid Getting Blocked](https://brightdata.com/blog/web-data/web-scraping-without-getting-blocked)\n- **Technical Support:** [Contact Us](mailto:support@brightdata.com)\n\n---\n\n**Interested in other scrapers? Check out the list below:**\n\n- [LinkedIn Scraper](https://github.com/luminati-io/LinkedIn-Scraper)\n- [Google News Scraper](https://github.com/luminati-io/Google-News-Scraper)\n- [Google Maps Scraper](https://github.com/luminati-io/Google-Maps-Scraper)\n- [Amazon Scraper](https://github.com/luminati-io/Amazon-scraper)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluminati-io%2Ftwitter-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fluminati-io%2Ftwitter-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluminati-io%2Ftwitter-scraper/lists"}