{"id":50795484,"url":"https://github.com/scrape-do/scrapedo-scrapers","last_synced_at":"2026-06-12T14:02:26.395Z","repository":{"id":272766289,"uuid":"917681002","full_name":"scrape-do/scrapedo-scrapers","owner":"scrape-do","description":"Web scraping examples with Scrape.do 😎","archived":false,"fork":false,"pushed_at":"2026-04-16T07:39:28.000Z","size":370,"stargazers_count":17,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-04-16T09:32:54.996Z","etag":null,"topics":["antibot","crawler","datascraping","proxies","python","scraper","spider","web-crawler","web-scraping","webscraper"],"latest_commit_sha":null,"homepage":"https://scrape.do","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scrape-do.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-16T12:59:34.000Z","updated_at":"2026-04-16T07:39:31.000Z","dependencies_parsed_at":"2025-12-17T00:06:26.468Z","dependency_job_id":null,"html_url":"https://github.com/scrape-do/scrapedo-scrapers","commit_stats":null,"previous_names":["scrape-do/scrapedo-scrapers"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/scrape-do/scrapedo-scrapers","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrape-do%2Fscrapedo-scrapers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrape-do%2Fscrapedo-scrapers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrape-do%2Fscrapedo-scrapers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrape-do%2Fscrapedo-scrapers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scrape-do","download_url":"https://codeload.github.com/scrape-do/scrapedo-scrapers/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrape-do%2Fscrapedo-scrapers/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34247461,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-12T02:00:06.859Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["antibot","crawler","datascraping","proxies","python","scraper","spider","web-crawler","web-scraping","webscraper"],"created_at":"2026-06-12T14:02:24.462Z","updated_at":"2026-06-12T14:02:26.373Z","avatar_url":"https://github.com/scrape-do.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scrape.do Scrapers\n\n_Stop relying on overpriced APIs for popular domains!_ 🤚🏻\n\nThis repository contains scraping samples for the most scraped domains on the web, with full-length tutorials and functioning scrapers mainly built with Python and NodeJS (_NodeJS files included in separate folders_).\n\nScrapers in this repository are programmed to help you understand the basics of scraping these popular websites and serving as a base for you to customize and use in your own projects.\n\nFor Python, you'll be using: \n- __Python 3.8+__, \n- [__Requests__](https://github.com/psf/requests) for handling HTTP requests, \n- [__Beautiful Soup__](https://pypi.org/project/beautifulsoup4/) for parsing information, \n- and [__urllib.parse__](https://docs.python.org/3/library/urllib.parse.html) for URL encoding.\n\nThis repository also uses [Scrape.do](https://scrape.do/)'s web scraping API, __to bypass Web Application Firewalls (WAFs)__ of the domains you're scraping and also __interacting with the browser__.\n\n## Available Scrapers\n\n| Domain | Status | Scraped Data | Tutorial | Last Updated |\n|--------|--------|---------|----------|--------------|\n| [Amazon (External)](https://github.com/scrape-do/amazon-scraper) | 🟢 Working | • Product data from single pages\u003cbr\u003e• Product variants with different sizes\u003cbr\u003e• Search results and ads | [Scrape Product Data from Amazon](https://scrape.do/blog/amazon-scraping/) | Dec 18, 2025 |\n| [eBay](./ebay-scraper/) | 🟢 Working | • Basic product listings\u003cbr\u003e• Product reviews with pagination\u003cbr\u003e• Search results with multiple layouts\u003cbr\u003e• Product variants and hidden data | [Scrape Products, Reviews, and Variants from eBay](https://scrape.do/blog/ebay-scraping/) | Aug 1, 2025 |\n| [Etsy](./etsy-scraper/) | 🟢 Working | • Category and search result listings\u003cbr\u003e• Single product details via JSON-LD\u003cbr\u003e• Product reviews via internal API\u003cbr\u003e• Pricing, ratings, and seller information | [Scrape Products and Reviews from Etsy](https://scrape.do/blog/etsy-scraping/) | Oct 30, 2025 |\n| [Google AI Overview](./google-ai-overview-scraper/) | 🟢 Working | • AI Overview text blocks and references\u003cbr\u003e• Deferred state handling via SERP API\u003cbr\u003e• Playwright browser extraction | [Scrape AI Overview Summaries from Google](https://scrape.do/blog/scrape-google-ai-overview/) | Mar 19, 2026 |\n| [Google Maps](./google-maps-scraper/) | 🟢 Working | • Business listings via protobuf API\u003cbr\u003e• Place details, hours, and contact info\u003cbr\u003e• Customer reviews with sorting and pagination | [Scrape Business Data and Reviews from Google Maps](https://scrape.do/blog/google-maps-scraping/) | Mar 19, 2026 |\n| [Google News](./google-news-scraper/) | 🟢 Working | • News tab headlines and metadata\u003cbr\u003e• Full article content as markdown\u003cbr\u003e• Top stories via SERP API | [Scrape Headlines and Articles from Google News](https://scrape.do/blog/google-news-scraping/) | Mar 19, 2026 |\n| [Google Scholar](./google-scholar-scraper/) | 🟢 Working | • Search results with paper metadata\u003cbr\u003e• Article download as markdown\u003cbr\u003e• Author profiles and citation stats | [Scrape Papers and Author Profiles from Google Scholar](https://scrape.do/blog/google-scholar-scraping/) | Mar 19, 2026 |\n| [Google Search](./google-search-scraper/) | 🟢 Working | • First page organic results\u003cbr\u003e• All paginated organic results\u003cbr\u003e• Paid search ads and FAQs\u003cbr\u003e• Related search terms\u003cbr\u003e• SERP API structured search | [Scrape Organic Results, Ads, and FAQs from Google](https://scrape.do/blog/scraping-google-search-results/) | Mar 19, 2026 |\n| [Google Shopping](./google-shopping-scraper/) | 🟢 Working | • Search results with async pagination\u003cbr\u003e• Product details via hidden OAPV endpoint\u003cbr\u003e• Seller offers, reviews, and forums\u003cbr\u003e• Full pipeline with JSON export\u003cbr\u003e• AI Mode shopping recommendations | [Scrape Products, Sellers, and Reviews from Google Shopping](https://scrape.do/blog/google-shopping-scraping/) | Mar 19, 2026 |\n| [Google Trends](./google-trends-scraper/) | 🟢 Working | • Interest over time and by region\u003cbr\u003e• Related queries (top and rising)\u003cbr\u003e• Trending Now topics with news articles | [Scrape Trends, Regional Data, and Related Queries from Google](https://scrape.do/blog/google-trends-scraping/) | Mar 19, 2026 |\n| [Bing](./bing-scraper/) | 🟢 Working | • Organic web search results with pagination\u003cbr\u003e• Image search results with download option\u003cbr\u003e• Shopping/product listings | [Scrape Search Results, Images, and Products from Bing](https://scrape.do/blog/cineworld-scraping/) | Oct 21, 2025 |\n| [G2](./g2-scraper/) | 🟢 Working | • Company and product data\u003cbr\u003e• Product reviews\u003cbr\u003e• Search results and categories | [Scrape Company Data and Reviews from G2](https://scrape.do/blog/g2-scraping/) | Aug 5, 2025 |\n| [DoorDash](./doordash-scraper/) | 🟢 Working | • Consumer address registration\u003cbr\u003e• Restaurant and store listings\u003cbr\u003e• Category-specific products\u003cbr\u003e• Full menu catalogs | [Scrape Restaurants, Products, and Menus from DoorDash](https://scrape.do/blog/doordash-scraping/) | Jul 17, 2025 |\n| [Best Buy](./bestbuy-scraper/) | 🟢 Working | • Product listings with Selenium\u003cbr\u003e• Product data with cloud rendering\u003cbr\u003e• Brand, price, rating, SKU, and URL extraction | [Scrape Products from Best Buy](https://scrape.do/blog/best-buy-scraping/) | Feb 18, 2026 |\n| [Uber Eats](./ubereats-scraper/) | 🟢 Working | • Store listings via API\u003cbr\u003e• Store listings via frontend\u003cbr\u003e• Restaurant menus\u003cbr\u003e• Chain store products | [Scrape Stores, Menus, and Products from Uber Eats](https://scrape.do/blog/ubereats-scraping/) | Aug 1, 2025 |\n| [Zillow](./zillow-scraper/) | 🟢 Working | • Property listing details\u003cbr\u003e• Pricing and market data | [Scrape Property Data from Zillow](https://scrape.do/blog/zillow-scraping/) | Aug 5, 2025 |\n| [Naver](./naver-scraper/) | 🟢 Working | • E-commerce product pages\u003cbr\u003e• Product data via API\u003cbr\u003e• Paid search advertisements\u003cbr\u003e• Organic search results\u003cbr\u003e• Image search results | [Scrape Products and Search Results from Naver](https://scrape.do/blog/naver-scraping/) | Sep 1, 2025 |\n| [Chewy](./chewy-scraper/) | 🟢 Working | • Pet product information\u003cbr\u003e• Names and pricing | [Scrape Pet Products from Chewy](https://scrape.do/blog/chewy-scraping/) | Aug 5, 2025 |\n| [Mouser](./mouser-scraper/) | 🟢 Working | • Electronic component data\u003cbr\u003e• Product names and pricing | [Scrape Electronic Components from Mouser](https://scrape.do/blog/mouser-scraping/) | Aug 5, 2025 |\n| [Fnac](./fnac-scraper/) | 🟢 Working | • French retail product data\u003cbr\u003e• Names and pricing | [Scrape Products from Fnac](https://scrape.do/blog/fnac-scraping/) | Aug 5, 2025 |\n| [Ubersuggest](./ubersuggest-scraper/) | 🟢 Working | • Bearer token generation\u003cbr\u003e• Keyword overview data\u003cbr\u003e• SERP results\u003cbr\u003e• Automated keyword analysis | [Scrape Keyword Data from Ubersuggest](https://scrape.do/blog/ubersuggest-scraping/) | Jun 23, 2025 |\n| [Zomato](./zomato-scraper/) | 🟢 Working | • Restaurant information\u003cbr\u003e• Delivery menu data | [Scrape Restaurant Data from Zomato](https://scrape.do/blog/zomato-scraping/) | Jul 2, 2025 |\n| [Allegro](./allegro-scraper/) | 🟢 Working | • Polish e-commerce products\u003cbr\u003e• Names, pricing, and ratings | [Scrape Products from Allegro](https://scrape.do/blog/allegro-scraping/) | Aug 5, 2025 |\n| [AUTODOC](./AUTODOC-scraper/) | 🟢 Working | • Automotive parts data\u003cbr\u003e• Product details and pricing | [Scrape Auto Parts from AUTODOC](https://scrape.do/blog/autodoc-de-scraping/) | Aug 5, 2025 |\n| [AutoScout24](./autoscout24-scraper/) | 🟢 Working | • Car listing information\u003cbr\u003e• Vehicle names and pricing | [Scrape Car Listings from AutoScout24](https://scrape.do/blog/autoscout24-scraping/) | Aug 5, 2025 |\n| [Idealista](./idealista-scraper/) | 🟢 Working | • Spanish real estate listings\u003cbr\u003e• Property details, pricing, and location data\u003cbr\u003e• Search results with pagination | [Scrape Property Data from Idealista](https://scrape.do/blog/idealista-scraping/) | Aug 6, 2025 |\n| [FastPeopleSearch](./fastpeoplesearch-scraper/) | 🟢 Working | • Person information lookup\u003cbr\u003e• Names, ages, addresses, and ZIP codes\u003cbr\u003e• Phone numbers and email addresses\u003cbr\u003e• Aliases and relatives | [Scrape People Data from FastPeopleSearch](https://scrape.do/blog/fast-people-search-scraping/) | Feb 24, 2026 |\n| [HungerStation](./hungerstation-scraper/) | 🟢 Working | • Restaurant listings\u003cbr\u003e• Store menu items | [Scrape Restaurants and Menus from HungerStation](https://scrape.do/blog/hunger-station-scraping/) | Jul 17, 2025 |\n| [Imovelweb](./imovelweb-scraper/) | 🟢 Working | • Brazilian real estate data\u003cbr\u003e• Property details and pricing | [Scrape Properties from Imovelweb](https://scrape.do/blog/imovelweb-scraping/) | Aug 5, 2025 |\n| [Klium](./klium-scraper/) | 🟢 Working | • Tool and equipment data\u003cbr\u003e• Product names, pricing, and stock | [Scrape Tools from Klium](https://scrape.do/blog/klium-scraping/) | Aug 5, 2025 |\n| [MSC Direct](./mscdirect-scraper/) | 🟢 Working | • Industrial supply products\u003cbr\u003e• Brand, names, and pricing | [Scrape Industrial Supplies from MSC Direct](https://scrape.do/blog/mscdirect-scraping/) | Aug 5, 2025 |\n| [TicketMaster](./ticketmaster-scraper/) | 🟢 Working | • Event details and schedules\u003cbr\u003e• Dates, venues, and locations | [Scrape Events from TicketMaster](https://scrape.do/blog/ticketmaster-scraping/) | Aug 5, 2025 |\n| [TruePeopleSearch](./truepeoplesearch-scraper/) | 🟢 Working | • Person information lookup\u003cbr\u003e• Names, ages, addresses, and ZIP codes\u003cbr\u003e• Phone numbers and email addresses\u003cbr\u003e• Aliases and relatives | [Scrape People Data from TruePeopleSearch](https://scrape.do/blog/true-people-search-scraping/) | Feb 24, 2026 |\n| [Walmart](./walmart-scraper/) | 🟢 Working | • Individual product details with store-specific pricing\u003cbr\u003e• Product variations (sizes, colors, packs)\u003cbr\u003e• Category pages with pagination\u003cbr\u003e• Price and stock change tracking | [Scrape Complete Grocery Data from Walmart](https://scrape.do/blog/walmart-scraping/) | Sep 2, 2025 |\n| [Zoro](./zoro-scraper/) | 🟢 Working | • Industrial business supplies\u003cbr\u003e• Product names and manufacturer data | [Scrape Business Supplies from Zoro](https://scrape.do/blog/zoro-scraping/) | Aug 5, 2025 |\n| [Cineworld](./cineworld-scraper/) | 🟢 Working | • Movie screenings and showtimes\u003cbr\u003e• Ticket prices for all screening types | [Scrape Screenings and Ticket Prices from Cineworld](https://scrape.do/blog/cineworld-scraping/) | Aug 27, 2025 |\n| [Regmovies](./regmovies-scraper/) | 🟢 Working | • Regal Cinemas movie screenings\u003cbr\u003e• Ticket prices and availability | [Scrape Screenings and Ticket Prices from Regmovies](https://scrape.do/blog/regmovies-com-scraping/) | Aug 27, 2025 |\n| [Redfin](./redfin-scraper/) | 🟢 Working | • Property details\u003cbr\u003e• Local listings and details | [Scrape Property Details and Search Results from Redfin](https://scrape.do/blog/redfin-scraping/) | Oct 03, 2025 |\n| [PeopleSearchNow](./peoplesearchnow-scraper/) | 🟢 Working | • Person information lookup\u003cbr\u003e• Names, ages, and addresses | - | Feb 19, 2026 |\n| [SearchPeopleFree](./search-people-free-scraper/) | 🟢 Working | • Person information lookup\u003cbr\u003e• Names, ages, addresses, and ZIP codes\u003cbr\u003e• Phone numbers and email addresses\u003cbr\u003e• Spouse and family members | [Scrape Person Data from SearchPeopleFree](https://scrape.do/blog/search-people-free-scraping/) | Feb 24, 2026 |\n| [WhitePages](./white-pages-scraper/) | 🟢 Working | • Person information lookup\u003cbr\u003e• Names, ages, and addresses\u003cbr\u003e• Phone numbers and ZIP codes | [Scrape Person Data from WhitePages](https://scrape.do/blog/white-pages-scraping/) | Feb 24, 2026 |\n| [Reddit](./reddit-scraper/) | 🟢 Working | • Subreddit posts with pagination\u003cbr\u003e• Search results with cursor-based pagination\u003cbr\u003e• Comments matching search queries | [Scrape Posts, Search Results, and Comments from Reddit](https://scrape.do/blog/reddit-scraping/) | Mar 11, 2026 |\n| [Rightmove](./rightmove-scraper/) | 🟢 Working | • Paginated property listing results\u003cbr\u003e• Address, price, type, bedrooms, agent, and phone\u003cbr\u003e• Full property details via embedded PAGE_MODEL JSON | [Scrape Property Listings and Details from Rightmove](https://scrape.do/blog/rightmove-scraping/) | Apr 6, 2026 |\n| [Wayfair](./wayfair-scraper/) | 🟢 Working | • Single product details with images and reviews\u003cbr\u003e• Category pages with pagination and deduplication\u003cbr\u003e• Full variation map (color × size) with per-variation pricing and stock | [Scrape Products, Categories, and Variations from Wayfair](https://scrape.do/blog/wayfair-scraping/) | Apr 6, 2026 |\n| [Alibaba](./alibaba-scraper/) | 🟢 Working | • Paginated category listings via embedded JSON\u003cbr\u003e• Full product details with price tiers, MOQ, and attributes\u003cbr\u003e• Paginated reviews via signed MTOP API requests | [How to Scrape Alibaba Product Listings, Prices, and Reviews](https://scrape.do/blog/alibaba-scraping/) | Apr 8, 2026 |\n| [DigiKey](./digikey-scraper/) | 🟢 Working | • Paginated category listings via LZ-String cursor pagination\u003cbr\u003e• Full product details with pricing tiers and dynamic attributes | [How to Scrape DigiKey: Product Data, Pricing, and Specifications](https://scrape.do/blog/digikey-scraping/) | Apr 8, 2026 |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrape-do%2Fscrapedo-scrapers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscrape-do%2Fscrapedo-scrapers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrape-do%2Fscrapedo-scrapers/lists"}