Projects in Awesome Lists tagged with python-scraper

https://github.com/seleniumbase/seleniumbase

Python APIs for web automation, testing, and bypassing bot-detection.

anti-detection behave bot-detection cdp chromedriver cloudflare-bypass e2e-testing pytest pytest-plugin python python-scraper selenium selenium-python seleniumbase test-automation web-automation web-scraping web-scraping-python webdriver webkit

Last synced: 13 May 2025

https://github.com/seleniumbase/SeleniumBase

Python APIs for web automation, testing, and bypassing bot-detection.

anti-detection behave bot-detection cdp chromedriver cloudflare-bypass e2e-testing pytest pytest-plugin python python-scraper selenium selenium-python seleniumbase test-automation web-automation web-scraping web-scraping-python webdriver webkit

Last synced: 26 Mar 2025

https://github.com/omkarcloud/botasaurus

The All in One Framework to Build Undefeatable Scrapers

anti-bot anti-detect anti-detect-browser anti-detection antidetect-browser bot-detection bypass-cloudflare cloudflare-bypass cloudflare-scrape python-scraper python-web-scraper python-web-scraping scraping-framework scraping-python scraping-tool undetectable undetected undetected-chromedriver web-crawling web-scraping-python

Last synced: 13 May 2025

https://github.com/chris-greening/instascrape

Powerful and flexible Instagram scraping library for Python, providing easy-to-use and expressive tools for accessing data programmatically

beginner-friendly data-mining data-science instagram instagram-data instagram-scraper lightweight python python-scraper python3 webscraping

Last synced: 07 Apr 2025

https://github.com/oxylabs/how-to-scrape-google-scholar

A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.

google-scholar google-scholar-scraper google-scholar-scrapper google-search-scraper python python-scraper scraper-api web-scraper web-scraping

Last synced: 15 May 2025

https://github.com/oxylabs/how-to-scrape-amazon-prices

A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.

amazon amazon-scraper api python python-scraper scraper-api web-scraper web-scraping

Last synced: 16 May 2025

https://github.com/scrapfly/scrapfly-scrapers

Scalable Python web scraping scripts for +40 popular domains

antibot automation captcha-bypass crawler crawling crawling-python datascraping proxies python python-scraper scraper scraping scraping-python spider twitter-scraper web-crawler web-scraping web-scraping-python webscraper webscraping

Last synced: 11 Apr 2025

https://github.com/thespeedx/whatscraper

A Whatsapp Group Link Scraper And Checker

hacktoberfest python-scraper scraper speedx whatsapp whatsapp-bot

Last synced: 09 Apr 2025

https://github.com/thewebscraping/tls-requests

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

anti-bot anti-bot-detection anti-bot-page cf-clearance cloudflare-bypass cloudflare-scraper crawling-python python-crawler python-scraper python-spider python-tls-client python-web-crawler python-web-scraper python-web-scraping scraping-python tls-client web-crawler-python web-scraping-api web-scraping-python web-spider

Last synced: 09 Apr 2025

https://github.com/harismuneer/android-apps-downloader

📱 A utility for downloading Android apps from the Google Play Store and Xiaomi App Store (the Chinese App Store).

android-application-downloader android-apps-crawler android-market-scraper android-research android-scraper android-tool app-downloader crawler crawling-tool google-play-application-downloader google-play-store-scraper gplaycli open-source-project python-scraper research-tool scraper scraping-tool wget-utility xiaomi-apps xiaomi-store-scraper

Last synced: 30 Apr 2025

https://github.com/superkogito/coinmarketcapscraper

a small python scraper to scrape historical data from the CoinMarketCap website and convert it to csv files . This is an initial step for a data mining process to develop a predictive model of cryptocurrencies prices.

big-data coinmarketcap coinmarketcap-website cryptocurrencies-prices cryptocurrency csv csv-files python-scraper scraped-data scraper

Last synced: 04 Apr 2025

https://github.com/hhhrrrttt222111/selenium_python

beatifulsoup beautifulsoup-library beautifulsoup4 chromedriver geckodriver hacktoberfest pycharm-ide python python-parser python-requests python-scraper requests-html scraping-python scraping-websites selenium selenium-python selenium-webdriver spiders webscraper webscraping

Last synced: 08 Feb 2025

https://github.com/christivn/mapscraper

🤖🗺️ Headless browser scraper written in python to extract Places data from Google Maps.

google-map-api google-maps google-maps-api google-maps-scraper google-maps-scraping google-places google-scraper lead lead-generation prospecting prospects python python-scraper serapi

Last synced: 05 May 2025

https://github.com/shaheenfazim/url-scraper

A python script to scrape URL's from major search engines.

bing-scraper duckduckgo-scraper google-scraper python-scraper scraper web-scraper yahoo-scraper

Last synced: 02 Feb 2025

https://github.com/mzubairtahir/google-maps-scraper

Scraper that scrapes data from search results of google maps

business-data data data-scraping google-map-api google-map-scraper google-maps lead-generator python-scraper python-scraping python3 scraper scraping-websites

Last synced: 05 Apr 2025

https://github.com/theadnan/hadith-of-the-day

Firefox WebExtension that displays a different hadith on click

beautifulsoup4 firefox firefox-addon firefox-extension firefox-webextension hadith python-scraper python3 scraper webextension

Last synced: 01 Dec 2024

https://github.com/ximaz/instagram-api

This project is an Instagram API guest-oriented, meaning you can perform actions of data collection without having to register. However, you may be facing some Ratelimits, so I recommend you to put some delay between operations.

api instagram instagram-api instagram-bot instagram-client instagram-downloader instagram-feed instagram-scraper python python-scraper python3 scraper scraping scraping-websites

Last synced: 19 Apr 2025

https://github.com/atia-farha/html-fetcher-script

A Python script that allows users to fetch and optionally save the HTML content from a specified URL using `requests` library.

fetch-data fetcher fetching fetching-data fetching-data-from-server fetching-data-from-web html-fetcher html-fetcher-script html-scraper html-scraping python python-scraper python-script python3 pythonscript script web-scraper web-scraper-python web-scraping web-scraping-python

Last synced: 05 Apr 2025

https://github.com/oxylabs/scraping-real-estate-data-with-python

A guide for extracting public data from Redfin using Python and Oxylabs Web Scraper API. From using the API to processing data and saving the result.

e-commerce-scraper ecommerce-api ecommerce-scraper python python-scraper real-estate scraper-api web-scraping

Last synced: 14 Apr 2025

https://github.com/rounakdatta/xcheck

Fake news detection

detector django fake hackathon machine-learning news python-scraper

Last synced: 19 Jun 2025

https://github.com/farukalamai/yelp-scraper-scrapy-python

Yelp Restaurant data scraping using python, scrapy spider

ai-bot data-extraction data-mining data-scraper data-scraping python python-scraper scrapy scrapy-crawler scrapy-spider web-scraper web-scraping web-scraping-python web-scraping-software yelp yelp-api yelp-restaurants yelp-resturant-data-scraping yelp-scraper

Last synced: 29 Dec 2024

https://github.com/abhinavrajesh/amazon-scraper

Built as part of an assignment for S5 OOSE Subject CSE

python-scraper python-script python3

Last synced: 22 Mar 2025

https://github.com/franckferman/censys-api-python

A Python utility for effective extraction and analysis of internet data through Censys.

censys censys-api censys-client censys-search cti cyber-threat-intelligence data-visualization python python-3 python-api python-scraper python-scraping python-script python-security python-security-tool python3 python3-script requests scraper scraping

Last synced: 19 Apr 2025

https://github.com/sabujxi/python-scraper-and-data-analysts-admin-panel-in-django

A data scraper from texas govt site and a helping web app for managing, reviewing and editing the data

analyst data data-analysis data-entry data-scraper django django-application python python-scraper real-estate regex scraper texas

Last synced: 05 Apr 2025

https://github.com/raedaddala/scraping-imdb

This Python script extracts comprehensive movie data from IMDB, focusing on top-grossing movies from 1960 to 2024. The scraper collects detailed information including box office performance, cast & crew, awards, and other key metrics.

beautifulsoup crawling data-mining dataset imdb imdb-dataset imdb-webscrapping movies python python-scraper scraping selenium webscraping

Last synced: 03 Jan 2025

https://github.com/pinedah/instagram-photo-downloader

This project is a Python-based scraper that allows you to download all photos from a public Instagram profile. The script automates the process of retrieving images , making it easy to back up or analyze content from any accessible Instagram account. The code help itself with a basic graphic interface.

photo-scrapper photos python python-scraper python-script webscraping webscrapping-python

Last synced: 11 Apr 2025

https://github.com/aa-nadim/google-search-sponsored-list

python python-scraper scraping-websites

Last synced: 11 Apr 2025

https://github.com/rozhakxd/pdfinder

flask pdf pdf-api pdf-finder pdf-scraper pdf-scraping pdf-search pdf-search-engine pdf-viewer python python-scraper

Last synced: 22 Mar 2025

https://github.com/ap-atul/youtube-scrapper

a simple youtube scrapper uses youtube_dl

python-scraper scraper youtube-scraper

Last synced: 04 Apr 2025

https://github.com/pinedah/instragram-followers-scraper

Not Finished. - Python script that scrapes the list of followers and following from a given Instagram profile and exports the lists to either a PDF or TXT file, sorted alphabetically.

automated instagram-scraper python python-scraper scraper web-scraping

Last synced: 03 Mar 2025

https://github.com/franckferman/scraping-deputes-france

Script pour scraper les député·e·s français (Nom, Région, Email, Groupe, Circonscription) depuis le site de l'Assemblée nationale.

deputes francais france francophone python python-3 python-scraper python-scraping python-script python3 scraper scraping scraping-websites scrapper scrapping

Last synced: 09 Mar 2025

https://github.com/jacyuan1/fuel-price-tracker-in-the-greater-toronto-area

A Python script designed to monitor and track price changes within the Greater Toronto Area (GTA).

gas-prices gas-tracker python python-scraper python-script selenium selenium-webdriver toronto toronto-gas-price

Last synced: 18 Jun 2025

https://github.com/sreejoy/crawlerfriend

A light weight crawler which gives search results in HTML form or in Dictionary form, given URLs and keywords.

crawler python-crawler python-scraper python27 scrapper

Last synced: 12 Jun 2025

https://github.com/euronymou5/address-scrapper

Un simple web scrapping de direcciones, útil para el uso de bins, métodos de compras, etc.

address-generator addresses python-scraper python-scraping scraper scraper-python scraping scraping-websites scrapping

Last synced: 12 Mar 2025

https://github.com/std-enigma/typeracer-scraper

Scraper for https://play.typeracer.com texts database

python python-scraper python-script typeracer web-scraper

Last synced: 26 Mar 2025

https://github.com/tufayellus/leadersleague.com-scraper

This is a Python based web scraper that scrapes a list of URLs from LeadersLeague.com website that stores email, first name, last name and other leads from the website automatically

data-miner data-mining data-mining-python python-bot python-scraper python-scraper-project-for-beginners python-scraping python-script web-scraper web-scraper-python web-scrapers web-scraping