Projects in Awesome Lists tagged with python-web-scraper

https://github.com/d60/twikit

bot client python python-web-scraper python3 scrape scraper scraping search tweepy twitter twitter-api twitter-bot twitter-client twitter-internal-api twitter-scraper wrapper x x-api

Last synced: 15 May 2025

https://github.com/omkarcloud/botasaurus

The All in One Framework to Build Undefeatable Scrapers

anti-bot anti-detect anti-detect-browser anti-detection antidetect-browser bot-detection bypass-cloudflare cloudflare-bypass cloudflare-scrape python-scraper python-web-scraper python-web-scraping scraping-framework scraping-python scraping-tool undetectable undetected undetected-chromedriver web-crawling web-scraping-python

Last synced: 13 May 2025

https://github.com/je-suis-tm/web-scraping

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

bloomberg data-scraper data-scraping financial-data financial-times futures futures-historical-data news-scraper news-websites newsletter options-data python-web-scraper reuters scrapper sraping wall-street-journal wallstreetbets web-scraper web-scrapers web-scraping

Last synced: 04 Apr 2025

https://github.com/davidteather/everything-web-scraping

Learn everything web scraping with David Teather Codes on YouTube

course courses everything hacktoberfest hacktoerfest project-based-learning project-based-learning-courses project-based-tutorials python python-web-scraper python3 reverse-engineering web-scraping web-scraping-python web-scraping-tutorial webscraping youtube-series

Last synced: 20 Apr 2025

https://github.com/oxylabs/python-web-scraping-tutorial

In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move on to relatively more complex.

amazon-scraper-python crawler github-python json-database-python python python-projects python-web-crawler python-web-scraper scraper-python scraping web-crawler-python web-scraping web-scraping-api web-scraping-python webscraping

Last synced: 16 May 2025

https://github.com/oxylabs/aliexpress-scraper

Easily collect public AliExpress data like prices, product details, and ratings with AliExpress Scraper API

ali-scraper alibaba-api aliexpress aliexpress-api aliexpress-crawler aliexpress-scraper aliexpress-spider e-commerce-api e-commerce-scraper email-crawler github-python python-web-scraper scraper-api

Last synced: 16 May 2025

https://github.com/oxylabs/scrape-google-python

In this tutorial, we showcase how to scrape public Google data with Python and Oxylabs API.

google python python-scraping python-web-scraper scrape scrape-google scraper-api scraping scraping-api web-scraping

Last synced: 13 Jun 2025

https://github.com/Conso1eCowb0y/Deepminer

Deep web crawler and search engine

crawler crawling dark-web data-mining deepminer deepweb github hacking onion osint python-web-scraper python3 search-engine security security-tools spider the-onion-router tor tor-network webcrawler

Last synced: 20 Apr 2025

https://github.com/thewebscraping/tls-requests

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

anti-bot anti-bot-detection anti-bot-page cf-clearance cloudflare-bypass cloudflare-scraper crawling-python python-crawler python-scraper python-spider python-tls-client python-web-crawler python-web-scraper python-web-scraping scraping-python tls-client web-crawler-python web-scraping-api web-scraping-python web-spider

Last synced: 09 Apr 2025

https://github.com/oxylabs/ebay-scraper

eBay Scraper for extracting product titles, descriptions, prices, specifications, customer reviews, and ratings from eBay

ebay ebay-api ebay-apis ebay-scraper ebay-search ebay-searches python-web-scraper scrape-ebay url-scraper

Last synced: 19 Dec 2024

https://github.com/Decodo/Python-scraper-tutorial

A short introduction to scraping with Python with given steps and an example scraper script.

beautifulsoup crawler data-mining data-science github-python json-database-python learning python python-projects python-web-crawler python-web-scraper scraper-python scraping web-crawler-python web-scraping web-scraping-api web-scraping-python webscraping

Last synced: 02 May 2025

https://github.com/stabldev/mangareader-api

A Python based web scraping api built with fastapi to get manga contents.

anime fastapi manga manga-api mangareader python python-web-scraper scraping web-scraping

Last synced: 17 Jun 2025

https://github.com/Smartproxy/Python-scraper-tutorial

A short introduction to scraping with Python with given steps and an example scraper script.

beautifulsoup crawler data-mining data-science github-python json-database-python learning python python-projects python-web-crawler python-web-scraper scraper-python scraping web-crawler-python web-scraping web-scraping-api web-scraping-python webscraping

Last synced: 20 Nov 2024

https://github.com/moonlitgrace/mangareader-api

A Python based web scraping api built with fastapi that provides easy access to manga contents

anime fastapi manga manga-api mangareader python python-web-scraper scraping web-scraping

Last synced: 21 Mar 2025

https://github.com/nirantak/scraper

Python web scrapers

beautifulsoup playwright python python-web-scraper scraper scraping selenium

Last synced: 01 May 2025

https://github.com/julzerinos/python-scraping-tools

A collection of repeatable methods and concepts appearing in python web scraping with the use of Scrapy and Selenium

bot python python-bot python-web-scraper scraping scrapy selenium selenium-python web-scraping

Last synced: 20 Apr 2025

https://github.com/oxylabs/web-scraping-google-sheets

Guide to Using Google Sheets for Basic Web Scraping

email-scraper google-news-api google-news-scraper google-search-scraper google-sheets-web-scraping google-trends-api python-web-scraper web-crawler-python web-scraping web-scraping-google-sheets web-scraping-python

Last synced: 11 Mar 2025

https://github.com/gayanukabulegoda/web-scraping-starter-kit

Repository designed to help freshers easily grasp the basics of web scripting, offering simple guides and examples to build a strong foundation.

python python-web-scraper python3 scrap-data scrape scraping scraping-data scraping-images scraping-python scraping-web simple-scraping web-scraper web-scraping web-scraping-project web-scraping-python web-scraping-tutorials web-scrapper-python web-scrapping

Last synced: 18 Jan 2025

https://github.com/oxylabs/beautiful-soup-parsing-tutorial

Using Python & Beautiful Soup to Parse Data

beautiful-soup-parsing beautifulsoup beautifulsoup-html-parser github-python how-to-use-beautiful-soup-in-python parser python python-beautiful-soup python-web-scraper

Last synced: 14 Apr 2025

https://github.com/nazir20/python-google-image-scrapper

Python Google Image Scraper using Selenium

google-image-scraper python-google-image-scraper python-web-scraper selenium-google-image-scraper selenium-web-scraper selenium-webdriver

Last synced: 22 Mar 2025

https://github.com/oxylabs/how-to-build-web-scraper

Step by step guide to building a web scraper with Python

build-a-web-scraper github-python parser python python-web-scraper python-web-scraping python3 url-scraper web-scraping

Last synced: 11 Mar 2025

https://github.com/maxinexiong/music-events-web-scraper

A web scraping program built to extract attribute data on music events from a popular event ticketing website. The program collects event data based on the specified country and city, which is then exported to a CSV file for marketing emails and stored efficiently in a local PostgreSQL database.

beautifulsoup4 csv-export data-migration email-sender pandas-dataframe postgresql postgresql-database psycopg2 python python-web-scraper python-web-scraping requests-python smtplib web-scraping