Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with scraping-python

A curated list of projects in awesome lists tagged with scraping-python .

https://github.com/lobstrio/shadow-useragent

Pick the most common user-agents on the Internet 👻

scraping scraping-python useragent useragent-switch

Last synced: 14 Jan 2026

https://github.com/outscraper/outscraper-python

The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

api google google-api google-maps google-maps-api google-maps-scraping google-rating-api google-reviews-api google-scraping google-search-api google-serp-api scraper scraping scraping-python scraping-websites serp-api

Last synced: 14 Dec 2025

https://github.com/fcoagz/pydolarvenezuela

Esta librería desarrollada en Python te brinda la capacidad de consultar los precios del dólar y/o euro en diversos monitores en Venezuela, también como las tasas de cambio del BCV.

bcv dolar dolar-cotizacion dolar-oficial dollar library pydolarvenezuela python scraping-python scraping-websites venezuela

Last synced: 05 Apr 2025

https://github.com/xtekky/tikdown

Fast TikTok NO Watermark Video Downloader (username or url)

python requests scraping scraping-python tiktok tiktok-api tiktok-downloader tiktok-scraper tiktokbot

Last synced: 22 Mar 2025

https://github.com/kevinam99/instagram-like-bot

A simple Instagram like bot that likes posts based on the hashtags you provide.

instagram-bot kivy python python3 scraping-python selenium-python

Last synced: 12 Apr 2025

https://github.com/outscraper/outscraper-php

The library provides convenient access to the Outscraper API from applications written in the PHP language. Allows using Outscraper's services from your code.

google-api google-maps-api google-maps-scraping google-rating-api google-reviews-api google-scraping google-serp-api php scraper scraping scraping-python scraping-websites serp-api

Last synced: 29 Jan 2026

https://github.com/easonlai/facebook_post_scraping_and_text_analytics

This is demo repo to demostrate how to scrape post data from Facebook by Python with library facebook_scraper. And then use Azure Text Analytics to perform sentiment analysis for post text content.

azure azure-text-analysis azure-text-analytics data-scraping facebook facebook-post facebook-scraper facebook-scraping key-phrase-extraction microsoft-azure microsoft-cognitive-services pandas python python3 scraping-python seaborn text-analytics wordcloud

Last synced: 26 Apr 2025

https://github.com/arthurverrez/linkedin-scraping-tools

A set of simple tools leveraging Selenium for scraping LinkedIn through LinkedIn Sales Navigator or LinkedIn Recruiter.

linkedin sales scraping-python selenium-python

Last synced: 22 Jun 2025

https://github.com/gurjeet29/telegram_members_scrapper

Python Script to scrape members from a selected Telegram group.

api csv excel extract python scraping scraping-python telegram telegram-bot

Last synced: 22 Jul 2025

https://github.com/sawyerclick/scrapers

Scrapin' some data, man

actions beautifulsoup python scraping-python

Last synced: 14 Aug 2025

https://github.com/regulad/freebooter

freebooter downloads photos & videos from the internet and uploads it onto your social media accounts.

content-theft freebooting oauth2 oauth2-authorization-code-flow python scraping-python youtube youtube-api youtube-api-v3 youtube-dl youtube-upload youtube-uploader

Last synced: 04 Oct 2025

https://github.com/tortuc/python-scrapying-toexcel

Python Scraping Script made using python/ Scrapy and Selenium to extract data and export data as an Excel. main features are Auto-Search and extract in multi-section, Auto pagination clicking. Possibility of the store to databases including MySQL, SQLite, MongoDB

excel excel-export export-to-excel python python-scrapying-toexcel scraping-data scraping-python scraping-websites scrapy selenium web-scraping

Last synced: 15 Apr 2025

https://github.com/umihico/minigun-requests

Web scraping API to outsource tons of GET & xpath to cloud computing

crawler crawling scraping scraping-api scraping-framework scraping-python web-scraping

Last synced: 13 Apr 2025

https://github.com/samrb-dev/autoseekout

A simple web scraping bot for scraping information from seekout.com written in Python and Selenium

bot dataextraction python3 scraping-python seekout selenium selenium-python webscraping

Last synced: 11 Oct 2025

https://github.com/saadmairaj/oauthbrowser

This module will let you authenticate OAuth 2.0 via the system browser and get the response URL.

applescript authentication google oauth2 osascript scraping-python url webscraping

Last synced: 12 Apr 2025

https://github.com/wyndace/wynscrape

Wyndace's python scraping dev kit

parser parsing python scraping scraping-python

Last synced: 13 May 2025

https://github.com/ohn0/youtube-livechat-scraper

grab youtube live chat data from existing VODs

chat livestreaming python python3 scraper scraping-python streaming youtube

Last synced: 06 Oct 2025

https://github.com/christopherkindl/web-scraper-template

Web scraping template to scrape typical search-based platforms

beautifulsoup4 python scraping-python scraping-websites

Last synced: 18 Jun 2025

https://github.com/brunobastosg/scraping-hidroweb

Scraping dos dados do Portal HidroWeb

python python3 scraping scraping-python

Last synced: 21 Feb 2026

https://github.com/wazzabeee/scraping_tool

A python application providing a GUI to scrape tweets from Twitter using Tweepy library and scripts allowing cron jobs.

api cron gui python scraping-python tkinter tweepy tweets twitter

Last synced: 02 Mar 2025

https://github.com/edi-id/image-downloader

download the first image from google image search

scraping-python scraping-websites youtube-dl yt-downloader

Last synced: 01 Apr 2025

https://github.com/jeffersonsimplicio/scraping-favorite-anime

A aplicação Favorite Anime Scraping, faz a raspagem do HTML de sites de animes e criar uma lista com o nome de todos os seus animes favoritos

beautifulsoup4 design-pattern executable pep8 pyinstaller pytest python python3 scraping scraping-python strategy test-automation

Last synced: 14 Mar 2025

https://github.com/vgvr0/el-corte-ingles-supermarket-scraper

A Python script for web scraping various product categories from an online supermarket (elcorteingles.es) and saving product details into a CSV file

beautifulsoup chromedriver el-corte-ingles el-corte-ingles-scraping requests scraping-python selenium selenium-base selenium-python supermarket-scraping

Last synced: 21 Mar 2025

https://github.com/tanzeelgcuf/idealo-product-scraping-

This code is develop to scrape data from idealo.de website and compare the top products with good prices.

bs4 flask-application python scraping-python

Last synced: 11 Jun 2025

https://github.com/alanjumeaucourt/tca-net

Discord bot that send notification 10 minutes before classes as a remind of the classroom and which professor

crawling-python discord-bot python scraping-python

Last synced: 17 Aug 2025

https://github.com/javiser/combined-shop-scraper

A simple, configurable and expandable combined shop scraper to minimize the costs of ordering several items

online-shop scraper scraping-python shopping

Last synced: 06 Mar 2025

https://github.com/iamsad5566/py_scraping

Twitter scraping by Python with Selenium, GUI manipulating built by Go Fyne.io

fyne golang python scraper scraping-python selenium-python twitter twitter-bot

Last synced: 26 Feb 2025

https://github.com/harmindersinghnijjar/netsu-dagneels-scraping-solutions

Sample Python code to scrape https://www.complex.management/legal/documents/STSMA-S:1

playwright python reddit scraper scraping scraping-python

Last synced: 09 Apr 2025

https://github.com/virajmadhu/mailminer

MailMiner is a robust Python tool designed for efficiently extracting email addresses from websites.

email-scraper free-for-u python-script python3 scraper scraping-python scraping-websites scrapper-script

Last synced: 15 Feb 2026

https://github.com/1337w0rm/lecturenotesdownloader

A Python Script to download study notes from LectureNotes in PDF or Image (jpeg) format

downloader python python3 scraping-python

Last synced: 16 Mar 2025

https://github.com/ryosukedtomita/twitter-api

仕様変更後に対応してません。

scraping-python twitter-api

Last synced: 27 Sep 2025

https://github.com/mostafahima/insta-data-scraper

An automated tool for scraping Instagram account data, collecting post URLs, comments, and organizing the data into Excel files for easy analysis.

scraping-python

Last synced: 16 Nov 2025

https://github.com/alimghmi/starbyfaceapi

https://starbyface.com Scraper. ETHICAL CAUTION: DON'T STRESS THEIR SERVERS

scraping scraping-python scraping-websites starbyface

Last synced: 01 Mar 2025

https://github.com/mostafahima/bachelor-salary-extractor

A project to scrape data about top-paying jobs for bachelor’s degree holders using Python tools, organizing data from PayScale source.

scraping-python

Last synced: 29 Jul 2025

https://github.com/pnguyen215/instagram-crawler

Instagram Crawler is a Python script to download posts from a specified Instagram account.

crawler crawling-python instagram instagram-crawler scraper scraping-python scraping-websites scrapper scrapy-crawler

Last synced: 01 Mar 2025

https://github.com/marcogdepinto/amazon-jobs-scraper

Creating a CSV file with the jobs available at a particular URL of the Amazon.jobs website.

amazon beatiful csv jobs-scraper python python3 scraping scraping-python selenium

Last synced: 23 Apr 2025

https://github.com/tezcatlipoca0000/trevi-spider

A tailored program that scrapes data from the website and updates my DB

gmail-api pandas python3 scraping-python selenium-python

Last synced: 09 Apr 2025

https://github.com/rp-bot/ultimate-midi-scraper

Scrape All MIDI files from the internet using Selenium

midi scraping-python web

Last synced: 28 Oct 2025

https://github.com/machinia/scraper-factory

Scraping library to retrieve data from useful pages, such as Amazon wishlists

amazon amazon-wishlist scrape scrapers scraping scraping-python scraping-websites scrapy scrapy-crawler scrapy-spider scrapy-spiders wishlist

Last synced: 08 Apr 2026

https://github.com/souvic/mtweepy

Fastest scraping using multiple apps and user tokens for Twitter API!

scraping scraping-python scraping-web tweepy twitter twitter-api

Last synced: 12 Mar 2025

https://github.com/ximaz/lmdoit

Let Me DOwnload IT is a project which lets you schemafy your download process. You'll be able to use different method for authentication, such as username:password, cookies, and so on, to use an HTML Parser, regex or just get JSON server response to download resources, and so on...

api api-client api-gateway api-rest download download-manager downloader http-client python requests scraper scraping scraping-python scraping-websites scrapper scrapping scrapy

Last synced: 30 Mar 2025

https://github.com/wkrzywiec/tasty-python

Command Line Tool for finding recipes on Tasty.co (written in Python)

beautifulsoup4 cli pipenv pytest python python-click python3 recipes scraping-python scraping-websites web-scraper

Last synced: 04 Apr 2025

https://github.com/cqllum/namecheap-domain-auction-scrape

Gathers a list of expiring domains on NameCheap auction, and gathers details based on if domains are potentially rare.

csv domain-scraper namecheap namecheap-domains nltk pandas python python-requests requests scraping-python

Last synced: 02 Apr 2025

https://github.com/mostafahima/saco-automation-extractor

A project automating the extraction of product data from Saco website using Selenium, collecting links, prices, and availability, and organizing them into Excel files.

scraping-python

Last synced: 14 Oct 2025

https://github.com/nightmachinery/ddg2json

Parses scraped HTML of DuckDuckGo pages into JSON

ddg ddgr duckduckgo parsing scraping scraping-python search search-engine searching

Last synced: 14 Mar 2026

https://github.com/mostafahima/law-firm-data-extractor

A project to collect detailed law firm data from "Law.com" using Selenium, storing it in a structured, easily analyzable format.

scraping-python

Last synced: 17 Jul 2025

https://github.com/mostafahima/speed-test-twitter-bot

A project that uses Selenium to test internet speed and automatically posts the results on Twitter.

scraping-python scraping-tool selenium selenium-python

Last synced: 07 Apr 2025

https://github.com/arkaniightt/scraper_vagas_bot

Este repositório contém um script para varrer um site específico em busca de vagas de emprego. O script coleta e organiza informações sobre oportunidades disponíveis neste site, facilitando a busca por emprego para os candidatos interessados. Os usuários podem personalizar a busca de acordo com suas preferências e qualificações específicas.

scraper scrapers scraping-python scraping-websites scrapy selenium selenium-python selenium-webdriver spiders

Last synced: 03 Nov 2025

https://github.com/odyash/mp3quran_scraper_and_tageditor

Scrape Quran Recitations Then Edit Their Metadata With Scraped Translations & Tafsir

metadata-editor quran quran-audio quran-info quran-translation scraping-python

Last synced: 19 Jun 2025

https://github.com/estebandem/itjobsscrape

Web scraping of a job portal to get the complete description of each job and save it in a .csv

beautifulsoup4 scraping-python scraping-websites

Last synced: 09 Mar 2026

https://github.com/mostafahima/nba-stats-scraper

A project that extracts comprehensive NBA game, player, and team data, storing it in CSV and JSON formats for analysis.

scraping-python

Last synced: 07 Sep 2025

https://github.com/romerorodriguezd/housing-market-tracker

Jupyter Notebook to store and evaluate long-run evolution of house sales, based on Idealista website.

data-visualization matplotlib pandas python scraping scraping-python sqlite3

Last synced: 29 Oct 2025

https://github.com/exizt/blog-category-data-scraping

네이버, 티스토리 블로그에서 특정 하나의 카테고리의 게시글을 스크래핑하여 저장합니다.

naver-blog python scraping-python tistory tistory-backup

Last synced: 26 Jun 2025

https://github.com/tadghh/python-weather-app

Scrapes all weather data from Enviroment Canada in under 30s*

canada scraping-python weather

Last synced: 19 Jul 2025

https://github.com/wsijp/scrapepath

Web scraping syntax

easy python scraping-python syntax templates

Last synced: 04 Apr 2026

https://github.com/vishal815/flight_details_scraper_and_visualization

This project focuses on scraping flight details from Google Flights, processing the data, and performing cleaning and visualization for future use in analytics or predictive modeling.

beautifulsoup end-to-end-project flight-data flight-data-analysis google-flights google-flights-scraper pandas pyhon scraping-project scraping-python selenium vishal-lazrus vishallazrus visualization webscraping webscraping-data

Last synced: 13 Apr 2026