Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with scraping-python

A curated list of projects in awesome lists tagged with scraping-python .

https://github.com/outscraper/outscraper-python

The library provides convenient access to the Outscraper API from applications written in the Python language. Allows using Outscraper's services from your code.

api google google-api google-maps google-maps-api google-maps-scraping google-rating-api google-reviews-api google-scraping google-search-api google-serp-api scraper scraping scraping-python scraping-websites serp-api

Last synced: 12 Apr 2025

https://github.com/fcoagz/pydolarvenezuela

Esta librería desarrollada en Python te brinda la capacidad de consultar los precios del dólar y/o euro en diversos monitores en Venezuela, también como las tasas de cambio del BCV.

bcv dolar dolar-cotizacion dolar-oficial dollar library pydolarvenezuela python scraping-python scraping-websites venezuela

Last synced: 05 Apr 2025

https://github.com/xtekky/tikdown

Fast TikTok NO Watermark Video Downloader (username or url)

python requests scraping scraping-python tiktok tiktok-api tiktok-downloader tiktok-scraper tiktokbot

Last synced: 22 Mar 2025

https://github.com/kevinam99/instagram-like-bot

A simple Instagram like bot that likes posts based on the hashtags you provide.

instagram-bot kivy python python3 scraping-python selenium-python

Last synced: 12 Apr 2025

https://github.com/outscraper/outscraper-php

The library provides convenient access to the Outscraper API from applications written in the PHP language. Allows using Outscraper's services from your code.

google-api google-maps-api google-maps-scraping google-rating-api google-reviews-api google-scraping google-serp-api php scraper scraping scraping-python scraping-websites serp-api

Last synced: 17 Mar 2025

https://github.com/easonlai/facebook_post_scraping_and_text_analytics

This is demo repo to demostrate how to scrape post data from Facebook by Python with library facebook_scraper. And then use Azure Text Analytics to perform sentiment analysis for post text content.

azure azure-text-analysis azure-text-analytics data-scraping facebook facebook-post facebook-scraper facebook-scraping key-phrase-extraction microsoft-azure microsoft-cognitive-services pandas python python3 scraping-python seaborn text-analytics wordcloud

Last synced: 26 Apr 2025

https://github.com/arthurverrez/linkedin-scraping-tools

A set of simple tools leveraging Selenium for scraping LinkedIn through LinkedIn Sales Navigator or LinkedIn Recruiter.

linkedin sales scraping-python selenium-python

Last synced: 22 Jun 2025

https://github.com/gurjeet29/telegram_members_scrapper

Python Script to scrape members from a selected Telegram group.

api csv excel extract python scraping scraping-python telegram telegram-bot

Last synced: 22 Jul 2025

https://github.com/sawyerclick/scrapers

Scrapin' some data, man

actions beautifulsoup python scraping-python

Last synced: 14 Aug 2025

https://github.com/regulad/freebooter

freebooter downloads photos & videos from the internet and uploads it onto your social media accounts.

content-theft freebooting oauth2 oauth2-authorization-code-flow python scraping-python youtube youtube-api youtube-api-v3 youtube-dl youtube-upload youtube-uploader

Last synced: 23 Jan 2025

https://github.com/tortuc/python-scrapying-toexcel

Python Scraping Script made using python/ Scrapy and Selenium to extract data and export data as an Excel. main features are Auto-Search and extract in multi-section, Auto pagination clicking. Possibility of the store to databases including MySQL, SQLite, MongoDB

excel excel-export export-to-excel python python-scrapying-toexcel scraping-data scraping-python scraping-websites scrapy selenium web-scraping

Last synced: 15 Apr 2025

https://github.com/umihico/minigun-requests

Web scraping API to outsource tons of GET & xpath to cloud computing

crawler crawling scraping scraping-api scraping-framework scraping-python web-scraping

Last synced: 13 Apr 2025

https://github.com/samrb-dev/autoseekout

A simple web scraping bot for scraping information from seekout.com written in Python and Selenium

bot dataextraction python3 scraping-python seekout selenium selenium-python webscraping

Last synced: 18 Sep 2025

https://github.com/wyndace/wynscrape

Wyndace's python scraping dev kit

parser parsing python scraping scraping-python

Last synced: 13 May 2025

https://github.com/saadmairaj/oauthbrowser

This module will let you authenticate OAuth 2.0 via the system browser and get the response URL.

applescript authentication google oauth2 osascript scraping-python url webscraping

Last synced: 12 Apr 2025

https://github.com/ohn0/youtube-livechat-scraper

grab youtube live chat data from existing VODs

chat livestreaming python python3 scraper scraping-python streaming youtube

Last synced: 24 Jan 2025

https://github.com/christopherkindl/web-scraper-template

Web scraping template to scrape typical search-based platforms

beautifulsoup4 python scraping-python scraping-websites

Last synced: 18 Jun 2025

https://github.com/edi-id/image-downloader

download the first image from google image search

scraping-python scraping-websites youtube-dl yt-downloader

Last synced: 01 Apr 2025

https://github.com/wazzabeee/scraping_tool

A python application providing a GUI to scrape tweets from Twitter using Tweepy library and scripts allowing cron jobs.

api cron gui python scraping-python tkinter tweepy tweets twitter

Last synced: 02 Mar 2025

https://github.com/javiser/combined-shop-scraper

A simple, configurable and expandable combined shop scraper to minimize the costs of ordering several items

online-shop scraper scraping-python shopping

Last synced: 06 Mar 2025

https://github.com/alanjumeaucourt/tca-net

Discord bot that send notification 10 minutes before classes as a remind of the classroom and which professor

crawling-python discord-bot python scraping-python

Last synced: 17 Aug 2025

https://github.com/iamsad5566/py_scraping

Twitter scraping by Python with Selenium, GUI manipulating built by Go Fyne.io

fyne golang python scraper scraping-python selenium-python twitter twitter-bot

Last synced: 26 Feb 2025

https://github.com/jeffersonsimplicio/scraping-favorite-anime

A aplicação Favorite Anime Scraping, faz a raspagem do HTML de sites de animes e criar uma lista com o nome de todos os seus animes favoritos

beautifulsoup4 design-pattern executable pep8 pyinstaller pytest python python3 scraping scraping-python strategy test-automation

Last synced: 14 Mar 2025

https://github.com/vgvr0/el-corte-ingles-supermarket-scraper

A Python script for web scraping various product categories from an online supermarket (elcorteingles.es) and saving product details into a CSV file

beautifulsoup chromedriver el-corte-ingles el-corte-ingles-scraping requests scraping-python selenium selenium-base selenium-python supermarket-scraping

Last synced: 21 Mar 2025

https://github.com/tanzeelgcuf/idealo-product-scraping-

This code is develop to scrape data from idealo.de website and compare the top products with good prices.

bs4 flask-application python scraping-python

Last synced: 11 Jun 2025

https://github.com/code-alchemist01/intellifist-ai

An artificial intelligence model that randomly generates questions according to class, course and subject for the educational application we will make.

api beautifulsoup fastapi gemini python random-generation scraping-python scraping-websites token tokenization

Last synced: 11 Apr 2025

https://github.com/virajmadhu/mailminer

MailMiner is a robust Python tool designed for efficiently extracting email addresses from websites.

email-scraper free-for-u python-script python3 scraper scraping-python scraping-websites scrapper-script

Last synced: 12 Apr 2025

https://github.com/harmindersinghnijjar/netsu-dagneels-scraping-solutions

Sample Python code to scrape https://www.complex.management/legal/documents/STSMA-S:1

playwright python reddit scraper scraping scraping-python

Last synced: 09 Apr 2025

https://github.com/1337w0rm/lecturenotesdownloader

A Python Script to download study notes from LectureNotes in PDF or Image (jpeg) format

downloader python python3 scraping-python

Last synced: 16 Mar 2025

https://github.com/marcogdepinto/amazon-jobs-scraper

Creating a CSV file with the jobs available at a particular URL of the Amazon.jobs website.

amazon beatiful csv jobs-scraper python python3 scraping scraping-python selenium

Last synced: 23 Apr 2025

https://github.com/ximaz/lmdoit

Let Me DOwnload IT is a project which lets you schemafy your download process. You'll be able to use different method for authentication, such as username:password, cookies, and so on, to use an HTML Parser, regex or just get JSON server response to download resources, and so on...

api api-client api-gateway api-rest download download-manager downloader http-client python requests scraper scraping scraping-python scraping-websites scrapper scrapping scrapy

Last synced: 30 Mar 2025

https://github.com/mostafahima/speed-test-twitter-bot

A project that uses Selenium to test internet speed and automatically posts the results on Twitter.

scraping-python scraping-tool selenium selenium-python

Last synced: 07 Apr 2025

https://github.com/ryosukedtomita/twitter-api

仕様変更後に対応してません。

scraping-python twitter-api

Last synced: 27 Sep 2025

https://github.com/mostafahima/saco-automation-extractor

A project automating the extraction of product data from Saco website using Selenium, collecting links, prices, and availability, and organizing them into Excel files.

scraping-python

Last synced: 19 Feb 2025

https://github.com/mostafahima/insta-data-scraper

An automated tool for scraping Instagram account data, collecting post URLs, comments, and organizing the data into Excel files for easy analysis.

scraping-python

Last synced: 19 Feb 2025

https://github.com/mostafahima/bachelor-salary-extractor

A project to scrape data about top-paying jobs for bachelor’s degree holders using Python tools, organizing data from PayScale source.

scraping-python

Last synced: 29 Jul 2025

https://github.com/tezcatlipoca0000/trevi-spider

A tailored program that scrapes data from the website and updates my DB

gmail-api pandas python3 scraping-python selenium-python

Last synced: 09 Apr 2025

https://github.com/cqllum/namecheap-domain-auction-scrape

Gathers a list of expiring domains on NameCheap auction, and gathers details based on if domains are potentially rare.

csv domain-scraper namecheap namecheap-domains nltk pandas python python-requests requests scraping-python

Last synced: 02 Apr 2025

https://github.com/alimghmi/starbyfaceapi

https://starbyface.com Scraper. ETHICAL CAUTION: DON'T STRESS THEIR SERVERS

scraping scraping-python scraping-websites starbyface

Last synced: 01 Mar 2025

https://github.com/estebandem/itjobsscrape

Web scraping of a job portal to get the complete description of each job and save it in a .csv

beautifulsoup4 scraping-python scraping-websites

Last synced: 20 Feb 2025

https://github.com/nightmachinery/ddg2json

Parses scraped HTML of DuckDuckGo pages into JSON

ddg ddgr duckduckgo parsing scraping scraping-python search search-engine searching

Last synced: 20 Feb 2025

https://github.com/pnguyen215/instagram-crawler

Instagram Crawler is a Python script to download posts from a specified Instagram account.

crawler crawling-python instagram instagram-crawler scraper scraping-python scraping-websites scrapper scrapy-crawler

Last synced: 01 Mar 2025

https://github.com/mostafahima/nba-stats-scraper

A project that extracts comprehensive NBA game, player, and team data, storing it in CSV and JSON formats for analysis.

scraping-python

Last synced: 07 Sep 2025

https://github.com/souvic/mtweepy

Fastest scraping using multiple apps and user tokens for Twitter API!

scraping scraping-python scraping-web tweepy twitter twitter-api

Last synced: 12 Mar 2025

https://github.com/romerorodriguezd/housing-market-tracker

Jupyter Notebook to store and evaluate long-run evolution of house sales, based on Idealista website.

data-visualization matplotlib pandas python scraping scraping-python sqlite3

Last synced: 21 Feb 2025

https://github.com/mostafahima/law-firm-data-extractor

A project to collect detailed law firm data from "Law.com" using Selenium, storing it in a structured, easily analyzable format.

scraping-python

Last synced: 17 Jul 2025

https://github.com/odyash/mp3quran_scraper_and_tageditor

Scrape Quran Recitations Then Edit Their Metadata With Scraped Translations & Tafsir

metadata-editor quran quran-audio quran-info quran-translation scraping-python

Last synced: 19 Jun 2025

https://github.com/rp-bot/ultimate-midi-scraper

Scrape All MIDI files from the internet using Selenium

midi scraping-python web

Last synced: 24 Feb 2025

https://github.com/wkrzywiec/tasty-python

Command Line Tool for finding recipes on Tasty.co (written in Python)

beautifulsoup4 cli pipenv pytest python python-click python3 recipes scraping-python scraping-websites web-scraper

Last synced: 04 Apr 2025

https://github.com/exizt/blog-category-data-scraping

네이버, 티스토리 블로그에서 특정 하나의 카테고리의 게시글을 스크래핑하여 저장합니다.

naver-blog python scraping-python tistory tistory-backup

Last synced: 26 Jun 2025

https://github.com/arkaniightt/scraper_vagas_bot

Este repositório contém um script para varrer um site específico em busca de vagas de emprego. O script coleta e organiza informações sobre oportunidades disponíveis neste site, facilitando a busca por emprego para os candidatos interessados. Os usuários podem personalizar a busca de acordo com suas preferências e qualificações específicas.

scraper scrapers scraping-python scraping-websites scrapy selenium selenium-python selenium-webdriver spiders

Last synced: 18 Feb 2025

https://github.com/akanshdivker/gitscrape

Flexible and efficient Python program to query for GitHub repositories, scrape relevant data, and then clone them for later use.

collector github github-api python python3 research-tool scrape scraper scraping scraping-python scraping-websites tool

Last synced: 07 Apr 2025

https://github.com/oanderoficial/scraping

Web Scraping - Utilizando a biblioteca beautifulsoup4

python scraping scraping-python scraping-websites web webscraping

Last synced: 26 Mar 2025

https://github.com/cops3956/lecture-downloader

# Lecture DownloaderA Python toolkit for downloading and managing lecture videos from platforms like Canvas and Brightspace. 📚🚀 This tool helps you easily merge, transcribe, and embed subtitles for your educational needs.

bash browser-extension chrome-extension downloader firefox-extension fit javascript mit-license python scraping-python shell video

Last synced: 18 Jun 2025

https://github.com/sharph/phl-party

An interface to Philly City Council that is as bad as legistar, but with more hyperlinks

beautifulsoup4 django philadelphia scraping scraping-python

Last synced: 12 Mar 2025

https://github.com/dms-codes/scrape_dosen_fh_ui

Python Web Scraping for Law Faculty Staff Information This Python script is designed to scrape information about staff members from the Law Faculty of the University of Indonesia's website. It collects data such as names, titles, fields of study, biographies, education, courses taught, books

python scraper scraping scraping-python scraping-websites

Last synced: 14 Jul 2025

https://github.com/tadghh/python-weather-app

Scrapes all weather data from Enviroment Canada in under 30s*

canada scraping-python weather

Last synced: 19 Jul 2025

https://github.com/vishal815/flight_details_scraper_and_visualization

This project focuses on scraping flight details from Google Flights, processing the data, and performing cleaning and visualization for future use in analytics or predictive modeling.

beautifulsoup end-to-end-project flight-data flight-data-analysis google-flights google-flights-scraper pandas pyhon scraping-project scraping-python selenium vishal-lazrus vishallazrus visualization webscraping webscraping-data

Last synced: 25 Mar 2025