Projects in Awesome Lists tagged with git-scraping

https://github.com/jstrieb/github-stats

Better GitHub statistics images for your profile, with stats from private repos too

async asyncio git-scraping github github-actions github-api github-stats profile python python3 readme-md readme-template statistics statistics-images stats-images visualizations

Last synced: 14 May 2025

https://github.com/alex/nyt-2020-election-scraper

git-scraping

Last synced: 15 May 2025

https://github.com/factbook/factbook.json

World Factbook Country Profiles in JSON - Free Open Public Domain Data - No API Key Required ;-)

africa america asia countries economy europe factbook git-scraping government json oceania opendata people publicdomain religion world

Last synced: 15 May 2025

https://github.com/mackorone/spotify-playlist-archive

Daily snapshots of public Spotify playlists

archive git-scraping hacktoberfest history playlist snapshot spotify spotify-playlists versions

Last synced: 12 Feb 2026

https://github.com/simonw/csv-diff

Python CLI tool and library for diffing CSV and JSON files

click csv csv-diff datasette-io datasette-tool diff git-scraping tsv-diff

Last synced: 16 May 2025

https://github.com/femueller/cloud-ip-ranges

An up-to-date export of cloud provider IP address ranges

akamai aws azure cloud-ip-ranges cloudflare digitalocean gcloud git-scraping github-ipaddress ip iprange ipranges linode microsoft-azure oracle oracle-cloud

Last synced: 16 May 2025

https://github.com/vinayak-mehta/conrad

Track conferences and meetups on your terminal.

git-scraping

Last synced: 15 May 2025

https://github.com/swyxio/gh-action-data-scraping

this shows how to use github actions to do periodic data scraping

cron gh-actions git-scraping

Last synced: 14 Oct 2025

https://github.com/simonw/ca-fires-history

Tracking fire data from www.fire.ca.gov

disasters fires git-scraping

Last synced: 08 Apr 2025

https://github.com/endoflife-date/release-data

Common Release Data for various projects in a consumable format, automatically updated.

git-scraping hacktoberfest

Last synced: 21 Jun 2025

https://github.com/mary-ext/atproto-scraping

Git scraping of AT Protocol/Bluesky instances

atcute atproto bluesky git-scraping

Last synced: 06 Apr 2025

https://github.com/tobilg/public-cloud-provider-ip-ranges

Unified datasets for public cloud provider IP ranges. Providers include AWS, Azure, CloudFlare, DigitalOcean, Fastly, Google Cloud and Oracle Cloud.

aws azure cloud gcp git-scraping ipv4

Last synced: 14 Oct 2025

https://github.com/vitorbaptista/google-covid19-mobility-reports

Data extraction of Google's COVID-19 Mobility Reports

covid-19 dataset git-scraping scraping

Last synced: 30 Jan 2026

https://github.com/mary-ext/bluesky-labeler-scraping

Git scraping of Bluesky labelers/label providers

atcute bluesky git-scraping

Last synced: 17 Mar 2025

https://github.com/simonw/scrape-hacker-news-by-domain

Scrape HN to track links from specific domains

git-scraping

Last synced: 05 May 2025

https://github.com/datadesk/california-coronavirus-scrapers

The open-source web scrapers that feed the Los Angeles Times California coronavirus tracker.

california coronavirus covid-19 data-journalism git-scraping journalism jupyter-notebook news python scraper

Last synced: 09 Apr 2025

https://github.com/simonw/pge-outages-pre-2024

Tracking PG&E outages

git-scraping pge-outages power scraping

Last synced: 21 Sep 2025

https://github.com/pl4nty/intune-change-tracking

Track changes to Microsoft Intune with git and RSS

git-scraping

Last synced: 16 Jan 2026

https://github.com/tobilg/aws-iam-data

This repository contains the full dataset of AWS IAM data (services, actions, resource types and conditions keys). It's updated on a daily basis at 4AM UTC.

aws data git-scraping iam

Last synced: 07 Apr 2025

https://github.com/simonw/disaster-scrapers

Scrapers for disaster data - writes to https://github.com/simonw/disaster-data

git-scraping

Last synced: 19 Apr 2025

https://github.com/captn3m0/india-isin-data

International Securities Identification Numbers for various Indian Securities

csv dataset funds git-scraping india isin nsdl securities

Last synced: 02 Apr 2026

https://github.com/simonw/help-scraper

Record a history of --help for various commands

git-scraping

Last synced: 08 Mar 2026

https://github.com/simonw/sf-tree-history

Tracking the history of trees in San Francisco

circleci git-scraping san-francisco trees

Last synced: 14 Apr 2025

https://github.com/captn3m0/historical-mf-data

Historical Mutual Funds data

git-scraping indian-finance mutual-funds open-datasets pricing-information sqlite-dataset

Last synced: 02 Jun 2026

https://github.com/simonw/disaster-data

Data scraped by https://github.com/simonw/disaster-scrapers

data-scraping git-scraping irma-response json

Last synced: 19 Apr 2025

https://github.com/simonw/scrape-open-data

Scrape various open data directories to create an index of what's available out there

git-scraping socrata

Last synced: 16 Apr 2025

https://github.com/ifoukarakis/jobscrapper

An automated job scrapper

git-scraping scrapy

Last synced: 03 Apr 2025

https://github.com/fedora-python/portingdb

Database & tools to track Python 2 removal from Fedora

fedora git-scraping python2-python3

Last synced: 07 Apr 2025

https://github.com/pkmn/smogon

Wrapper around Smogon's analyses and usage statistics

data git-scraping pokemon smogon

Last synced: 09 Apr 2025

https://github.com/pkmn/randbats

Pokémon Showdown's Random Battle sets

data git-scraping pokemon pokemon-showdown

Last synced: 29 Jul 2025

https://github.com/mikepqr/real-estate-scrape-eg

A repository demonstrating the use of real-estate-scrape to store the estimated value of a property on Redfin and Zillow every night using Github Actions.

git-scraping

Last synced: 30 Dec 2025

https://github.com/bobek/masscan_as_a_service

masscan as a service

audit bare-metal cloud containers git-scraping masscan phabricator security security-scanner security-tools sre

Last synced: 25 Jan 2026

https://github.com/sarojbelbase/nepstonks

An automated bot that scrapes the latest upcoming issues, news, and investment opportunities that are announced inside Nepal and sends them to a telegram channel.

bot debenture fpo git-scraping hacktoberfest investment-opportunities ipo made-in-nepal meroshare mutual-funds nepal nepali-app nepse news right-share share-market sharesansar stock-market telegram-bot telegram-channel

Last synced: 15 Apr 2025

https://github.com/iandees/usps-collection-boxes

US Postal Service collection box locations.

git-scraping usps

Last synced: 13 Apr 2025

https://github.com/ahmedshahriar/depression-tweets-scraper

A Scraper that scrapes '#depression' tweets daily powered by GitHub action and snscrape (stopped at June 30,2023)

automation dataset depression git-automation git-scraper git-scraping github-action snscrape social-media twitter twitter-scraper web-scraping

Last synced: 22 Apr 2025

https://github.com/rdmurphy/actblue-ticker-tracker

Keeps tabs on the ticking donation amount found on ActBlue's home page.

git-scraping

Last synced: 24 Jul 2025

https://github.com/simonw/graphql-scraper

Track changes to GraphQL APIs by git scraping their schemas

git-scraping graphql

Last synced: 19 Apr 2025

https://github.com/maxhalford/bike-sharing-history

🚲 Git scraping for bike sharing APIs

bike-sharing git-scraping

Last synced: 04 Jul 2025

https://github.com/knudmoeller/berlin_corona_cases

Scraper for the official dashboard with current Corona case numbers, traffic light indicators ("Corona-Ampel") and vaccination situation for Berlin.

berlin corona covid19 dataset git-scraping nokogiri opendata ruby scraper vaccination

Last synced: 20 Jan 2026

https://github.com/beatrizmilz/mananciais

Base de dados sobre volume operacional em mananciais de abastecimento público na Região Metropolitana de São Paulo (SP - Brasil).

git-scraping

Last synced: 18 Jul 2025

https://github.com/simonw/scrape-fediverse

Git scrapers for scraping the fediverse

git-scraping

Last synced: 01 Feb 2026

https://github.com/punchagan/playo-find-venue

Find good Playo venues in convenient locations

bangalore git-scraping google-maps playo sports

Last synced: 07 May 2025

https://github.com/mary-ext/bluesky-verifier-scraping

Git scraping of Bluesky trusted verifiers

atcute bluesky git-scraping

Last synced: 07 Apr 2026

https://github.com/dbreunig/git-scraper-extractor

Pull out versions of specific files from a gitscraping repo into individual files.

git-scraping

Last synced: 17 Jan 2026

https://github.com/simonw/fara-history

Tracking the history of the FARA data from https://www.justice.gov/nsd-fara

csv datasette git-scraping

Last synced: 19 Apr 2025

https://github.com/simonw/conditional-get

CLI tool for fetching data using HTTP conditional get

git-scraping http

Last synced: 05 Oct 2025

https://github.com/pl4nty/web-admx-tool

Windows group policy editor in your browser, preloaded with popular ADMX files

git-scraping

Last synced: 16 Jan 2026

https://github.com/openclimatedata/paris-agreement-entry-into-force

Data Package of ratification status of the Paris Climate Agreement and the emissions shares used for entry into force

data-package git-scraping

Last synced: 19 Feb 2026

https://github.com/captn3m0/mf.captnemo.in

Get information about Indian Mutual Funds from their ISIN numbers.

git-scraping mutual-funds public-api public-apis

Last synced: 23 Mar 2025

https://github.com/beatrizmilz/noticiasgov

Raspagem de dados de portais de noticias governamentais

git-scraping r rstats web-scraping

Last synced: 30 Jul 2025

https://github.com/simonw/pge-outages

Tracking PG&E power outages

git-scraping

Last synced: 14 Apr 2025

https://github.com/hueyy/lacuna-db

legal data in machine-readable form

dataset datasette git-scraping law lawtech legal legaltech open-data singapore

Last synced: 05 Mar 2026

https://github.com/tobilg/aws-iam-managed-policies

Automatically populated repository of AWS IAM Managed Policies

aws git-scraping iam managed-policies policy

Last synced: 02 Jul 2025

https://github.com/palewire/noaa-hurricane-gis-scraper

Automated downloads of geographic information system data posted by the National Oceanic and Atmospheric Administration's National Hurricane Center and Central Pacific Hurricane Center

data-journalism gis git-scraping hurricanes journalism news nhc noaa python rss scraper weather

Last synced: 19 Apr 2025

https://github.com/simonw/irma-scrapers

Screen scrapers relating to natural disasters. See their output in https://github.com/simonw/disaster-data/

civic-hacking git-scraping irma-response scraper slack

Last synced: 19 Apr 2025

https://github.com/danp/nspoweroutages

Git scraping of the Nova Scotia Power Outage Map

git-scraping

Last synced: 14 Mar 2026

https://github.com/mrflynn/mcbroken-archive

:inbox_tray: Archive for data from mcbroken.com.

data-archive dataset git-scraping mcbroken mcbroken-archive

Last synced: 02 Mar 2025

https://github.com/matchilling/hmrc-exchange-rates

🇬🇧 HMRC Exchange Rates API for Customs & VAT 💸

api exchange-rates git-scraping hmrc united-kingdom vat

Last synced: 06 Oct 2025

https://github.com/outages/vultr-outages

Track Vultr outages via Git History

git-history git-scrape git-scraping outage outages scrape scraping status vultr vultr-outages vultr-status

Last synced: 18 Jan 2026

https://github.com/openclimatedata/ndcs

Data Package with Nationally Determined Contributions (NDCs)

data-package git-scraping

Last synced: 19 Feb 2026

https://github.com/richardsondev/pse-outages

Tracking Puget Sound Energy outage history since March 2021

git-history git-scrape git-scraper git-scraping outages power-outage power-outages puget-sound-data pugetsound scrape scraping washington-state

Last synced: 06 Jan 2026

https://github.com/captn3m0/india-mutual-fund-ter-tracker

Tracking Total Expense Ratios of Indian Mutual Funds. Automatically updated daily.

amfi-data git-scraping indian-mutual-funds open-data

Last synced: 23 Mar 2025

https://github.com/schwanksta/irs-bmf-changelog

Creates a changelog for the IRS' exempt org business master file

git-scraping irs nonprofits

Last synced: 07 Jan 2026

https://github.com/maliayas/sublimetext_documentation

Daily unofficial mirror of the ST documentation

git-scraping sublime-text

Last synced: 16 Mar 2026

https://github.com/radames/google-fonts-analytics-archive

Archiving Google Fonts analytics data for fun https://fonts.google.com/analytics

archive data-visualization git-scraping google-fonts

Last synced: 01 Apr 2025

https://github.com/blr-today/ingest

Ingestion pipeline for blr.today

bangalore blr-today events git-scraping

Last synced: 30 Apr 2025

https://github.com/sgraaf/openapi-scraper

Track changes to RESTful APIs by git scraping their OpenAPI descriptions

git-scraping openapi rest-api

Last synced: 24 Mar 2025

https://github.com/rdmurphy/tx-covid-vaccine-data

Tracking data on the progress of vaccine distribution and adminstration in Texas.

git-scraping

Last synced: 12 Apr 2025

https://github.com/fasiha/finviz-git-scraper

FinViz map of sectors and sub-sectors

git-scraping stock-data

Last synced: 26 Oct 2025

https://github.com/raylas/sbc-reservoirs-history

Logging reservoir level data from https://rain.cosbpw.net

climate drought git-scraping reservoirs water

Last synced: 12 Jan 2026

https://github.com/Joel-hanson/Iceberg-locations

Current Antarctic large iceberg positions derived from ASCAT and OSCAT-2

beautifulsoup4 climate-change git-scraping iceberg python scraping

Last synced: 20 Jul 2025

https://github.com/joel-hanson/iceberg-locations

Current Antarctic large iceberg positions derived from ASCAT and OSCAT-2

beautifulsoup4 climate-change git-scraping iceberg python scraping

Last synced: 06 Jun 2026

https://github.com/thejeshgn/karnataka-eletricity-generation

Karnataka State Electricity Generation and Load data.

datameet electricity git-scraping open-data open-data-india

Last synced: 09 Feb 2026

https://github.com/ohbarye/git-scraping-template

A template of a git scraping

git git-scraping rss-feed ruby

Last synced: 09 May 2026

https://github.com/ahmedshahriar/burnout-tweets-scraper

A Scraper that scrapes '#burnout' tweets daily powered by GitHub action and snscrape (stopped at June 30,2023)

automation burnout dataset git-automation git-scraper git-scraping github-action snscrape social-media twitter twitter-scraper web-scraping

Last synced: 02 Aug 2025

https://github.com/iris-hep/analysis-community-summary

Summary report on community interactions and contributions with IRIS-HEP Analysis Systems related tools

git-scraping iris-hep

Last synced: 11 Apr 2025

https://github.com/OSUKED/ETS-Watch

Python client for retrieving the latest data on the EU ETS market and its participants

eu-ets-market git-scraping

Last synced: 07 May 2025

https://github.com/openclimatedata/kigali-amendment-entry-into-force

Data Package of entry into force status of the Kigali Amendment to the Montreal Protocol

data-package git-scraping

Last synced: 19 Feb 2026

https://github.com/palewire/nyc-open-data-monitor

Automated monitoring of new and updated datasets posted to New York City's data portal

git-scraper git-scraping nyc-opendata python

Last synced: 08 Mar 2026

https://github.com/simonw/scrape-github-actions-package-versions

Git scraper recording the package versions installed on the defaul GitHub Actions ubuntu-latest worker

git-scraping

Last synced: 14 Apr 2025

https://github.com/jlumbroso/basic-git-scraper-template

🔬 Starter template for automating web scrapers using GitHub Actions workflows to incrementally commit data to Git 📈 Includes sample script, scheduling, dependency installation, output to CSV/JSON, and ethics guide 🤖 Customizable for diverse sites and use cases!

git-scraping github-template template web-scraping

Last synced: 12 Oct 2025

https://github.com/mauforonda/precios

git-scraping

Last synced: 16 Jan 2026

https://github.com/simonw/sce-outages

Tracking SCE outages

git-scraping

Last synced: 07 Jan 2026

https://github.com/honzajavorek/czech-political-parties

Tracking changes in Czech political parties

czech czech-republic czechia git-scraping parties political-parties registry scraper scrapy

Last synced: 12 May 2025

https://github.com/ohbarye/finance-app-ranking

Git scraping for finance app ranking

git git-scraping

Last synced: 11 Jun 2025

https://github.com/captn3m0/electron-fingerprints

Generates fingerprints for electron version detection by downloading electron releases and generating checksums of the files contained in each release.

git-scraping

Last synced: 16 Apr 2026