Projects in Awesome Lists tagged with git-scraping

https://github.com/jstrieb/github-stats

Better GitHub statistics images for your profile, with stats from private repos too

async asyncio git-scraping github github-actions github-api github-stats profile python python3 readme-md readme-template statistics statistics-images stats-images visualizations

Last synced: 31 Dec 2024

https://github.com/alex/nyt-2020-election-scraper

git-scraping

Last synced: 03 Dec 2024

https://github.com/factbook/factbook.json

World Factbook Country Profiles in JSON - Free Open Public Domain Data - No API Key Required ;-)

africa america asia countries economy europe factbook git-scraping government json oceania opendata people publicdomain religion world

Last synced: 05 Jan 2025

https://github.com/mackorone/spotify-playlist-archive

Daily snapshots of public Spotify playlists

archive git-scraping hacktoberfest history playlist snapshot spotify spotify-playlists versions

Last synced: 20 Nov 2024

https://github.com/simonw/csv-diff

Python CLI tool and library for diffing CSV and JSON files

click csv csv-diff datasette-io datasette-tool diff git-scraping tsv-diff

Last synced: 03 Jan 2025

https://github.com/vinayak-mehta/conrad

Track conferences and meetups on your terminal.

git-scraping

Last synced: 04 Jan 2025

https://github.com/femueller/cloud-ip-ranges

An up-to-date export of cloud provider IP address ranges

akamai aws azure cloud-ip-ranges cloudflare digitalocean gcloud git-scraping github-ipaddress ip iprange ipranges linode microsoft-azure oracle oracle-cloud

Last synced: 03 Jan 2025

https://github.com/swyxio/gh-action-data-scraping

this shows how to use github actions to do periodic data scraping

cron gh-actions git-scraping

Last synced: 26 Nov 2024

https://github.com/sw-yx/gh-action-data-scraping

this shows how to use github actions to do periodic data scraping

cron gh-actions git-scraping

Last synced: 23 Nov 2024

https://github.com/simonw/ca-fires-history

Tracking fire data from www.fire.ca.gov

disasters fires git-scraping

Last synced: 07 Dec 2024

https://github.com/mary-ext/atproto-scraping

Git scraping of AT Protocol/Bluesky instances

atcute atproto bluesky git-scraping

Last synced: 01 Jan 2025

https://github.com/vitorbaptista/google-covid19-mobility-reports

Data extraction of Google's COVID-19 Mobility Reports

covid-19 dataset git-scraping scraping

Last synced: 27 Oct 2024

https://github.com/simonw/pge-outages-pre-2024

Tracking PG&E outages

git-scraping pge-outages power scraping

Last synced: 18 Oct 2024

https://github.com/tobilg/aws-iam-data

This repository contains the full dataset of AWS IAM data (services, actions, resource types and conditions keys). It's updated on a daily basis at 4AM UTC.

aws data git-scraping iam

Last synced: 07 Nov 2024

https://github.com/simonw/scrape-hacker-news-by-domain

Scrape HN to track links from specific domains

git-scraping

Last synced: 03 Jan 2025

https://github.com/simonw/disaster-scrapers

Scrapers for disaster data - writes to https://github.com/simonw/disaster-data

git-scraping

Last synced: 18 Oct 2024

https://github.com/simonw/help-scraper

Record a history of --help for various commands

git-scraping

Last synced: 27 Dec 2024

https://github.com/simonw/sf-tree-history

Tracking the history of trees in San Francisco

circleci git-scraping san-francisco trees

Last synced: 07 Dec 2024

https://github.com/simonw/disaster-data

Data scraped by https://github.com/simonw/disaster-scrapers

data-scraping git-scraping irma-response json

Last synced: 18 Oct 2024

https://github.com/ifoukarakis/jobscrapper

An automated job scrapper

git-scraping scrapy

Last synced: 04 Nov 2024

https://github.com/fedora-python/portingdb

Database & tools to track Python 2 removal from Fedora

fedora git-scraping python2-python3

Last synced: 02 Jan 2025

https://github.com/simonw/scrape-open-data

Scrape various open data directories to create an index of what's available out there

git-scraping socrata

Last synced: 16 Nov 2024

https://github.com/pkmn/randbats

Pokémon Showdown's Random Battle sets

data git-scraping pokemon pokemon-showdown

Last synced: 03 Dec 2024

https://github.com/pkmn/smogon

Wrapper around Smogon's analyses and usage statistics

data git-scraping pokemon smogon

Last synced: 01 Jan 2025

https://github.com/sarojbelbase/nepstonks

An automated bot that scrapes the latest upcoming issues, news, and investment opportunities that are announced inside Nepal and sends them to a telegram channel.

bot debenture fpo git-scraping hacktoberfest investment-opportunities ipo made-in-nepal meroshare mutual-funds nepal nepali-app nepse news right-share share-market sharesansar stock-market telegram-bot telegram-channel

Last synced: 01 Nov 2024

https://github.com/mikepqr/real-estate-scrape-eg

A repository demonstrating the use of real-estate-scrape to store the estimated value of a property on Redfin and Zillow every night using Github Actions.

git-scraping

Last synced: 27 Oct 2024

https://github.com/iandees/usps-collection-boxes

US Postal Service collection box locations.

git-scraping usps

Last synced: 07 Nov 2024

https://github.com/captn3m0/india-isin-data

International Securities Identification Numbers for various Indian Securities

csv dataset funds git-scraping india isin nsdl securities

Last synced: 28 Oct 2024

https://github.com/simonw/graphql-scraper

Track changes to GraphQL APIs by git scraping their schemas

git-scraping graphql

Last synced: 18 Oct 2024

https://github.com/rdmurphy/actblue-ticker-tracker

Keeps tabs on the ticking donation amount found on ActBlue's home page.

git-scraping

Last synced: 07 Nov 2024

https://github.com/ahmedshahriar/depression-tweets-scraper

A Scraper that scrapes '#depression' tweets daily powered by GitHub action and snscrape (stopped at June 30,2023)

automation dataset depression git-automation git-scraper git-scraping github-action snscrape social-media twitter twitter-scraper web-scraping

Last synced: 16 Nov 2024

https://github.com/beatrizmilz/mananciais

Base de dados sobre volume operacional em mananciais de abastecimento público na Região Metropolitana de São Paulo (SP - Brasil).

git-scraping

Last synced: 04 Dec 2024

https://github.com/knudmoeller/berlin_corona_cases

Scraper for the official dashboard with current Corona case numbers, traffic light indicators ("Corona-Ampel") and vaccination situation for Berlin.

berlin corona covid19 dataset git-scraping nokogiri opendata ruby scraper vaccination

Last synced: 26 Oct 2024

https://github.com/maxhalford/bike-sharing-history

🚲 Git scraping for bike sharing APIs

bike-sharing git-scraping

Last synced: 09 Nov 2024

https://github.com/mary-ext/bluesky-labeler-scraping

Git scraping of Bluesky labelers/label providers

atcute bluesky git-scraping

Last synced: 13 Nov 2024

https://github.com/simonw/conditional-get

CLI tool for fetching data using HTTP conditional get

git-scraping http

Last synced: 18 Oct 2024

https://github.com/simonw/fara-history

Tracking the history of the FARA data from https://www.justice.gov/nsd-fara

csv datasette git-scraping

Last synced: 18 Oct 2024

https://github.com/openclimatedata/paris-agreement-entry-into-force

Data Package of ratification status of the Paris Climate Agreement and the emissions shares used for entry into force

data-package git-scraping

Last synced: 03 Nov 2024

https://github.com/captn3m0/mf.captnemo.in

Get information about Indian Mutual Funds from their ISIN numbers.

git-scraping mutual-funds public-api public-apis

Last synced: 28 Oct 2024

https://github.com/dbreunig/git-scraper-extractor

Pull out versions of specific files from a gitscraping repo into individual files.

git-scraping

Last synced: 19 Nov 2024

https://github.com/beatrizmilz/noticiasgov

Raspagem de dados de portais de noticias governamentais

git-scraping r rstats web-scraping

Last synced: 04 Dec 2024

https://github.com/punchagan/playo-find-venue

Find good Playo venues in convenient locations

bangalore git-scraping google-maps playo sports

Last synced: 10 Dec 2024

https://github.com/simonw/irma-scrapers

Screen scrapers relating to natural disasters. See their output in https://github.com/simonw/disaster-data/

civic-hacking git-scraping irma-response scraper slack

Last synced: 18 Oct 2024

https://github.com/simonw/scrape-fediverse

Git scrapers for scraping the fediverse

git-scraping

Last synced: 07 Dec 2024

https://github.com/mrflynn/mcbroken-archive

:inbox_tray: Archive for data from mcbroken.com.

data-archive dataset git-scraping mcbroken mcbroken-archive

Last synced: 13 Nov 2024

https://github.com/captn3m0/india-mutual-fund-ter-tracker

Tracking Total Expense Ratios of Indian Mutual Funds. Automatically updated daily.

amfi-data git-scraping indian-mutual-funds open-data

Last synced: 28 Oct 2024

https://github.com/simonw/pge-outages

Tracking PG&E power outages

git-scraping

Last synced: 07 Dec 2024

https://github.com/schwanksta/irs-bmf-changelog

Creates a changelog for the IRS' exempt org business master file

git-scraping irs nonprofits

Last synced: 07 Dec 2024

https://github.com/blr-today/ingest

Ingestion pipeline for blr.today

bangalore blr-today events git-scraping

Last synced: 12 Nov 2024

https://github.com/rdmurphy/tx-covid-vaccine-data

Tracking data on the progress of vaccine distribution and adminstration in Texas.

git-scraping

Last synced: 07 Nov 2024

https://github.com/hueyy/lacuna-db

legal data in machine-readable form

dataset datasette git-scraping law lawtech legal legaltech open-data singapore

Last synced: 09 Nov 2024

https://github.com/sgraaf/openapi-scraper

Track changes to RESTful APIs by git scraping their OpenAPI descriptions

git-scraping openapi rest-api

Last synced: 29 Oct 2024

https://github.com/richardsondev/pse-outages

Tracking Puget Sound Energy outage history since March 2021

git-history git-scrape git-scraper git-scraping outages power-outage power-outages puget-sound-data pugetsound scrape scraping washington-state

Last synced: 07 Dec 2024

https://github.com/captn3m0/historical-mf-data

Historical Mutual Funds data

git-scraping indian-finance mutual-funds open-datasets pricing-information sqlite-dataset

Last synced: 28 Oct 2024

https://github.com/danp/nspoweroutages

Git scraping of the Nova Scotia Power Outage Map

git-scraping

Last synced: 06 Dec 2024

https://github.com/Joel-hanson/Iceberg-locations

Current Antarctic large iceberg positions derived from ASCAT and OSCAT-2

beautifulsoup4 climate-change git-scraping iceberg python scraping

Last synced: 27 Nov 2024

https://github.com/radames/google-fonts-analytics-archive

Archiving Google Fonts analytics data for fun https://fonts.google.com/analytics

archive data-visualization git-scraping google-fonts

Last synced: 14 Dec 2024

https://github.com/OSUKED/ETS-Watch

Python client for retrieving the latest data on the EU ETS market and its participants

eu-ets-market git-scraping

Last synced: 14 Nov 2024

https://github.com/ahmedshahriar/burnout-tweets-scraper

A Scraper that scrapes '#burnout' tweets daily powered by GitHub action and snscrape (stopped at June 30,2023)

automation burnout dataset git-automation git-scraper git-scraping github-action snscrape social-media twitter twitter-scraper web-scraping

Last synced: 16 Nov 2024

https://github.com/iris-hep/analysis-community-summary

Summary report on community interactions and contributions with IRIS-HEP Analysis Systems related tools

git-scraping iris-hep

Last synced: 07 Nov 2024

https://github.com/fasiha/finviz-git-scraper

FinViz map of sectors and sub-sectors

git-scraping stock-data

Last synced: 29 Oct 2024

https://github.com/brian14708/wh-briefings

The White House Briefing Room

git-scraping

Last synced: 22 Nov 2024

https://github.com/simonw/sce-outages

Tracking SCE outages

git-scraping

Last synced: 07 Dec 2024

https://github.com/honzajavorek/czech-political-parties

Tracking changes in Czech political parties

czech czech-republic czechia git-scraping parties political-parties registry scraper scrapy

Last synced: 18 Nov 2024

https://github.com/junosuarez/git-scraper-oregon-covid

Oregon Covid-19 data, scraped from Oregon Health Authority.

covid-data covid19-data data git-scraping

Last synced: 28 Dec 2024

https://github.com/ngshiheng/cafireshistorydb

Tracking fire data from www.fire.ca.gov, in SQLite

datasette disaster fires git-scraping

Last synced: 24 Oct 2024

https://github.com/palewire/nyc-open-data-monitor

Automated monitoring of new and updated datasets posted to New York City's data portal

git-scraper git-scraping nyc-opendata python

Last synced: 18 Oct 2024

https://github.com/nightmachinery/sharif_course_list

A git-scrape of SUT's course lists, in HTML and JSON

course-list data git-scrape git-scraping sharif sharif-university sut

Last synced: 31 Dec 2024

https://github.com/openclimatedata/doha-amendment-entry-into-force

Data Package of ratification status of the Doha Amendment to the Kyoto Protocol

data-package git-scraping

Last synced: 03 Nov 2024

https://github.com/beardicus/scrape-nws-alerts

Scraping weather alerts from the US National Weather Service's XML feed

git-scraping nodejs weather

Last synced: 06 Nov 2024

https://github.com/daniel-j-h/bundesrecht-scraper

Github Action based scraper to capture changes to Bundesrecht at gesetze-im-internet.de

bundesrecht gesetze-im-internet git-scraping

Last synced: 12 Oct 2024

https://github.com/dwillis/wvu-projects

Scripts and data from West Virginia University journalism class

git-scraping

Last synced: 15 Oct 2024

https://github.com/simonw/nhs-risky-venues

Archiving a history of NHS risky venue alerts

git-scraping

Last synced: 18 Oct 2024

https://github.com/jlumbroso/basic-git-scraper-template

🔬 Starter template for automating web scrapers using GitHub Actions workflows to incrementally commit data to Git 📈 Includes sample script, scheduling, dependency installation, output to CSV/JSON, and ethics guide 🤖 Customizable for diverse sites and use cases!

git-scraping github-template template web-scraping

Last synced: 02 Dec 2024

https://github.com/captn3m0/kuvera-mutual-funds-lookup

Simple dataset containing a mapping between Kuvera's Mutual Fund IDs and their ISINs.

git-scraping mutual-funds open-datasets

Last synced: 28 Oct 2024

https://github.com/metamoof/restricciones-covid

Archivo documental con los cambios de las recomendaciones y normativa de COVID-19 en España.

covid-19 git-scraping

Last synced: 03 Nov 2024

https://github.com/openclimatefix/metrics

Toolkit to automatically collect OCF metrics and store them over time.

git-scraping metrics-gathering

Last synced: 14 Nov 2024

https://github.com/simonw/scrape-github-actions-package-versions

Git scraper recording the package versions installed on the defaul GitHub Actions ubuntu-latest worker

git-scraping

Last synced: 07 Dec 2024

https://github.com/palewire/noaa-hurricane-gis-scraper

Automated downloads of geographic information system data posted by the National Oceanic and Atmospheric Administration's National Hurricane Center and Central Pacific Hurricane Center

data-journalism gis git-scraping hurricanes journalism news nhc noaa python rss scraper weather