{"id":37229442,"url":"https://github.com/rezmoss/cloudip-db","last_synced_at":"2026-06-14T03:02:26.871Z","repository":{"id":331949281,"uuid":"1132324185","full_name":"rezmoss/cloudip-db","owner":"rezmoss","description":"Compact, daily-updated MessagePack database of cloud-provider IP ranges (AWS, GCP, Azure, Cloudflare, DigitalOcean, Oracle). ~743 KB gzipped, SHA-256 verified, built from cloud-provider-ip-addresses. Powers go-cloudip and js-cloudip","archived":false,"fork":false,"pushed_at":"2026-05-30T01:21:45.000Z","size":166195,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-05-30T03:21:45.347Z","etag":null,"topics":["aws","azure","cidr","cloud-ip-ranges","cloud-providers","cloudflare","database","digitalocean","gcp","ip-database","ip-detection","ip-lookup","ip-ranges","messagepack","oracle-cloud"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rezmoss.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-11T18:41:52.000Z","updated_at":"2026-05-30T01:21:47.000Z","dependencies_parsed_at":"2026-04-08T03:00:53.926Z","dependency_job_id":null,"html_url":"https://github.com/rezmoss/cloudip-db","commit_stats":null,"previous_names":["rezmoss/cloudip-db"],"tags_count":143,"template":false,"template_full_name":null,"purl":"pkg:github/rezmoss/cloudip-db","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rezmoss%2Fcloudip-db","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rezmoss%2Fcloudip-db/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rezmoss%2Fcloudip-db/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rezmoss%2Fcloudip-db/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rezmoss","download_url":"https://codeload.github.com/rezmoss/cloudip-db/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rezmoss%2Fcloudip-db/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33803771,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-02T02:00:07.132Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","azure","cidr","cloud-ip-ranges","cloud-providers","cloudflare","database","digitalocean","gcp","ip-database","ip-detection","ip-lookup","ip-ranges","messagepack","oracle-cloud"],"created_at":"2026-01-15T03:32:45.583Z","updated_at":"2026-06-02T03:05:11.603Z","avatar_url":"https://github.com/rezmoss.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cloudip-db\n\n[![Build CloudIP Data](https://github.com/rezmoss/cloudip-db/actions/workflows/build.yml/badge.svg)](https://github.com/rezmoss/cloudip-db/actions/workflows/build.yml)\n[![Latest Release](https://img.shields.io/github/v/release/rezmoss/cloudip-db?label=data%20version)](https://github.com/rezmoss/cloudip-db/releases/latest)\n[![Last Updated](https://img.shields.io/github/last-commit/rezmoss/cloudip-db?label=updated)](https://github.com/rezmoss/cloudip-db/commits/main)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n**A tiny, daily-updated, MessagePack-encoded database of cloud-provider IP ranges — purpose-built for fast client libraries.**\n\n`cloudip-db` is the compiled, compressed binary form of [rezmoss/cloud-provider-ip-addresses](https://github.com/rezmoss/cloud-provider-ip-addresses). It packs every CIDR block from the major cloud providers into a single ~**743 KB gzipped** [MessagePack](https://msgpack.org/) file that client libraries — like [go-cloudip](https://github.com/rezmoss/go-cloudip) and [js-cloudip](https://github.com/rezmoss/js-cloudip) — can download once and use offline for sub-microsecond IP-to-provider lookups.\n\n\u003e Source data (rich JSON, per provider) → **cloudip-db** (compact binary) → client libraries (Go, JS, …)\n\n## What's inside\n\n| File | Description |\n|------|-------------|\n| [`data/cloudip.msgpack`](data/cloudip.msgpack) | Uncompressed MessagePack database (~6.5 MB) |\n| [`data/cloudip.msgpack.gz`](data/cloudip.msgpack.gz) | Gzipped MessagePack database (~743 KB) — recommended for downloads |\n| [`data/version.json`](data/version.json) | Version metadata: build date, range count, SHA-256, sizes |\n\n### Current stats\n\n\u003e See [`data/version.json`](data/version.json) for the live numbers — typically **120,000+ CIDR ranges** across **6 providers**, rebuilt every day.\n\n### Providers covered\n\n- **AWS** (Amazon Web Services)\n- **GCP** (Google Cloud Platform)\n- **Azure** (Microsoft)\n- **Cloudflare**\n- **DigitalOcean**\n- **Oracle Cloud**\n\nNeed more providers (Fastly, GitHub, Linode, bot crawlers, …)? The full firehose with 22+ providers and 12+ output formats lives in the upstream [cloud-provider-ip-addresses](https://github.com/rezmoss/cloud-provider-ip-addresses) repo.\n\n## Why a separate repo?\n\nThe upstream [cloud-provider-ip-addresses](https://github.com/rezmoss/cloud-provider-ip-addresses) repo is optimized for **humans and ops tooling**: per-provider JSON/CSV/SQL, ready-to-paste Nginx/Apache/iptables/nftables/UFW/HAProxy/Caddy configs, and a daily changelog.\n\n`cloudip-db` is optimized for **client libraries**:\n\n- **One file, one fetch.** No per-provider HTTP requests, no format negotiation.\n- **Compact binary format.** MessagePack with short field names (`p`, `r`, `s`) keeps the gzipped payload under 1 MB.\n- **Verifiable.** Every build publishes a SHA-256 in `version.json` so clients can detect corruption or tampering.\n- **Versioned releases.** Each daily build is tagged `vYYYY-MM-DD` with a GitHub Release — pin to a version or always grab `latest`.\n- **CDN-friendly.** Raw GitHub URLs are immutable per-commit, perfect for caching.\n\n## Download\n\nLatest data (always-current, served from `main`):\n\n```\nhttps://github.com/rezmoss/cloudip-db/raw/main/data/cloudip.msgpack.gz\nhttps://github.com/rezmoss/cloudip-db/raw/main/data/version.json\n```\n\nPinned release (immutable):\n\n```\nhttps://github.com/rezmoss/cloudip-db/releases/download/v2026-05-12/cloudip.msgpack.gz\n```\n\n## Schema\n\nThe MessagePack file decodes to:\n\n```jsonc\n{\n  \"version\":     \"2026-05-12\",          // ISO date of the build\n  \"build_time\":  1778548431,            // Unix timestamp\n  \"providers\":   [\"aws\", \"gcp\", \"cloudflare\", \"azure\", \"digitalocean\", \"oracle\"],\n  \"ranges\": [\n    {\n      \"cidr\": \"52.94.76.0/22\",          // IPv4 or IPv6 CIDR\n      \"p\":    0,                        // Provider index into `providers`\n      \"r\":    \"us-east-1\",              // Region (optional)\n      \"s\":    \"EC2\"                     // Service (optional)\n    },\n    // …\n  ]\n}\n```\n\nField names are intentionally short (`p`, `r`, `s`) to minimize on-wire size.\n\n`version.json` has its own small schema:\n\n```json\n{\n  \"version\":   \"2026-05-12\",\n  \"build_time\": 1778548431,\n  \"sha256\":    \"b712e66bb16d3da4e9347a67829444343471fedef23882f3e5c182a5e1a92e19\",\n  \"ranges\":    121392,\n  \"size\":      6552383,\n  \"size_gzip\": 743009\n}\n```\n\nClients typically:\n1. Fetch `version.json` to check `version` + `sha256`.\n2. Download `cloudip.msgpack.gz` only if changed.\n3. Verify SHA-256, decompress, parse, build a trie.\n\n## Use it from your code\n\n### Go — [`go-cloudip`](https://github.com/rezmoss/go-cloudip)\n\n```bash\ngo get github.com/rezmoss/go-cloudip\n```\n\n```go\nimport \"github.com/rezmoss/go-cloudip\"\n\nif cloudip.IsAWS(\"52.94.76.1\") {\n    // …\n}\n```\n\n### JavaScript / TypeScript — [`js-cloudip`](https://github.com/rezmoss/js-cloudip)\n\n```bash\nnpm install js-cloudip\n```\n\n```ts\nimport { isAws, getProvider, lookup } from 'js-cloudip';\n\nawait isAws('52.94.76.1');        // true\nawait getProvider('34.64.0.1');   // \"gcp\"\n```\n\nBoth libraries embed a fallback copy of `cloudip-db` for offline / air-gapped use and refresh from this repo in the background.\n\n### Roll your own client\n\nAny MessagePack library will do. Example in Python:\n\n```python\nimport gzip, urllib.request, msgpack\n\nurl = \"https://github.com/rezmoss/cloudip-db/raw/main/data/cloudip.msgpack.gz\"\nwith urllib.request.urlopen(url) as r:\n    db = msgpack.unpackb(gzip.decompress(r.read()), raw=False)\n\nprint(db[\"version\"], len(db[\"ranges\"]), \"ranges\")\n```\n\n## How the build works\n\n[`scripts/build.go`](scripts/build.go) is a small Go program that:\n\n1. Fetches the per-provider JSON files from [cloud-provider-ip-addresses](https://github.com/rezmoss/cloud-provider-ip-addresses).\n2. Normalizes them into a single `Range` schema (CIDR + provider index + optional region/service).\n3. Sorts ranges deterministically (so daily diffs stay clean).\n4. Encodes to MessagePack, gzips, and writes `version.json` with SHA-256 + sizes.\n\n[`.github/workflows/build.yml`](.github/workflows/build.yml) runs this every day at **00:30 UTC** (30 minutes after upstream updates), commits any changes, tags `vYYYY-MM-DD`, and publishes a GitHub Release with the three data files attached.\n\nTo rebuild locally:\n\n```bash\ncd scripts\ngo mod download\ngo run build.go ../data\n```\n\n## Update cadence\n\n- **Daily** at 00:30 UTC via GitHub Actions.\n- A new tag + release is created only when the data actually changes.\n- Clients should poll `version.json` (a few hundred bytes) to decide whether to re-download the full blob.\n\n## Versioning\n\nTags follow `vYYYY-MM-DD` (the build date). There is no SemVer here — the *schema* is stable, and the *data* changes every day. If the schema ever changes in a breaking way, the change will be announced in a release note and a `schema_version` field will be added.\n\n## Contributing\n\nThis repo only contains the build script and generated artifacts. **Open issues and PRs against the upstream data repo:** [rezmoss/cloud-provider-ip-addresses](https://github.com/rezmoss/cloud-provider-ip-addresses).\n\nBug reports about the build script, schema, or client integration are welcome here.\n\n## Related projects\n\n- [**cloud-provider-ip-addresses**](https://github.com/rezmoss/cloud-provider-ip-addresses) — upstream daily data, 22+ providers, 12+ formats (JSON/CSV/SQL/Nginx/iptables/…)\n- [**go-cloudip**](https://github.com/rezmoss/go-cloudip) — Go client, sub-microsecond lookups via Patricia trie\n- [**js-cloudip**](https://github.com/rezmoss/js-cloudip) — JS/TS client for Node and the browser\n\n## License\n\nMIT — see [LICENSE](LICENSE). Upstream provider data is published under each provider's own terms; this repo only redistributes publicly available IP-range information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frezmoss%2Fcloudip-db","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frezmoss%2Fcloudip-db","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frezmoss%2Fcloudip-db/lists"}