{"id":49567426,"url":"https://github.com/internethealthreport/rpki-history","last_synced_at":"2026-05-03T12:04:24.661Z","repository":{"id":304653162,"uuid":"1019466986","full_name":"InternetHealthReport/rpki-history","owner":"InternetHealthReport","description":"An API to query historical RPKI data","archived":false,"fork":false,"pushed_at":"2025-08-26T08:23:20.000Z","size":79,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-26T10:31:04.600Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/InternetHealthReport.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-14T11:18:43.000Z","updated_at":"2025-08-26T08:23:23.000Z","dependencies_parsed_at":"2025-07-14T14:55:47.277Z","dependency_job_id":"d9094cff-e809-473d-9540-379e370e7842","html_url":"https://github.com/InternetHealthReport/rpki-history","commit_stats":null,"previous_names":["internethealthreport/rpki-history"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/InternetHealthReport/rpki-history","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternetHealthReport%2Frpki-history","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternetHealthReport%2Frpki-history/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternetHealthReport%2Frpki-history/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternetHealthReport%2Frpki-history/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/InternetHealthReport","download_url":"https://codeload.github.com/InternetHealthReport/rpki-history/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InternetHealthReport%2Frpki-history/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32568037,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-03T06:36:36.687Z","status":"ssl_error","status_checked_at":"2026-05-03T06:36:09.306Z","response_time":103,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-03T12:04:21.090Z","updated_at":"2026-05-03T12:04:24.656Z","avatar_url":"https://github.com/InternetHealthReport.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RPKI History\n\nAn API to query historical RPKI data, more specifically, Validated ROA Payloads (VRPs).\n\nLinks:\n\n* [GitHub](https://github.com/InternetHealthReport/rpki-history)\n* [API](https://www.ihr.live/rpki-history/api/)\n\nWe obtain VRP data from the [RPKI Views](https://www.rpkiviews.org/) project and store\nit in a compact form for fast access. From each dump we extract the\n`output/rpki-client.csv` file, which contains just the VRPs, and update our database.\n\nA VRP entry consists of five fields:\n\n```csv\nASN,IP Prefix,Max Length,Trust Anchor,Expires\nAS13335,1.0.0.0/24,24,apnic,1753280249\n```\n\nWe ignore the expiry time and only work on dump-time granularity. For each ingested dump\nwe compare the set of VRPs (in form of `asn, prefix, max_length`) with the previous\ndump. Each VRP has a `visible` time range, during which is was continuously visible.\nSince most VRPs are stable, this allows us to update just the time range, keeping the\ndatabase size compact.\n\nData limitations:\n\n* **High-frequency updates are invisible.** Currently, dumps are created roughly\nevery 20 minutes. If a VRP expires / is removed and potentially recreated between dumps,\nthis will not be reflected in the database.\n\n* **Data is at dump-time granularity.** If a VRP is created, expires, or is removed\nbetween dumps, the exact point in time is lost. For example, if there are two dumps at\ntime `A` and `C`, a VRP expires at time `B` (`A \u003c B \u003c C`), and a user queries for\ntimestamp `D` (`A \u003c D \u003c B`) then the query will not find the VRP, since its visibility\nended at `A`. Similarly, VRPs that are only visible in one dump will have equal values\nfor the start and end visibility time.\n\n* **Queryable time range is limited.** Naturally, we are limited by the amount of\navailable data. Trying queries outside of the available time range will result in an\nerror (to distinguish from non-existent VRPs).\n\nData sources:\n\n* 2020-12-06 16:37:23 UTC to 2022-06-14 14:54:59 UTC:\n  [josephine.sobornost.net](https://josephine.sobornost.net/rpkidata/)\n* 2022-06-14 15:08:09 UTC to now: [dango.attn.jp](https://dango.attn.jp/rpkidata/)\n\n## Endpoints\n\n### `/vrp`\n\nReturns the list of covering VRPs for a prefix at a specific time, time range, or at the\nlatest dump time if no time parameter is specified.\n\n#### Parameters\n\nMandatory:\n\n* `prefix`: The prefix for which to return covering VRPs.\n\nThere are two types of time parameters, which are mutually exclusive: Point-in-time and\ntime range. For a time range only one bound can be specified, in which case the returned\nlist will include *all* earlier/later data available.\n\nPoint-in-time:\n\n* `timestamp`: The timestamp (in `%Y-%m-%dT%H:%M:%S` [assumes UTC] or unix\nepoch format) for which to return VRPs.\n\nTime range:\n\n* `timestamp__gte`: The start of the time range (inclusive; in\n  `%Y-%m-%dT%H:%M:%S`[assumes UTC] or unix epoch format)\n  for which to return VRPs.\n* `timestamp__lte`: The end of the time range (inclusive; in `%Y-%m-%dT%H:%M:%S`\n  [assumes UTC] or unix epoch format) for which to return VRPs.\n\n#### Result Format\n\n```json\n// https://www.ihr.live/rpki-history/api/vrp?prefix=8.8.8.0/24\n[\n  {\n    \"prefix\": \"8.8.8.0/24\",\n    \"asn\": 15169,\n    \"max_length\": 24,\n    \"visible\": {\n      \"from\": \"2023-12-29T17:30:54+00:00\",\n      \"to\": \"2025-08-13T06:29:10+00:00\"\n    }\n  }\n]\n```\n\n`visible` refers to the timespan during which the VRP was *continuously* visible, i.e.,\npresent in the dumps. Thus, if a VRP is missing from a dump, a new entry with a separate\n`visible` range is created. **This time is unrelated to the validity time (`Not\nbefore`/`Not after`) of the ROA!**\n\n### `/status`\n\nReturns the RPKI status for the specified prefix/ASN combination at the specified time,\nor at the latest dump time if no timestamp is specified.\n\n#### Parameters\n\nMandatory:\n\n* `prefix`: The prefix to be checked.\n* `asn`: The origin ASN of the prefix.\n\nOptional:\n\n* `timestamp`: The timestamp (in `%Y-%m-%dT%H:%M:%S` [assumes UTC] or unix epoch format)\nfor which to check the status. If omitted, use the latest available dump time.\n\n#### Result Format\n\n```json\n// https://www.ihr.live/rpki-history/api/status?prefix=8.8.8.0/24\u0026asn=15169\n{\n  \"status\": \"Valid\"\n}\n\n// https://www.ihr.live/rpki-history/api/status?prefix=8.8.8.0/25\u0026asn=15169\n{\n  \"status\": \"Invalid\",\n  \"reason\": {\n    \"code\": \"moreSpecific\",\n    \"description\": \"Covering VRP with matching origin ASN found, but queried prefix is more specific than maxLength attribute allows.\"\n  }\n}\n```\n\n`status` is one of `[Valid, Invalid, NotFound]`.\n\n`reason` (only for `Invalid` status) gives more detailed information about why the\nstatus is invalid. `code` is for automatic processing, while `description` provides a\nhuman-readable explanation.\n\n### `/metadata`\n\nReturns the list of dumps contained in the database. Since this list is very long, this\nendpoint is paginated and returns at most 10000 results per page.\n\n#### Parameters\n\nMandatory: None\n\nOptional:\n\n* `timestamp__gte`: The start of the time range (inclusive; in\n  `%Y-%m-%dT%H:%M:%S`[assumes UTC] or unix epoch format)\n  for which to return data.\n* `timestamp__lte`: The end of the time range (inclusive; in `%Y-%m-%dT%H:%M:%S` [assumes UTC] or unix epoch format) for which to return data.\n* `page`: The page number to load (defaults to 1).\n* `page_size`: The number of results to include in one page (defaults to 10000).\n\n#### Result Format\n\n```json\n// https://www.ihr.live/rpki-history/api/metadata\n{\n  \"next\": \"https://www.ihr.live/rpki-history/api/metadata?page_size=1000\u0026page=2\",\n  \"results\": [\n    {\n      \"timestamp\": \"2020-12-06T16:37:23+00:00\",\n      \"deleted_vrps\": 0,\n      \"unchanged_vrps\": 0,\n      \"new_vrps\": 205850\n    },\n    // ...\n  ]\n}\n```\n\n`next` is the URL to the next page. It will be an empty string if there are no results\nleft.\n\n`timestamp` refers to the dump time (taken from the filename).\n\n`[deleted|updated|new]_vrps` indicates the number of VRPs differing from the previous\ndump. Note that `deleted` refers to a VRP that was present in the previous dump, but not\nin the current one.\n\n## Database Dump\n\nFor easier analysis of the dataset, or self-hosting, [a dump of the database (updated\nweekly) is available.](https://archive.ihr.live/ihr/rpki-history/)\n\n## Self-hosting\n\nIf for some reason you want to host your own version of this page, here is how.\n\n### Getting Started\n\nCreate secrets files containing the Postgres user passwords.\n\n```bash\n# For normal user\ntouch ./secrets/postgres-pw.txt\n# For read-only user\ntouch ./secrets/postgres-ro-pw.txt\n# Of course write actual passwords to these files...\n```\n\nInitialize database. This will also build the initial Docker image, which might take\nsome time.\n\n```bash\ndocker compose run --rm init-db\n```\n\n**Optional:** Restore database dump.\n\n```bash\ndocker compose exec -T database pg_restore -d rpki_history \u003c rpki-history.dump\n```\n\n**Optional (but recommended):** Create an index over the prefix column to greatly\ndecrease query time. This is not built into the init-db script, since it is faster to\nbuild the index once after all data was imported.\n\n```bash\ndocker compose exec database psql -c \"CREATE INDEX ON vrps USING gist (prefix inet_ops)\"\n```\n\nStart the API server.\n\n```bash\ndocker compose up -d api\n```\n\n### Updating the Data\n\nTo import the latest available data (uses RPKIViews by default):\n\n```bash\ndocker compose run --rm update-db\n```\n\nFor more usage options (e.g., importing a specific timestamp):\n\n```bash\ndocker compose run --rm update-db --help\n```\n\n### Backup/restore Database\n\n#### Backup\n\n```bash\ndocker compose exec database pg_dump --data-only -Fc \u003e rpki-history.dump\n```\n\n#### Restore\n\nAssumes fresh install or that `postgres_data` volume was deleted.\n\n```bash\n# Reinitialize database to create schema *and additional user*.\ndocker compose run --rm init-db\n# Restore data\ndocker compose exec -T database pg_restore -d rpki_history \u003c rpki-history.dump\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finternethealthreport%2Frpki-history","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finternethealthreport%2Frpki-history","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finternethealthreport%2Frpki-history/lists"}