{"id":20450069,"url":"https://github.com/ooni/historical-geoip","last_synced_at":"2026-05-03T07:04:45.316Z","repository":{"id":145890548,"uuid":"513971666","full_name":"ooni/historical-geoip","owner":"ooni","description":"Generate historical IP to country + ASN databases for processing historical OONI data","archived":false,"fork":false,"pushed_at":"2024-03-22T13:16:06.000Z","size":45,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-04-14T09:04:43.666Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ooni.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-07-14T16:25:21.000Z","updated_at":"2024-04-15T21:21:40.592Z","dependencies_parsed_at":null,"dependency_job_id":"b174b836-8693-4301-be08-8d456e5a3f4f","html_url":"https://github.com/ooni/historical-geoip","commit_stats":{"total_commits":28,"total_committers":3,"mean_commits":9.333333333333334,"dds":0.3571428571428571,"last_synced_commit":"7dd3e39570e22d233d463ce41313357be7283718"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ooni%2Fhistorical-geoip","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ooni%2Fhistorical-geoip/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ooni%2Fhistorical-geoip/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ooni%2Fhistorical-geoip/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ooni","download_url":"https://codeload.github.com/ooni/historical-geoip/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239869662,"owners_count":19710561,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T10:50:10.519Z","updated_at":"2026-04-19T11:32:48.851Z","avatar_url":"https://github.com/ooni.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Historical GeoIP databases\n\nThe purpose of this repo is to build historical IP to country + AS databases\nfor use in the OONI data processing pipeline, but potentially for use by the\nprobes as well.\n\nThe data sources used for IP to country mappings are:\n* Historical copies of [Maxmind GeoIP2 Country Lite](https://dev.maxmind.com/geoip/geolite2-free-geolocation-) from [20180206 to 20191224](https://archive.org/download/maxmind-geolite2-country).\n* From 2020-02 onward we use monthly [DB-IP IP2Country Lite](https://db-ip.com/db/download/ip-to-country-lite), which are also [publicly archived](https://archive.org/download/dbip-country-lite).\n\nFor mapping IP ranges to ASNs we use the [prefix2as mappings from CAIDA](https://publicdata.caida.org/datasets/routing/routeviews-prefix2as/).\n\nFor mapping ASNs with metadata about the organization, we use the as [AS to Organization mappings from CAIDA](https://publicdata.caida.org/datasets/as-organizations/).\n\nThe primary entry point for running the full workflow is the following:\n```\n./update_databases.sh\n```\n\nIn order to upload the built artifacts to archive.org, you should have the set\n`IA_ACCESS_KEY` and `IA_SECRET_KEY` environment variables.\n\nThe workflow for generating the final artifacts (the IP to country + ASN mmdb\nfiles) is the following:\n```mermaid\ngraph\n    A[AS Organizations] --\u003e E{{AS to ORG Map}}\n    E --\u003e D{Enrich country DB}\n    B[Maxmind GeoIP2 Country] --\u003e D\n    C[DB-IP IP2Country] --\u003e D\n    F[prefix2AS] --\u003e D\n    D --\u003e O{{IP to Country + ASN mmdb}}\n```\n\nBoth the AS to ORG Map and the timestamped IP to Country + ASN mmdb files are\n[published as artifacts on archive.org](https://archive.org/download/ip2country-as).\n\nNote: if the repository is inactive for more than 60 days, the GitHub action\nthat publishes artifacts is disabled and we don't publish artifacts.\n\nThe IP to Country + ASN is compatible with the mmdb file format, but country\nand ASN lookup are supported inside of the same call.\n\nThe keys used in the result for returning metadata information are the following:\n* `autonomous_system_number`, this is an INT indicating the ASN. It's a standard key.\n* `autonomous_system_organization`, this is a string indicating the\n  organization name for the given ASN. It's a standard key.\n* `autonomous_system_country`, is the country of registration of the AS\n  organization. This key is non-standard.\n* `autonomous_system_name`, is the name of the AS, which in most cases is\n  different from the organization name. This key is non-standard.\n\n## Skipped workflows\n\nIf the workflow happens to be skipped for more than a month you may need to backfill the missing older dates.\n\nThis can be done by passing the missing timestamp to the `./update_databases.sh` script, like so:\n\n```\n./update_databases.sh YYYYMM\n```\n\nYou can also manually trigger the workflow for debug purposes using the [github CLI](https://cli.github.com/) command:\n\n```\ngh workflow run .github/workflows/update_databases.yml\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fooni%2Fhistorical-geoip","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fooni%2Fhistorical-geoip","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fooni%2Fhistorical-geoip/lists"}