{"id":45150929,"url":"https://github.com/tn3w/ipblocklist","last_synced_at":"2026-06-04T06:00:32.576Z","repository":{"id":340022962,"uuid":"1162075659","full_name":"tn3w/IPBlocklist","owner":"tn3w","description":"Aggregated IP/ASN threat intel. ~20 MB mmap DB + ~5 MB netset blocklist.","archived":false,"fork":false,"pushed_at":"2026-05-31T20:01:57.000Z","size":6626,"stargazers_count":5,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-05-31T21:23:35.632Z","etag":null,"topics":["ip-address","ip-blocklist","ip-lookup","python","security","threat-intelligence"],"latest_commit_sha":null,"homepage":"http://ipblocklist.tn3w.dev/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tn3w.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-19T21:02:36.000Z","updated_at":"2026-05-31T19:57:20.000Z","dependencies_parsed_at":"2026-04-05T04:01:04.011Z","dependency_job_id":null,"html_url":"https://github.com/tn3w/IPBlocklist","commit_stats":null,"previous_names":["tn3w/ipblocklist"],"tags_count":69,"template":false,"template_full_name":null,"purl":"pkg:github/tn3w/IPBlocklist","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tn3w%2FIPBlocklist","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tn3w%2FIPBlocklist/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tn3w%2FIPBlocklist/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tn3w%2FIPBlocklist/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tn3w","download_url":"https://codeload.github.com/tn3w/IPBlocklist/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tn3w%2FIPBlocklist/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33891733,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-04T02:00:06.755Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ip-address","ip-blocklist","ip-lookup","python","security","threat-intelligence"],"created_at":"2026-02-20T03:02:17.167Z","updated_at":"2026-06-04T06:00:32.570Z","avatar_url":"https://github.com/tn3w.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IPBlocklist\n\n[![Build](https://img.shields.io/github/actions/workflow/status/tn3w/IPBlocklist/build.yml?label=build)](https://github.com/tn3w/IPBlocklist/actions)\n[![Intel sources](https://img.shields.io/badge/intel.bin-192_sources-blue)](feeds-intel.json)\n[![License](https://img.shields.io/badge/license-Apache_2.0-lightgrey)](LICENSE)\n\nAggregated IP/ASN threat intel. ~20 MB mmap DB + ~5 MB netset blocklist.\nConsumer-side 0-100 scoring.\n\n```bash\nwget https://github.com/tn3w/IPBlocklist/releases/latest/download/intel.bin\nwget https://github.com/tn3w/IPBlocklist/releases/latest/download/blocklist.netset\n```\n\nDemo: [ipblocklist.tn3w.dev](https://ipblocklist.tn3w.dev).\n\n## Artifacts\n\n| file | role |\n| ---- | ---- |\n| `intel.bin` | columnar mmap DB, 20-flag bitmask |\n| `blocklist.netset` | CIDR netset with metadata header |\n| `asns.json` | ASN lists by feed |\n| `blocklist.bin` | legacy IPBL v2 |\n\n# intel.bin\n\nBuilt by `builder/` (Rust) from `feeds-intel.json`. SoA columnar.\n\n## Layout\n\n256B LE header, version 6. IPv4 split by /16: in-bucket ranges as\n`(start_lo u16, len u16, val u16)` (6B); cross-/16 ranges in overflow\nwith full u32.\n\n| offset | size | field |\n| -----: | ---- | ----- |\n| 0 | u32 | version (6) |\n| 4 | u32 | reserved |\n| 8 | u64 | v4_compact_count |\n| 16 | u64 | v4_large_count |\n| 24 | u64 | v6_count |\n| 32 | u64 | val_count |\n| 40 | u64 | str_count |\n| 48..152 | u64 × 13 | section offsets + str_data_len |\n\nSections:\n\n- `v4_bucket`: `65537 × u32`, cumulative row offset per /16.\n- compact `starts_lo`, `lens`, `vals`: `v4_compact_count × u16`, sorted\n  by `(bucket, start_lo)`. Range = `[prefix|start_lo, prefix|(start_lo+len)]`.\n- large `starts`, `ends`: `× u32`. `vals`: `× u16`. Cross-/16 ranges.\n- v6: `starts`, `ends`: `× u128` sorted. `vals`: `× u16`.\n- values: `val_count × {flags u32, provider_id u32, source_id u32, _pad u32}`.\n- string index + data.\n\nFlags (LSB first): `vpn, proxy, tor, malware, c2, scanner, brute_force,\nspammer, compromised, datacenter, cdn, anycast, crawler, bot, cloud,\nprivate_relay, anonymizer, mobile, isp, government`.\n\nPrefix-max of ends computed at load (per-bucket compact; global large/v6).\n\nLookup v4: `bucket = ip\u003e\u003e16, lo = ip\u00260xFFFF`. Bisect compact bucket on\n`starts_lo`; scan back while `max_ends_lo[i] \u003e= lo`. Bisect large globally.\n`score` = max flag severity (0-95), computed on demand.\n\n## Build\n\n```bash\ncd builder\ncargo build --release\nFEEDS_FILE=../feeds-intel.json OUT_FILE=../intel.bin \\\n  ./target/release/builder update\n./target/release/builder check 1.2.3.4\n./target/release/builder bench 100000\n./target/release/builder blocklist 40 cidr   # range also valid\n./target/release/builder analyze\n./target/release/builder sweep\n```\n\nProduces: `builder` CLI, `libipintel.so`/`.a`, `include/ipintel.h`.\n\nASN resolved offline from `asndb-mini.bin` (`ASNDB_FILE` env). HTTP cache:\n`request_cache/`.\n\n## C ABI\n\nmmap, zero-copy, share one `ipintel_db*` across threads (read-only).\n\n```c\n#include \"ipintel.h\"\nipintel_db* db = ipintel_open(\"intel.bin\");\nuint32_t ip = (192u\u003c\u003c24)|(168u\u003c\u003c16)|(1u\u003c\u003c8)|1u;\nuint8_t  s = ipintel_lookup_v4_score(db, ip);\nuint32_t f = ipintel_lookup_v4_flags(db, ip);\nuint8_t  a = ipintel_lookup_v4_action(db, ip, 80, 35);\nipintel_close(db);\n```\n\n`*_action` returns `ALLOW` (0), `CHALLENGE` (1), `BLOCK` (2).\n\n## feeds-intel.json\n\n`{ \"flags\": [...], \"feeds\": [...] }`. Per source: `name`, `flags`\n(subset of 20), `url`+`regex`, or `is_asn: true` + `asns`/`url+regex`,\noptional `provider`. Optional `only_unique: true` runs the feed after\nall others and keeps only ranges not already covered (partial overlaps\nsplit). Optional `provider_map_url` points to an IP→provider TSV\n(`#dict` + `#data` sections) that splits the feed per provider.\n\n## Python lookup\n\n```bash\npython3 lookup.py 185.220.101.1\n```\n\n## golf/\n\nCompact, self-contained ports of the loader+lookup across many languages.\nSame JSON output as `lookup.py`, same scoring. Run `intel.\u003cext\u003e` directly\n(or compile per the language's conventions); each script reads `../intel.bin`.\n\nSeverities: `malware`/`c2`=95, `compromised`=75, `brute_force`=70,\n`spammer`=65, `scanner`=55, `tor`=45, `bot`=40, `anonymizer`=35,\n`vpn`=30, `proxy`=25, `private_relay`/`datacenter`=15, `cloud`/`crawler`=10,\n`cdn`=5, rest 0.\n\nScore: `severity × (1 + log2(1/prevalence)/24)`, top + 15% of rest,\nmulti-source boost `× (1 + 0.08·log2(sources+1))`, capped at 100.\nLevels: `critical \u003e=80`, `high \u003e=60`, `medium \u003e=35`, `low \u003e=15`, else `minimal`.\n\n20k sample: Spearman 0.94, Pearson 0.83 vs top-flag severity.\n\n# blocklist.netset\n\nPlain-text CIDR list with `#`-prefixed metadata header, emitted by\n`builder update`. CIDR is universal (iptables/ipset/nftables/ufw/pfSense).\nPass `range` for `start-end` form.\n\nPipeline: iter intel.bin ranges, filter `max_severity \u003e= threshold`,\nsort+merge, emit. Single IPs collapse to bare addr.\n\nDefault threshold **40** (`BLOCKLIST_THRESHOLD` env): malware/c2/compromised\n/brute_force/spammer/scanner/tor/bot. ~5.4 MB, 685M IPs. Below 40\nanonymizer/vpn add ASN-wide ranges (4x size, poor signal); above 55 drops\nto ~1-3 MB but misses scanners/tor. `builder sweep` prints the full table.\n\n```bash\nipset create blocklist hash:net\nawk '!/^#/' blocklist.netset | xargs -n1 ipset add blocklist\n```\n\n# Pipeline\n\n```mermaid\nflowchart LR\n    F[feeds-intel.json] --\u003e B[builder/]\n    B --\u003e I[intel.bin]\n    B --\u003e N[blocklist.netset]\n    A[feeds.json] --\u003e P[aggregator.py]\n    P --\u003e J[asns.json]\n    P --\u003e L[blocklist.bin]\n```\n\n# Deprecated: blocklist.bin\n\nIPBL v2 binary. Legacy; new integrations use `intel.bin`. Built from\n`feeds.json`.\n\n```text\n[4 magic \"IPBL\"][1 version=2][4 timestamp LE]\n[1 flag_count]  × { [1 len][N utf-8] }\n[1 cat_count]   × { [1 len][N utf-8] }\n[2 feed_count LE]\nfeed_count × {\n  [1 len][N name]\n  [1 base_score 0-200][1 confidence 0-200]\n  [4 flags LE][1 cats]\n  [4 range_count LE]\n  range_count × { [varint start_delta][varint size] }\n}\n```\n\n25-language lookup examples in `examples/`.\n\n# License\n\n[Apache-2.0](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftn3w%2Fipblocklist","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftn3w%2Fipblocklist","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftn3w%2Fipblocklist/lists"}