{"id":50182092,"url":"https://github.com/ipanalytics/blackroute","last_synced_at":"2026-06-14T10:00:15.530Z","repository":{"id":359097048,"uuid":"1244494417","full_name":"ipanalytics/BlackRoute","owner":"ipanalytics","description":"Security intelligence pipeline for aggregating hostile IP infrastructure, abuse feeds, anonymizers, and attack telemetry into runtime lookup databases.","archived":false,"fork":false,"pushed_at":"2026-05-24T09:40:32.000Z","size":1361,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-25T07:07:49.250Z","etag":null,"topics":["abuse-intelligence","asn","botnet","cidr","cybersecurity","geoip","ip-blocklist","ip-reputation","ipintel","maxmind","mmdb","network-security","reputation-database","security-research","threat-detection","threat-feed","threat-intelligence","tor","vpn-detection"],"latest_commit_sha":null,"homepage":"https://ipanalytics.github.io/BlackRoute/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ipanalytics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-20T10:13:38.000Z","updated_at":"2026-05-24T09:39:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ipanalytics/BlackRoute","commit_stats":null,"previous_names":["ipanalytics/blackroute"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ipanalytics/BlackRoute","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipanalytics%2FBlackRoute","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipanalytics%2FBlackRoute/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipanalytics%2FBlackRoute/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipanalytics%2FBlackRoute/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ipanalytics","download_url":"https://codeload.github.com/ipanalytics/BlackRoute/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipanalytics%2FBlackRoute/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34316823,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-14T02:00:07.365Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["abuse-intelligence","asn","botnet","cidr","cybersecurity","geoip","ip-blocklist","ip-reputation","ipintel","maxmind","mmdb","network-security","reputation-database","security-research","threat-detection","threat-feed","threat-intelligence","tor","vpn-detection"],"created_at":"2026-05-25T07:04:50.925Z","updated_at":"2026-06-14T10:00:15.524Z","avatar_url":"https://github.com/ipanalytics.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Blackroute\n\nBlackroute builds a local IP reputation database from public abuse, malware, botnet C2, spam, phishing, brute-force, bogon, and cybercrime-prefix feeds. The primary artifact is `blackroute.mmdb`, a MaxMind-compatible database that can be used in gateways, proxies, fraud checks, SIEM pipelines, and internal enrichment jobs.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./site/banner.png\" alt=\"Blackroute builds a local IP reputation database from public abuse, malware, botnet C2, spam, phishing, brute-force, bogon, and cybercrime-prefix feeds.\" width=\"100%\"\u003e\n\u003c/p\u003e\n\nBlackroute does not resolve hostnames, query PTR records, crawl DNS, fingerprint networks, or scan anything. It only downloads configured feeds, extracts public IP addresses and CIDR prefixes, attaches labels, merges duplicates, and writes deterministic output files.\n\nThe default catalog currently contains 52 enabled IP/CIDR sources. It does not include VPN, proxy, Tor, or generic anonymizer lists as threat evidence.\n\n## Dataset Coverage\n\nThe source catalog is grouped around observed abuse rather than network type. Release artifacts include exact post-cleanup counts for records, single IPs, CIDR prefixes, sources, threat labels, infrastructure labels, and classification labels.\n\n| Area | Enabled source labels | What it contributes |\n| --- | ---: | --- |\n| Recent attacks and abuse | 24 | SSH, mail, web, SIP, FTP, bot, scanner, and short-window attacker signals |\n| Compromised or hostile hosts | 10 | Confirmed compromised IPs, botnet infrastructure, and hostile reputation feeds |\n| Active malware and C2 | 11 | abuse.ch, ThreatFox, C2IntelFeeds, Threatview, MalSilo, RomainMarcoux, Bitwire, and related C2 indicators |\n| Community and multi-sensor risk | 15 | Correlated reputation from CINSscore, IPsum, GreenSnow, AlienVault, DShield, AbuseIPDB mirrors, Bitwire, and hourly aggregates |\n| Brute-force and spam | 7 | SSH, POP3, mail, form spam, and abuse sources |\n| Cybercrime and bogon infrastructure | 4 | Spamhaus DROP/DROPv6 and Team Cymru fullbogon prefixes |\n\nTrust mix:\n\n| Trust level | Enabled sources |\n| --- | ---: |\n| `aggregator` | 29 |\n| `curated` | 9 |\n| `community` | 13 |\n| `authoritative` | 1 |\n\n## Why Blackroute\n\n- Transparent source mapping: every record keeps the feed name, source URL, confidence, threat labels, and infrastructure labels.\n- Cron-friendly operation: one binary, one YAML file, stable outputs, no admin panel.\n- Low runtime cost: compile once, then perform fast MMDB lookups in your own stack.\n- Practical alternative or supplement to paid reputation databases when you need local control, auditability, and repeatable builds.\n- Conservative parsing: private, local, multicast, unspecified, and overly broad ranges are ignored before output.\n\n## Outputs\n\n| File | Purpose |\n| --- | --- |\n| `release/blackroute.mmdb` | MaxMind DB for runtime IP and prefix lookups |\n| `release/blackroute.csv` | Flat table for review, diffing, and import jobs |\n| `release/blackroute.jsonl` | Line-delimited records for pipelines |\n| `release/run_stats.json` | Build summary, IP/CIDR counts, and label counts |\n\nMMDB records use this shape:\n\n```json\n{\n  \"matched_prefix\": \"203.0.113.10/32\",\n  \"threat\": [\"recent_attack_any\", \"recent_attack_ssh\"],\n  \"infrastructure\": [\"bogon\"],\n  \"classification\": [\"national_cert_malicious\"],\n  \"sources\": [\"blocklist_de_ssh\"],\n  \"confidence\": 70,\n  \"score\": 55,\n  \"level\": \"medium\",\n  \"observed_at\": \"2026-05-20T12:00:00Z\",\n  \"database_built_at\": \"2026-05-20T12:05:00Z\"\n}\n```\n\n## Quick Start\n\n```bash\nbash scripts/setup-server.sh\n./run.sh\n```\n\nBuild without MMDB when you only need CSV and JSONL:\n\n```bash\n./run.sh --skip-mmdb\n```\n\nRun only selected feeds:\n\n```bash\n./run.sh --only=blocklist_de_ssh,emergingthreats_compromised\n```\n\nUse a custom feed file or output directory:\n\n```bash\n./run.sh --feeds=configs/feeds.yaml --output=release\n```\n\nBuild the binary directly:\n\n```bash\ngo build -o ./bin/blackroute ./cmd/collector\n./bin/blackroute --feeds=configs/feeds.yaml --output=release\n```\n\nRun tests:\n\n```bash\ngo test ./...\n```\n\n## Releases\n\nThe release workflow runs daily at 03:17 UTC and can also be started manually from GitHub Actions. Releases use date tags in `YYYY.MM.DD` format and publish generated database artifacts:\n\n- `blackroute_\u003cYYYY.MM.DD\u003e.mmdb`\n- `blackroute_\u003cYYYY.MM.DD\u003e_exports.tar.gz` with CSV, JSONL, and run stats\n- `blackroute_\u003cYYYY.MM.DD\u003e_run_stats.json`\n- `blackroute_\u003cYYYY.MM.DD\u003e_cleanup_stats.json`\n- `blackroute_\u003cYYYY.MM.DD\u003e_release_summary.md`\n- `checksums.txt`\n\nRelease artifacts are cleaned before publication with [BogonForge](https://github.com/ipanalytics/BogonForge)-compatible public IP filtering. The release summary reports configured source count, source count after cleanup, records before cleanup, single IP and CIDR counts, records removed as bogon/reserved/invalid, and records left in the published database.\n\nBuild a local ThreatFox IP feed directly from the public abuse.ch export:\n\n```bash\nscripts/build-threatfox-feed.sh\n./run.sh --only=threatfox_ioc_ips\n```\n\nCheck configured HTTP feeds for availability and stale `Last-Modified` headers:\n\n```bash\nscripts/check-feeds.sh\nMAX_FEED_AGE_HOURS=72 scripts/check-feeds.sh configs/feeds.yaml\n```\n\n## Cron\n\nUse the wrapper when running from cron. It builds the binary if needed, prevents overlapping runs, and keeps Go build caches outside the repository by default.\n\n```cron\n17 * * * * cd /opt/blackroute \u0026\u0026 APP_DIR=/opt/blackroute scripts/run-cron.sh \u003e\u003e var/log/cron.log 2\u003e\u00261\n```\n\nManual cron-style run:\n\n```bash\nAPP_DIR=/opt/blackroute /opt/blackroute/scripts/run-cron.sh\n```\n\nOptional cache override:\n\n```bash\nBLACKROUTE_CACHE_DIR=/var/cache/blackroute/go ./run.sh\n```\n\n## Feed Configuration\n\nFeeds live in `configs/feeds.yaml`.\nReviewed upstream mappings are tracked in `docs/source-audit.md`.\n\n```yaml\nfeeds:\n  - kind: textlist\n    name: blocklist_de_ssh\n    display_name: blocklist.de SSH\n    trust: community\n    threat: [recent_attack_any, recent_attack_ssh]\n    urls:\n      - https://lists.blocklist.de/lists/ssh.txt\n```\n\nSupported fields:\n\n| Field | Meaning |\n| --- | --- |\n| `kind` | Currently `textlist`; extracts public IPs and CIDRs from text, CSV, JSON-ish, and netset-style lines |\n| `name` | Stable source identifier written to output records |\n| `display_name` | Human-readable source name for operators |\n| `disabled` | Set to `true` to keep a feed configured but inactive |\n| `trust` | `aggregator`, `community`, `curated`, or `authoritative`; controls default confidence |\n| `threat` | Labels for hostile behavior or active reputation |\n| `infrastructure` | Labels for network context such as bogons, hosting, cybercrime prefixes, or high-risk ASNs |\n| `classification` | Labels for source-specific category context such as scam, policy, C2, DNSBL, or national CERT signals |\n| `urls` | One or more feed URLs |\n\n## Included Sources\n\nThe default configuration includes:\n\n- blocklist.de: SSH, mail, web, IMAP, FTP, SIP, bots, and strong IP lists.\n- Emerging Threats: compromised and hostile hosts.\n- CINSscore: multi-sensor high-risk addresses.\n- FireHOL: conservative attacker and 1-day abuser aggregation.\n- Spamhaus: DROP and DROPv6 cybercrime prefixes.\n- Team Cymru: IPv4 and IPv6 full bogon prefixes.\n- abuse.ch Feodo Tracker: active botnet C2 IPs.\n- SANS ISC DShield, GreenSnow, and IPsum for community risk signals.\n- Binary Defense Banlist, ThreatFox IOC IPs, C2IntelFeeds, USOM malicious IPs, Inversion Cloud IPs, Inversion DNSBL IPv4, Ukrainian EMA fraud IPs, Global Anti Scam IPs, AlienVault reputation, Dataplane attack feeds, ZiyadNZ hourly aggregate IPs, StopForumSpam, Blocklist.net.ua, Threatview, MalSilo, Rutgers, BruteForceBlocker, POP3 Gropers, Phishing.Database IPs, AbuseIPDB high-confidence mirrors, RomainMarcoux 40K inbound/outbound, ShadowWhisperer scanners, IP BlockList v4 Level 3+, and Bitwire inbound/outbound.\n\nCommercial feeds and API-key feeds are intentionally not bundled. Add them as private entries in `configs/feeds.yaml` when your license allows local redistribution or internal use.\n\nBlackroute prefers direct upstream feeds when they are public and parser-compatible. Mirror feeds are kept only where upstream access is API-key based, browser-session based, unstable, or domain-heavy. The default ThreatFox entry checks `local/feeds/threatfox_ips.txt` first and then reads the official abuse.ch ZIP/CSV export directly.\n\n## Labels\n\nThreat labels describe behavior:\n\n```json\n[\n  \"recent_attack_any\",\n  \"recent_attack_ssh\",\n  \"recent_attack_mail\",\n  \"recent_attack_web\",\n  \"recent_attack_imap\",\n  \"recent_attack_ftp\",\n  \"recent_attack_sip\",\n  \"recent_badbot_or_regbot\",\n  \"persistent_attacker\",\n  \"malware_host_active\",\n  \"compromised_or_hostile_host\",\n  \"community_high_risk\",\n  \"multi_sensor_high_risk\",\n  \"aggregate_abuser_1d\",\n  \"c2_ioc\",\n  \"bruteforce\",\n  \"spam\",\n  \"abuse\",\n  \"phishing_or_scam\",\n  \"network_scan_or_abuse\"\n]\n```\n\nInfrastructure labels describe network context:\n\n```json\n[\n  \"hosting\",\n  \"bogon\",\n  \"prefix_cybercrime\",\n  \"asn_high_risk\"\n]\n```\n\nClassification labels describe source category without forcing everything into `threat`:\n\n```json\n[\n  \"c2_ioc\",\n  \"national_cert_malicious\",\n  \"malicious_url_or_ip\",\n  \"cloud_hosting_abuse_derived\",\n  \"dnsbl_malicious\",\n  \"safe_browsing_malicious\",\n  \"phishing_or_scam\",\n  \"financial_fraud\",\n  \"policy_illegal_gambling\",\n  \"scam_or_fraud\",\n  \"alienvault_otx_reputation\",\n  \"network_scan_or_abuse\",\n  \"aggregate_threat_intel_hourly\",\n  \"cobalt_strike_c2\",\n  \"command_and_control\",\n  \"malware_distribution\",\n  \"ssh_bruteforce\",\n  \"mail_bruteforce\",\n  \"phishing_ip\",\n  \"spam_source\",\n  \"aggregate_abuseipdb_confidence_100\",\n  \"aggregate_blacklist_scored\",\n  \"aggregate_inbound_threat\",\n  \"aggregate_outbound_threat\",\n  \"internet_scanner\",\n  \"reconnaissance\"\n]\n```\n\n## Project Layout\n\n```text\ncmd/collector/              CLI entrypoint\nconfigs/                    Feed configuration\ninternal/config/            YAML loader\ninternal/domainx/           IP and CIDR normalization\ninternal/downloader/        HTTP fetch client\ninternal/source/textlist/   Feed parser\ninternal/pipeline/          Fetch, merge, and write flow\ninternal/output/            CSV, JSONL, stats, and MMDB writers\ninternal/record/            Shared record model\nscripts/                    Setup and cron wrappers\nsite/                       Static GitHub Pages site\n```\n\n## Notes\n\nBlackroute is a reputation compiler, not a verdict engine. Treat labels as signals, combine them with your own allowlists and policy, and review high-impact blocking decisions before enforcing them globally.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fipanalytics%2Fblackroute","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fipanalytics%2Fblackroute","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fipanalytics%2Fblackroute/lists"}