{"id":43948263,"url":"https://github.com/sol1/blocklist-domains","last_synced_at":"2026-02-07T03:07:54.004Z","repository":{"id":301485953,"uuid":"1008110410","full_name":"sol1/blocklist-domains","owner":"sol1","description":"A github action makes a blocklist of domains in cdb format for dnsdist","archived":false,"fork":false,"pushed_at":"2025-07-25T02:25:01.000Z","size":24072,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-25T07:35:06.679Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sol1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-25T03:41:53.000Z","updated_at":"2025-07-25T02:25:05.000Z","dependencies_parsed_at":"2025-06-27T05:48:18.351Z","dependency_job_id":null,"html_url":"https://github.com/sol1/blocklist-domains","commit_stats":null,"previous_names":["sol1/blocklist-domains"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sol1/blocklist-domains","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sol1%2Fblocklist-domains","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sol1%2Fblocklist-domains/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sol1%2Fblocklist-domains/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sol1%2Fblocklist-domains/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sol1","download_url":"https://codeload.github.com/sol1/blocklist-domains/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sol1%2Fblocklist-domains/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29185144,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-07T00:44:15.062Z","status":"online","status_checked_at":"2026-02-07T02:00:07.217Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-07T03:07:53.956Z","updated_at":"2026-02-07T03:07:53.999Z","avatar_url":"https://github.com/sol1.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Testing](https://github.com/dmachard/blocklist-aggregator/workflows/Testing/badge.svg) ![Build](https://github.com/dmachard/blocklist-aggregator/workflows/Build/badge.svg) ![Publish](https://github.com/dmachard/blocklist-aggregator/workflows/Publish/badge.svg) \n\n# Sol1 Blocklist Aggregator\n\nA fork of **[python-blocklist-aggregator](https://github.com/dmachard/python-blocklist-aggregator)** by dmachard.\nThis python module does the aggregation of several ads/tracking/malware lists, and merges them into a unified list with duplicates removed.\nCreate your own list from several sources.\n\nSee the **[blocklist-domains](https://github.com/dmachard/blocklist-domains)** repository for an implementation.\n\nDefault sources are defined on the [configuration file](../main/blocklist_aggregator/blocklist.conf)\n\n## Table of contents\n\n* [Installation](#installation)\n* [Get Started](#get-started)\n* [Custom Configuration](#custom-configuration)\n* [Fetch and save-it to files](#fetch-and-save-it-to-files)\n\n## Installation\n\n![python 3.13.x](https://img.shields.io/badge/python%203.13.x-tested-blue) ![python 3.12.x](https://img.shields.io/badge/python%203.12.x-tested-blue) ![python 3.11.x](https://img.shields.io/badge/python%203.11.x-tested-blue) ![python 3.10.x](https://img.shields.io/badge/python%203.10.x-tested-blue) ![python 3.9.x](https://img.shields.io/badge/python%203.9.x-tested-blue) ![python 3.8.x](https://img.shields.io/badge/python%203.8.x-tested-blue)\n\nIf you want to generate your own unified blocklist, \ninstall this module with the pip command.\n\n```python\npip install blocklist_aggregator\n```\n\n## Get started\n\nThis basic example enable to get a unified list of domains.\nYou can save-it in a file or do what you want.\n\n```python\nimport blocklist_aggregator\n\nunified = blocklist_aggregator.fetch()\nprint(unified)\n[ \"doubleclick.net\", ..., \"telemetry.dropbox.com\" ]\n\nprint(len(unified))\n152978\n```\n\n## Custom configuration\n\nSee the default [configuration file](../main/blocklist_aggregator/blocklist.conf)\n\nThe configuration contains:\n\n* the ads/tracking/malware URL lists with the pattern (regex) to use\n* the domains list to exclude (whitelist)\n* additionnal domains list to block (blacklist)\n\nThe configuration can be overwritten at runtime.\n\n```python\ncfg_yaml = \"verbose: true\"\nunified = blocklist_aggregator.fetch(cfg_update=cfg_yaml)\n```\n\nor loaded from external config file\n\n```python\nunified = blocklist_aggregator.fetch(cfg_filename=\"/home/custom-blocklist.conf\")\n```\n\n## Fetch and save-it to files\n\nThis module can be used to export the list in several format:\n\n* text\n* hosts\n* map (for use with TinyCDB)\n* CDB (key/value database)\n\n```python\nimport blocklist_aggregator\n\n# fetch domains\nunified = blocklist_aggregator.fetch()\n\n# save to a text file\nblocklist_aggregator.save_raw(filename=\"/tmp/unified_list.txt\")\n\n# save to hosts file\nblocklist_aggregator.save_hosts(filename=\"/tmp/unified_hosts.txt\", ip=\"0.0.0.0\")\n\n# save to a text file in map format\nblocklist_aggregator.save_map(filename=\"tmp/unified_map.txt\")\n\n# save to CDB\nblocklist_aggregator.save_cdb(filename=\"/tmp/unified_domains.cdb\")\n```\n\n## For developpers\n\nRun test units\n\n```bash\npython3 -m unittest discover tests/ -v\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsol1%2Fblocklist-domains","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsol1%2Fblocklist-domains","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsol1%2Fblocklist-domains/lists"}