{"id":18841699,"url":"https://github.com/asifbacchus/adserver-list-update","last_synced_at":"2025-06-15T00:05:28.181Z","repository":{"id":169612669,"uuid":"133781489","full_name":"asifbacchus/adserver-list-update","owner":"asifbacchus","description":"bash-script to download updated adserver list files, deduplicate them and format the combined output file for use with DNSMASQ external hosts file.","archived":false,"fork":false,"pushed_at":"2018-05-21T15:52:07.000Z","size":14,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-22T22:19:54.632Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/asifbacchus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-17T08:15:40.000Z","updated_at":"2020-04-20T11:40:27.000Z","dependencies_parsed_at":null,"dependency_job_id":"2c051147-c5d4-4754-929b-751cf8bc6255","html_url":"https://github.com/asifbacchus/adserver-list-update","commit_stats":null,"previous_names":["asifbacchus/adserver-list-update"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asifbacchus%2Fadserver-list-update","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asifbacchus%2Fadserver-list-update/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asifbacchus%2Fadserver-list-update/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asifbacchus%2Fadserver-list-update/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/asifbacchus","download_url":"https://codeload.github.com/asifbacchus/adserver-list-update/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asifbacchus%2Fadserver-list-update/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":257103736,"owners_count":22490478,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T02:52:11.526Z","updated_at":"2025-06-15T00:05:28.118Z","avatar_url":"https://github.com/asifbacchus.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# adserver-list-update\nbash-script to download updated adserver list files, deduplicate them and format\nthe combined output file for use with DNSMASQ external hosts file.\n\nThis script downloads several popular, well maintained, lists of known adservers\nthen amalagates those lists into one large list, removes duplicate entries and\nformats the whole thing to point to configurable IP4 and IP6 addresses in a\nformat usable as a general hosts file or with DNSMASQ as an external hosts file. \n\nLet me first say this is made for my personal use on my ad-blocking DNSMASQ\nserver (I'll link to my blog detailing full setup later).  I don't script often\nand the whole ad-blocking server is my way of better learning linux.  That being\nsaid, ANY suggestions (with explanations, please, so I can learn!) would be\nappreciated.  Otherwise, if this helps you too, then awesome!  \n\nObviously no warranties or guantees are made by me in any form nor do I accept\nany responsibility if this destroys your computer, your personal life, your\ncountry or the world in general.\n\nScript overview:\n\nVariables:\nIPv4 redirect address ($ipv4addr) - i.e. your webserver or 0.0.0.0\nIPv6 redirect address ($ipv6addr) - i.e. your webserver or ::\nScript's location ($dir) - this is auto-set to the current working directory\nLocation for downloaded files ($locationSourcefiles) - deleted after use \nLocation for working temp files ($locationWorkingfiles) - deleted after use\nLocation for output file ($listpath) - where final output file should be saved\n\nThis script downloads a set of known adserver addresses/hosts files from the\nfollowing sources:\n  https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts\n  https://mirror1.malwaredomains.com/files/justdomains\n  http://sysctl.org/cameleon/hosts\n  https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist\n  https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt\n  https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt\n  https://hosts-file.net/ad_servers.txt\n\nThe files are then parsed individually as needed to remove comments, interface\naddresses and target IP addresses.  A working copy of just domain names is saved\nin the $locationWorkingfiles directory.\n\nAll individual lists are combined into one list, sorted alphabetically for ease\nof reading and duplicates are removed.\n\nThe $ipv4addr is prepended to one copy of the combined list and the $ipv6addr\nis prepended to a second copy of the same list.  The two lists are then merged\nand sorted so that each host has an IP4 entry immediately followed by the\ncorresponding IP6 entry.  This is the final list and is saved in $listpath as\n'adblock.dnsmasq'.\n\nThe script then deletes the $locationSourcefiles and $locationWorkingfiles\ndirectories and their contents so that future updates start 'clean'.\n\nFinally, DNSMASQ is restarted via systemctl so it reads the updated list file\nthat was generated by this script.\n\n\nTo Do:\n- Make it prettier\n- Add checking for root priviledges\n- Add entries into syslog? separate logfile?\n- Add chron entries to run this script at user-specified interval?\n\n\nLet me know if you have any suggestions!\nThanks for checking out this script :-)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasifbacchus%2Fadserver-list-update","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fasifbacchus%2Fadserver-list-update","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasifbacchus%2Fadserver-list-update/lists"}