{"id":16845231,"url":"https://github.com/stevenblack/ghosts","last_synced_at":"2025-03-17T05:32:06.010Z","repository":{"id":41177330,"uuid":"226625038","full_name":"StevenBlack/ghosts","owner":"StevenBlack","description":"🔎 A CLI to mess with hosts files and domain lists, local or remote.  It's written in Go.","archived":false,"fork":false,"pushed_at":"2021-10-17T02:51:02.000Z","size":80088,"stargazers_count":96,"open_issues_count":9,"forks_count":8,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-02-27T18:12:57.081Z","etag":null,"topics":["cli","golang","hosts"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/StevenBlack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-12-08T06:29:22.000Z","updated_at":"2025-02-07T07:57:08.000Z","dependencies_parsed_at":"2022-09-11T22:51:05.005Z","dependency_job_id":null,"html_url":"https://github.com/StevenBlack/ghosts","commit_stats":null,"previous_names":["stevenblack/hoststools","stevenblack/hostscompare"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StevenBlack%2Fghosts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StevenBlack%2Fghosts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StevenBlack%2Fghosts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StevenBlack%2Fghosts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/StevenBlack","download_url":"https://codeload.github.com/StevenBlack/ghosts/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243846976,"owners_count":20357294,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","golang","hosts"],"created_at":"2024-10-13T12:58:04.795Z","updated_at":"2025-03-17T05:32:00.987Z","avatar_url":"https://github.com/StevenBlack.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ghosts\n\n`ghosts` is a utility to evaluate, compare, and format hosts files.  It's written in [Go](https://golang.org/).\n\nHere is what `ghosts` does:\n\n* Summarize any hosts file retrieved over HTTP, or from a local file.\n* Compare two hosts files, and determine their intersection.\n* Compare a reference hosts file with a list of hosts presently in your system clipboard.\n* List the tally of TLDs in the hosts file.\n* Output the hosts as a plain list of domains, or with IP4 pefix.\n* Sort the hosts coherently by domain, TLD, subdomain, subsubdomain, and so on.\n\n## Getting started\n\n### Get help just as you might expect\n\n```\n$ ghosts -h\nUsage of ghosts:\n  -c string\n    \tHosts list to compare.\n    \tA shortcut code, full URL, or a local file.\n    \tUse the -m option for the main comparison list.\n    \tUse the -clip option to use what is on the system clipboard.\n\n    \tShortcut codes\n    \t==============\n    \tThe following shortcut codes can be used to select among preset main lists.\n\n    \tAmalgamated lists' shortcuts:\n    \t-c b or -m base // use Steven Black's base amalgamated list.\n    \t-c f    // use alternates/fakenews/hosts\n    \t-c fg   // use alternates/fakenews-gambling/hosts\n    \t-c fgp  // use alternates/fakenews-gambling-porn/hosts\n    \t-c fgps // use alternates/fakenews-gambling-porn-social/hosts\n    \t-c fgs  // use alternates/fakenews-gambling-social/hosts\n    \t-c fp   // use alternates/fakenews-porn/hosts\n    \t-c fps  // use alternates/fakenews-porn-social/hosts\n    \t-c fs   // use alternates/fakenews-social/hosts\n    \t-c g    // use alternates/gambling/hosts\n    \t-c gp   // use alternates/gambling-porn/hosts\n    \t-c gps  // use alternates/gambling-porn-social/hosts\n    \t-c gs   // use alternates/gambling-social/hosts\n    \t-c p    // use alternates/porn/hosts\n    \t-c ps   // use alternates/porn-social/hosts\n    \t-c s    // use alternates/social/hosts\n\n    \tSource lists' shortcuts:\n    \t-c adaway                // adaway.github.io\n    \t-c add2o7net             // FadeMind add.2o7Net hosts\n    \t-c adddead               // FadeMind add.Dead hosts\n    \t-c addrisk               // FadeMind add.Risk hosts\n    \t-c addspam               // FadeMind add.Spam hosts\n    \t-c adguard               // AdguardTeam cname-trackers\n    \t-c baddboyz              // mitchellkrogza Badd-Boyz-Hosts\n    \t-c clefspear             // Clefspeare13 pornhosts\n    \t-c digitalside           // davidonzo Threat-Intel\n    \t-c fakenews              // marktron/fakenews\n    \t-c hostsvn               // bigdargon hostsVN\n    \t-c kadhosts              // PolishFiltersTeam\n    \t-c metamask              // MetaMask eth-phishing hosts\n    \t-c mvps                  // winhelp2002.mvps.or\n    \t-c orca                  // orca.pet notonmyshift hosts\n    \t-c shady                 // hreyasminocha shady hosts\n    \t-c sinfonietta-gambling\n    \t-c sinfonietta-porn\n    \t-c sinfonietta-snuff\n    \t-c sinfonietta-social\n    \t-c someonewhocares       // Sam Pollock someonewhocares.org\n    \t-c stevenblack           // Steven Black ad-hoc list\n    \t-c tiuxo-porn\n    \t-c tiuxo-social\n    \t-c tiuxo                 // tiuxo list.\n    \t-c uncheckyads           // FadeMind  UncheckyAds\n    \t-c urlhaus               // urlhaus.abuse.ch\n    \t-c yoyo                  // Peter Lowe yoyo.org\n\n  -clip\n    \tThe comparison hosts are in the system clipboard\n  -d\tInclude default hosts at the top of file.\n  -intersection\n    \tReturn the list of intersection hosts? (default false)\n  -ip string\n    \tLocalhost IP address (default \"0.0.0.0\")\n  -m string\n    \tThe main list of hosts to analyze, or serve as a basis for comparison.\n    \tA shortcut code, a full URL, or a local file.\n    \tSee the -c flag for the list of shortcut codes. (default \"base\")\n  -noheader\n    \tRemove the file header from output? (default false)\n  -o\tReturn the list of hosts? (default false)\n  -p\tReturn a plain output list of hosts, with no IP address prefix? (default false)\n  -root\n    \tReturn the list of root domains and their tally (default false)\n  -s\tSort the hosts? (default false)\n  -stats\n    \tdisplay stats? (default true)\n  -tld\n    \tReturn the list of TLD and their tally (default false)\n  -unique\n    \tList the unique domains in the comparison list\n  -v\tReturn the current version\n```\n\n### Summarize statistics from any hosts file\n\n**If you specify no hosts file**, by default a summary of [StevenBlack/hosts](https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts) is produced.\n\n```\n$ ./ghosts\n--------------------------------------------------------------------------------\nBase hosts file summary:\n--------------------------------------------------------------------------------\nLocation: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts\nDomains: 54,702\nBytes: 1.7 MB\n--------------------------------------------------------------------------------\n```\n**Specify any hosts file to summarize** by using the `-m` option, like this:\n\n```\n$ ./ghosts -m https://someonewhocares.org/hosts/zero/hosts\n--------------------------------------------------------------------------------\nBase hosts file summary:\n--------------------------------------------------------------------------------\nLocation: https://someonewhocares.org/hosts/zero/hosts\nDomains: 14,401\nBytes: 417 kB\n--------------------------------------------------------------------------------\n```\n**Additionally produce a top-level-domain (TLD) report** by using the `-tld` option, like this:\n\n```\n$ ./ghosts -m https://someonewhocares.org/hosts/zero/hosts -tld\n----------------------------------------\nBase hosts file summary:\n----------------------------------------\nLocation: https://someonewhocares.org/hosts/zero/hosts\nDomains: 16,933\nBytes: 483 kB\nTLD tally:  (177 unique TLD)\n   com: 10,093\n   net: 2,634\n   info: 563\n   ru: 296\n   de: 263\n   org: 241\n   pl: 186\n   nl: 184\n   uk: 158\n\nskipping many lines for brevity\n\n   bo: 1\n   rw: 1\n   guru: 1\n   ae: 1\n   men: 1\n   ga: 1\n   watch: 1\n   ac: 1\n```\n**Additionally produce a root domain report** by using the `-root` option, like this:\n\n**Warning**: the `-root` option can produce thousands of lines of output. I recommend piping this to a file.\n\n```\n$ ghosts -m https://someonewhocares.org/hosts/zero/hosts -root\n----------------------------------------\nBase hosts file summary:\n----------------------------------------\nLocation: https://someonewhocares.org/hosts/zero/hosts\nDomains: 16,933\nBytes: 483 kB\nRoot domain tally:  (9,723 unique root domais)\n   2o7.net: 460\n   hitbox.com: 362\n   2mdn.net: 213\n   p2l.info: 198\n   co.uk: 152\n   oewabox.at: 125\n   am15.net: 120\n   intellitxt.com: 107\n   doubleclick.net: 103\n   fastclick.net: 102\n   checkm8.com: 102\n   adtech.de: 69\n   focalink.com: 65\n   adtech.us: 65\n   adtech.fr: 65\n   esomniture.com: 64\n   msn.com: 63\n   cjt1.net: 63\n   thruport.com: 59\n   imrworldwide.com: 55\n   bravenet.com: 54\n   plus.com: 52\n\nskipping many (many!) lines for brevity\n\n```\n\n**Compare two hosts files, local or remote, and assess their intersection** by specifying `-m \u003clocation\u003e` option for the main hosts file and `-c \u003clocation\u003e` option for the second comparison file.\n\nLet's compare the **someonewhocares.org** hosts file (15,474 domains) to the one at **mvps.org** (8,730 domains).  The basic report tells us there are 1,354 domains in the interseation of the two.\n\nHere we use **shortcut presets** to specify the two lists to compare, but we could have specified full URLs for either source.\n\n```\n$ ./ghosts -m someonewhocares -c mvps\n----------------------------------------\nBase hosts file summary:\n----------------------------------------\nLocation: https://someonewhocares.org/hosts/zero/hosts\nDomains: 15,474\nBytes: 445 kB\n----------------------------------------\n----------------------------------------\nCompared hosts file summary:\n----------------------------------------\nLocation: https://winhelp2002.mvps.org/hosts.txt\nDomains: 8,730\nBytes: 335 kB\n----------------------------------------\nIntersection: 1,354 domains\n```\n\n**Compare two hosts files, local or remote, and LIST their intersection** by specifying `-m \u003clocation\u003e` option for the main hosts file, `-c \u003clocation\u003e` option for the second comparison file, and add the `--intersection` flag to get the detailed list of the intersecting domains.\n\nLet's compare the **someonewhocares.org** hosts file (14,401 domains) to the one at **mvps.org** (10,473 domains).  The basic report shows us all 1,548 domains in the interseation of the two.\n\nHere we use **shortcut presets** to specify the two lists to compare, but we could have specified full URLs for either source.\n\n```\n$ ./ghosts -m someonewhocares -c mvps --intersection\n--------------------------------------------------------------------------------\nBase hosts file summary:\n--------------------------------------------------------------------------------\nLocation: https://someonewhocares.org/hosts/zero/hosts\nDomains: 15,474\nBytes: 445 kB\n--------------------------------------------------------------------------------\n--------------------------------------------------------------------------------\nCompared hosts file summary:\n--------------------------------------------------------------------------------\nLocation: http://winhelp2002.mvps.org/hosts.txt\nDomains: 8,730\nBytes: 335 kB\n--------------------------------------------------------------------------------\nintersection: [006.free-counter.co.uk 102.112.2o7.net 102.122.2o7.net 122.2o7.net 192.168.112.2o7.net\n1ca.cqcounter.com 1uk.cqcounter.com 1up.us.intellitxt.com 1us.cqcounter.com .... long list ]\nIntersection: 1,354 domains\n```\n\n**Compare two hosts files, local or remote, and list what's unique in the second file** by specifying `-m \u003clocation\u003e` option for the main hosts file, `-c \u003clocation\u003e` option for the second comparison file, and add the `--unique` flag to get the list of domains in the comparison file that are not in the main hoss file.\n\n\n### Output a list of domains in hosts format, or as a plaintext list\n\nTo list domains, use the `-o [optional file]` option.  If you provide no file mame, the list goes to `stdout`.\n\nTo sort the domains, use the `-s` flag.\n\n#### Hosts format output\n\nThe output is in hosts format by default.  The default IP address is `0.0.0.0`, and you can change that with the `-ip` option. for example, `-ip 127.0.0.1` will list the hosts with a `127.0.0.1` prefix.\n\nThe hosts output will include the header of the original source file.  This header typically includes information about the author, and sometimes includes a copyright statement.  To not include the header, use the `-noheader` flag.\n\nTo include the list of default hosts at the top of the hosts output, use the `d` flag.  By default the list of hosts does not include the loopback hosts at the top of the list.\n\nThese are the default hosts that will be included with the `-d` flag.\n\n```\n127.0.0.1 localhost\n127.0.0.1 localhost.localdomain\n127.0.0.1 local\n255.255.255.255 broadcasthost\n::1 localhost\n::1 ip6-localhost\n::1 ip6-loopback\nfe80::1%lo0 localhost\nff00::0 ip6-localnet\nff00::0 ip6-mcastprefix\nff02::1 ip6-allnodes\nff02::2 ip6-allrouters\nff02::3 ip6-allhosts\n0.0.0.0 0.0.0.0\n```\n\n#### Plaintext domains output\n\nTo get a plaintext list of domains, use the `-p` flag.\n\n\n\n## Running the tests\n\n`$ go test` runs the test suite.\n`$ gotest` runs colorized tests.\n\n## Contributing\n\nTBA.\n\n## License\n\nMIT.\n\n## Related repositories\n\n* [StevenBlack/hosts](https://github.com/StevenBlack/hosts) is my amalgamated hosts file, with custom variants, from various curated sources.\n* [StevenBlack/rhosts](https://github.com/StevenBlack/rhosts) hosts tools, written in Rust, just getting started on that.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstevenblack%2Fghosts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstevenblack%2Fghosts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstevenblack%2Fghosts/lists"}