Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/t145/black-mirror
Blacklists and whitelists built by open code, so you know what goes into them.
https://github.com/t145/black-mirror
adblock-list blackhole-lists blacklist blacklist-data blacklist-domains blacklist-feed blacklist-ips blacklist-management blacklist-script blacklist-service blocklist daily-build dns privacy whitelist whitelists whitelists-hosts whitelists-ip
Last synced: about 14 hours ago
JSON representation
Blacklists and whitelists built by open code, so you know what goes into them.
- Host: GitHub
- URL: https://github.com/t145/black-mirror
- Owner: T145
- License: agpl-3.0
- Created: 2021-05-28T17:01:48.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2024-03-19T13:43:39.000Z (10 months ago)
- Last Synced: 2024-03-19T16:47:10.102Z (10 months ago)
- Topics: adblock-list, blackhole-lists, blacklist, blacklist-data, blacklist-domains, blacklist-feed, blacklist-ips, blacklist-management, blacklist-script, blacklist-service, blocklist, daily-build, dns, privacy, whitelist, whitelists, whitelists-hosts, whitelists-ip
- Language: Shell
- Homepage:
- Size: 86.2 MB
- Stars: 168
- Watchers: 4
- Forks: 11
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.txt
- Code of conduct: CODE_OF_CONDUCT.md
- Citation: CITATION.cff
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
🌓 Reflection | 💿 Redundancy | ✅ Reliability
Aiming to promote privacy, security, and safety across the internet!
Made with 💖 by
## 🚚 Deliverables
> The whitelists are applied to all the blacklists!
List Name
File Content
Unique Entries
File Size
Mirror
BLOCK_DOMAIN
Punycode domains
8,480,311
182M
🔗
BLOCK_IPV4
IPv4 addresses
1,358,936
19M
🔗
BLOCK_IPV6
IPv6 addresses
932
27K
🔗
BLOCK_CIDR4
IPv4 CIDR blocks
9,913
163K
🔗
BLOCK_CIDR6
IPv6 CIDR blocks
213
3.6K
🔗
ALLOW_DOMAIN
Punycode domains
729,637
15M
🔗
ALLOW_IPV4
IPv4 addresses
75,723
1.1M
🔗
ALLOW_IPV6
IPv6 addresses
2,876
109K
🔗
ALLOW_CIDR4
IPv4 CIDR blocks
7,170
111K
🔗
ALLOW_CIDR6
IPv6 CIDR blocks
150,188
2.4M
🔗
### Source Code
- [Tarball](https://api.github.com/repos/T145/black-mirror/tarball/latest)
- [Zipball](https://api.github.com/repos/T145/black-mirror/zipball/latest)
## 🧮 Checksum Evaluation
```bash
sha256sum CHECKSUMS.txt --strict
```
The output will print `OK` if the check was successful. Be sure to run the command in the same directory as the lists!
## 🐙 Fetching GitHub Releases
Provided below are some examples to fetch release artifacts leveraging the GitHub API.
### Get all build artifacts
```bash
curl --proto '=https' --tlsv1.3 -H 'Accept: application/vnd.github.v3+json' -sSf https://api.github.com/repos/T145/black-mirror/releases/latest | jq -r '.assets[].browser_download_url'
```
### Get a build artifact & its checksum
```bash
curl --proto '=https' --tlsv1.3 -H 'Accept: application/vnd.github.v3+json' -sSf https://api.github.com/repos/T145/black-mirror/releases/latest | jq -r '.assets[] | select(.name | startswith("BLOCK_DOMAIN")).browser_download_url'
```
### Get a single build artifact
```bash
curl --proto '=https' --tlsv1.3 -H 'Accept: application/vnd.github.v3+json' -sSf https://api.github.com/repos/T145/black-mirror/releases/latest | jq -r '.assets[] | select(.name | startswith("BLOCK_DOMAIN")) | select(.name | endswith(".txt")).browser_download_url'
```
## 🖋️ Manifesto
> Please report any hosts or lists that do not align in an issue.
`Black Mirror` stands on three pillars: **privacy**, **security**, and **safety**.
- **Privacy**:
- Whitelist services like trusted torrenting providers, VPNs, and Tor.
- Blacklist deanonymized telemetry, data brokers, ad redirects, and stalkerware.
- **Security**:
- Whitelist services that promote security AND privacy.
- Blacklist illegal or sketchy sites and known malicious actors.
- Blacklist advertising to avoid the risks of malvertising.
- **Safety**:
- Blacklist deliberate misinformation, irreputable news sources, propaganda, etc.
- Blacklist vices such as vaping and smoking that cause individual harm.
- Use [GitHub's non-allowance policies](https://docs.github.com/en/github/site-policy/github-community-guidelines#what-is-not-allowed) at-scale where possible.
- [Acknowledge technological harm, and do what can be done in this project to prevent it.](https://ledger.humanetech.com/)
While `Black Mirror` censors content beyond privacy and security, it does not seek to do so irresponsibly.
Many other popular blacklists say they block specific hosts for certain reasoning, and when combed through turn out to have many false positives, or blocks that fall outside that reasoning. `Black Mirror` aims to be different by being as transparent as possible. It's open to critique, and encourages community feedback and involvement.
The real power behind the freedom of speech is the ability to hear many opinions to quickly deduce those that are correct or reasonable.
“Before you become too entranced with gorgeous gadgets and mesmerizing video displays, let me remind you that information is not knowledge, knowledge is not wisdom, and wisdom is not foresight. Each grows out of the other, and we need them all.”\
― *Arthur C. Clarke*
## Table of contents
- [Black Mirror](#black-mirror)
- [On Spyware/Stalkerware](#on-spywarestalkerware)
- [On Propaganda](#on-propaganda)
- [On Social Media](#on-social-media)
## On Spyware/Stalkerware
These services intrude on privacy, are usually malware, and compromise individual safety. Therefore any known parties are blocked.
“There will come a time when it isn't 'They're spying on me through my phone' anymore. Eventually, it will be 'My phone is spying on me'.”\
― *Philip K. Dick*
## On Propaganda
If it promotes any "knee-jerk contrarian" viewpoints, a website will only be blocked if they are a majority (>50%) of its content.
As an example, because we have conclusive evidence on the fact that the Earth is round
from the time of Eratosthenes, sites that feature contradictory viewpoints as an opinion majority get blocked.
“True liberty consists only in the power of doing what we ought to will, and in not being constrained to do what we ought not to will.”\
― *Jonathan Edwards*
“Make decency cool again.”\
― *Andrew Marantz*
## On Social Media
### The Zuckerburg Institute of Virology
#### Facebook/Meta
Due to consistent privacy and security concerns, this platform will be blocked.
##### References
- [The Facebook Files](https://www.wsj.com/articles/the-facebook-files-11631713039)
#### Instagram
Owned by Facebook, but does not have majority support on issues that would get the site blocked. Its trackers and such will be blocked.
#### WhatsApp
Owned and heavily monitored by Facebook. Communications may be secure between users, but are monitored by Facebook.
It also has a track record of significant security concerns. All services will be blocked.
##### References
- [NIST Vulnerabilities](https://nvd.nist.gov/vuln/search/results?form_type=Advanced&results_type=overview&query=whatsapp&search_type=all&isCpeNameSearch=true&cvss_version=3)
### Twitter
Though the cause of most internet drama, this platform is also where many cybersecurity community members communicate and collaborate.
While it has compromised privacy and serves personalized ads, the platform itself will remain unblocked to promote cybersecurity communities.
Its trackers and such will be blocked.
### Reddit
While there are harmful subreddits, they are not in the majority. Many have actually been banned, so this will remain unblocked.
Its trackers and such will be blocked.
### Snapchat
Largely supports privacy, even though there are some security concerns. It will remain unblocked.
#### References
- [Data Breach Timeline](https://firewalltimes.com/snapchat-data-breach-timeline)
### Tumblr
Blocked due to being largely marginalized with the potential of any subdomain to breach any pillar.
### TikTok
Blocked due to countless privacy concerns, regardless of its Chinese affiliations.
#### References
- [Times: TikTok Privacy Concerns](https://time.com/6265651/tiktok-security-us/)
- [Forbes: American Surveilance (Referenced by the above article)](https://www.forbes.com/sites/emilybaker-white/2022/10/20/tiktok-bytedance-surveillance-american-user-data/?sh=218b4cc36c2d)
- [Forbes: Spying on Forbes Journalists](https://www.forbes.com/sites/emilybaker-white/2022/12/22/tiktok-tracks-forbes-journalists-bytedance/?sh=645bc0227da5)
## 📋 Attributes
1. Produced in builds specific to each host's syntax.
2. Updates at [1:27 AM UTC](https://www.timeanddate.com/time/zone/timezone/utc).
3. No excess or trailing whitespace.
4. No lingering webscraper garbage.
5. Lines are terminated with `lf`.
6. No blank lines.
7. No comments.
## 🛠️ List Usage
### Hosts File
```bash
mawk '{print "0.0.0.0 " $0}' BLOCK_DOMAIN.txt >>hosts
# mawk '{print ":: " $0}' BLOCK_DOMAIN.txt >>hosts
mawk '{print "0.0.0.0 " $0}' BLOCK_IPV4.txt >>hosts
mawk '{print ":: " $0}' BLOCK_IPV6.txt >>hosts
```
### [Dnsmasq](https://dnsmasq.org/)
Many popular platforms such as OpenWRT, DDWRT, and Pi-hole use Dnsmasq as their choice TCP powerhouse. After inspecting many domain blocklists you'll inevitably run across a list in the `dnsmasq.conf` format. This list doesn't support it because you can use the `addn-hosts` parameter to add hosts in the list.
Target a file that has the hosts in a format similar to the __Hosts File__ format.
If you're using the `RADVD` daemon, prepend any listed hosts with [`::`](https://stackoverflow.com/questions/40189084/what-is-for-localhost-and-0-0-0-0). Otherwise, even if you have IPv6 support set up, prepend hosts with [`0.0.0.0`](https://github.com/StevenBlack/hosts#we-recommend-using-0000-instead-of-127001).
This has been tested across all the mentioned platforms using `dig{6}` on a small sample size and had each host null-routed successfully. [DNSmasq's man page](https://thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html) discusses configuration further, and [DDWRT's ad blocking wiki page](https://wiki.dd-wrt.com/wiki/index.php/Ad_blocking) provides some examples.
#### Amazon EC2 DNS Resolver
Follow [this guide](https://aws.amazon.com/premiumsupport/knowledge-center/dns-resolution-failures-ec2-linux/) to create a DNS server on an Amazon EC2 instance.
### [Pi-hole](https://pi-hole.net/) / [personalDNSfilter](https://github.com/IngoZenz/personaldnsfilter)
Domain list usage is recommended.
### [Unbound](https://nlnetlabs.nl/projects/unbound/about/)
Similar to dnsmasq, but requires more manual configuration. Name any products as a \*.conf file. [Then follow Steffinstanly's instructions on how to apply blocklists](https://medium.com/@steffinstanly/unbound-dns-blocking-3567986a5735).
### [AdGuard](https://adguard.com/en/welcome.html)
Leverage the [`ADGUARD_SOURCES.txt`](https://github.com/T145/black-mirror/blob/master/dist/ADGUARD_SOURCES.txt) list or the domain list.
## 🐋 Docker Usage
### Temporary Container
```bash
docker container run -u 0 -it --rm -h black-mirror ghcr.io/t145/black-mirror
```
### Persistant Container
```bash
docker container run -u 0 -it --name black-mirror -h black-mirror ghcr.io/t145/black-mirror
```
Then to access the container at a later date, run:
```bash
docker start black-mirror
docker exec -u 0 -it black-mirror /bin/bash
```
### Building the Image
```bash
docker build --no-cache --progress=plain -t black-mirror .
```
## 👨💻 Development
### Workspace Setup
Install Docker, [PNPM](https://pnpm.io/installation), and NodeJS. These can usually be installed with a popular package manager.
The specific Node version used in this project is provided in `.node-version`, but any version over `16` should work.
With Docker running in the background, execute `pnpm lint` to debug any changes.
## 🛡️ Privacy Protectors
> Good causes that help secure your digital life.
- *__[Stop using Cloudflare!](https://framagit.org/dCF/deCloudflare/-/tree/master)__*
- Keep up with [Chinese activity](https://www.chinasentry.com/congressional)!
- Use a [pcWRT](https://shop.pcwrt.com/) to easily secure your internet!
- Test your browser's tracking resilience with [CoverYourTracks](https://coveryourtracks.eff.org/)!
- Support [LetsBlockIt](https://letsblock.it/) to consolidate and simplify uBlock filters!
- Explore [PrivacyGuides](https://privacyguides.org/) and [Prism Break](https://prism-break.org/en/all/) to discover services that respect your privacy!
- Use [BypassPaywalls](https://github.com/iamadamdev/bypass-paywalls-chrome#readme) to access restricted and useful information, such as the [WSJ's Facebook Files](https://www.wsj.com/articles/the-facebook-files-11631713039).
- Skip over URL shortener links by using [FastForward](https://github.com/FastForwardTeam/FastForward#readme), which is a better alternative to outright domain blocking.
## 🎶 Notes
### Need for speed
-
-
-
-
### Some List References
-
-
-
-
### Lamers Unwelcome
- [cybercrime-tracker](https://atm.cybercrime-tracker.net/index.php)
- [EMV](https://emv.cybercrime-tracker.net/index.php?x=faq)
- [MalwareBazaar](https://bazaar.abuse.ch/browse/)
- [virusbay](https://beta.virusbay.io/sample/browse)
- [malpedia](https://malpedia.caad.fkie.fraunhofer.de/)
- [manalyzer](https://manalyzer.org/)
- [malshare](https://malshare.com/daily/malshare.current.all.txt)
- [Metasploit CVEs](https://feeds.ecrimelabs.net/data/metasploit-cve)
- [malarchive](https://github.com/raw-data/malarchive)
### Big Data Lists
> Typically used by other blacklist projects as whitelists.
| List Name | URL |
|--------------|---------------------------------------------------------------------------------------------------|
| Alexa | |
| Umbrella | |
| Statvoo | |
| OpenPageRank | |
| Quantcast | |
| Tranco list | |
### IP Block Providers
> Simply provide IP blocks for entire geographic regions.
- [openportstats](http://www.openportstats.com/)
- [ipdeny](https://www.ipdeny.com/ipblocks/)
- [IPverse](http://ipverse.net/)
- [IP2Location](https://lite.ip2location.com/ip2location-lite)
- [Okean China CIDRs](https://www.okean.com/thegoods.html)