https://github.com/efforg/badger-swarm
Runs distributed Badger Sett scans on Digital Ocean.
https://github.com/efforg/badger-swarm
Last synced: about 1 year ago
JSON representation
Runs distributed Badger Sett scans on Digital Ocean.
- Host: GitHub
- URL: https://github.com/efforg/badger-swarm
- Owner: EFForg
- License: mit
- Created: 2022-01-06T20:18:48.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2025-01-08T17:02:31.000Z (over 1 year ago)
- Last Synced: 2025-04-19T11:09:14.663Z (about 1 year ago)
- Language: Shell
- Homepage: https://www.eff.org/deeplinks/2023/10/privacy-badger-learns-block-ever-more-trackers
- Size: 153 KB
- Stars: 25
- Watchers: 6
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Badger Swarm
Runs distributed [Badger Sett](https://github.com/EFForg/badger-sett) scans on Digital Ocean. Yes, a group of badgers is called a _cete_, but "swarm" just sounds better.
Badger Swarm converts a Badger Sett scan of X sites into N Badger Sett scans of X/N sites. This makes medium scans complete as quickly as small scans, and large scans complete in a reasonable amount of time.
For more information, visit our [Introducing Badger Swarm: New Project Helps Privacy Badger Block Ever More Trackers](https://www.eff.org/deeplinks/2023/10/privacy-badger-learns-block-ever-more-trackers) blog post.
## Architecture
See [ARCHITECTURE.md](ARCHITECTURE.md).
## Setup
1. Check out this repository
2. [Install `doctl`](https://github.com/digitalocean/doctl#installing-doctl)
3. [Authenticate `doctl`](https://github.com/digitalocean/doctl#authenticating-with-digitalocean) with DigitalOcean
4. Copy `settings.ini.sample` to `settings.ini`
5. Review the settings. At minimum, specify your Digital Ocean SSH key (see `doctl compute ssh-key`). For Droplet sizes and hourly prices, see `doctl compute size list`.
6. To automatically merge results on completion, check out Badger Sett and [Privacy Badger](https://github.com/EFForg/privacybadger) (at the same directory level as this repository) and then [set up and activate a virtual environment](https://snarky.ca/a-quick-and-dirty-guide-on-how-to-install-packages-for-python/) for Badger Sett.
7. Run `./main.sh` to initiate a new run.
Once you are told the run is resumable, you can stop the script with Ctrl-C and then later resume the in-progress run with `./main.sh -r`.
## Helpful Bash scripting links
- https://google.github.io/styleguide/shellguide.html
- https://mywiki.wooledge.org/
- https://tldp.org/LDP/abs/html/index.html
- https://www.shellcheck.net/