https://github.com/zkemail/dkim-registry
DNS archival services do not archive DKIM, so this is our archival dkim registry. This repo runs a DNS scaper on a cron job and updates a local postgresql database, which is queryable by a simple website.
https://github.com/zkemail/dkim-registry
Last synced: over 1 year ago
JSON representation
DNS archival services do not archive DKIM, so this is our archival dkim registry. This repo runs a DNS scaper on a cron job and updates a local postgresql database, which is queryable by a simple website.
- Host: GitHub
- URL: https://github.com/zkemail/dkim-registry
- Owner: zkemail
- License: mit
- Created: 2023-06-26T22:16:41.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2023-06-26T22:19:38.000Z (almost 3 years ago)
- Last Synced: 2024-04-14T02:13:12.238Z (about 2 years ago)
- Language: Python
- Homepage:
- Size: 21.4 MB
- Stars: 1
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## To use with PostgreSQL (debian-based OS), run these commands first to prep the database:
Set up PostgreSQL
```
$ sudo apt update
$ sudo apt install postgresql postgresql-contrib
$ sudo systemctl start postgresql.service
$ sudo -u postgres psql
postgres=# CREATE USER dkim password 'changeme'; # please use a better password
CREATE ROLE
postgres=#
$ sudo -u postgres createdb dkim
```
To set up python environment, run this:
```
$ python3 -m venv venv
$ source ./venv/bin/activate
$ pip install -r requirements.txt
```
To begin (or resume) collection of data, run this:
```
(venv) $ ./main.py --collect # collect the data (may take some time)
```
To browse the data, run this command, and view it from browser
```
(venv) $ ./main.py --webpage # launches web server on port 8001
```
You can run the server in one process while leaving the collector running
in a separate process.
To add additional sites to serve beyond the ones from alexa, add them
to the file "additional_sites.txt".
To add more selectors to query on, add them to the `selectors.txt` file. Each
additional selector added will double the total number of queries made, so
the number of queries grows geometrically. But it will be faster in subsequent runs unless you specify the `rescan` flag for particular selectors.
## Knobs (see `main.py`)
Depending on resources available on the host running the collection, you
may choose to increase or decrease the number of simultaneous queries by
changing `chunk_size`.
## Webpage
Browse to here to see the list of domains for which DKIM(s) were found in DNS:
http://localhost:8001/
## API
To query all records of a specific domain, e.g., arca.live:
http://localhost:8001/api/dkim/?domain=arca.live
To query ALL the data (might be slow):
http://localhost:8001/api/dkim/