https://github.com/thescriptguy/quad9-threat-intel-api
Quad9 Threat Intel API (new and improved!)
https://github.com/thescriptguy/quad9-threat-intel-api
Last synced: 21 days ago
JSON representation
Quad9 Threat Intel API (new and improved!)
- Host: GitHub
- URL: https://github.com/thescriptguy/quad9-threat-intel-api
- Owner: TheScriptGuy
- Created: 2021-02-18T06:57:48.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2021-04-18T21:42:41.000Z (about 5 years ago)
- Last Synced: 2025-01-12T01:13:28.088Z (over 1 year ago)
- Language: Python
- Homepage:
- Size: 11.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# How to operate this heavy machinery
First and foremost an honorary mention of the original Quad9 threat-intel-api https://github.com/Quad9DNS/threat-intel-api
I used their python script as a starting point and built on from there.
The Docker container things I worked on myself.
I'm open to positive feedback/input.
Now onwards to the instructions! Avante!
## First create the image from Dockerfile
```bash
docker build -t /quad9-threat-intel: -f Dockerfile .
```
## Initial setup
Create the directory where you want the data to be downloaded to
```bash
mkdir /quad9-intel-data
```
Assuming scripts directory is in /quad9-threat-intel-api
## Create the container
```bash
docker create -v /quad9-threat-intel-api/scripts/:/scripts/ \
-v /quad9-intel-data/:/quad9-intel-data/ \
/quad9-threat-intel: /scripts/start.sh
```
## Update API Environment variables
Edit the start.sh script and insert your API token and number of connections needing to be made to Quad9
```bash
QUAD9_API_TOKEN=
QUAD9_CONNECTIONS=
```
## Info to know about the python script
The python script writes the downloaded content into the /quad9-intel-data directory in the filename format of
quad9.Year-Month-DayHHour.json
Example file names:
* quad9.2021-01-01H05.json
* quad9.2021-01-01H22.json
* quad9.2021-01-02H15.json
The python script outputs content to files in chunks of 100,000 entries. If the script crashes, or the container stops, the variables in memory are output to disk according to the format above.
## To start the container
```bash
docker start
```
## To stop the container
```bash
docker stop
```
### Potential improvements on the horizon...who knows? ¯\_(ツ)_/¯
* Multiprocessing in addition to multithreading (is this worthwhile given that multiple containers can be spun up to create "multiprocessing"?)
* other?