https://github.com/spacepatcher/softrace
Lightweight and fast NSRL RDS storage
https://github.com/spacepatcher/softrace
Last synced: 6 months ago
JSON representation
Lightweight and fast NSRL RDS storage
- Host: GitHub
- URL: https://github.com/spacepatcher/softrace
- Owner: spacepatcher
- License: apache-2.0
- Created: 2018-12-27T07:38:18.000Z (over 7 years ago)
- Default Branch: develop
- Last Pushed: 2021-03-30T19:19:32.000Z (over 5 years ago)
- Last Synced: 2024-06-20T00:36:04.049Z (about 2 years ago)
- Language: Go
- Size: 11.7 KB
- Stars: 1
- Watchers: 1
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Softrace
Softrace is a simple Golang application for storing NIST National Software Reference Library Reference Data Set (NSRL RDS). Softrace is based on Bolt database, so it is fast and tiny.
The application is able to process md5 and sha1 hash lookup searches.
Hash lookup example:
```
curl -XGET http://localhost:8001/lookup/6d62c33de5e65af81ae52a529ed2f385
```
Softrace response:
```
{
"sha1":"00012b50bc346b4b3cecf13cdbf734ad03258322",
"md5":"6d62c33de5e65af81ae52a529ed2f385",
"crc32":"a681c4f7",
"file_name":"APPLYStrings.xml",
"file_size":"6820",
"product_name":"Microsoft Office 2000 Standard",
"product_version":"2000",
"application_type":"Operating System",
"os_name":"Windows Media Center 2005",
"os_version":"2005"
}
```
### Create database file
To create the required database file you first need to download archive `Modern RDS Minimal` from Current RDS Hash Sets page https://www.nist.gov/itl/ssd/software-quality-group/national-software-reference-library-nsrl/nsrl-download/current-rds.
You should unpack the downloaded archive into `data/nsrl_rds/rds_modern/` with names of the extracted files:
```
NSRLFile.txt
NSRLMfg.txt
NSRLOS.txt
NSRLProd.txt
```
Build docker image `insert_bolt` for creating Bolt database with NSRL data set:
```
docker build -t insert_bolt -f docker/Dockerfile.insert .
```
To avoid the problem with access rights to Docker volume, set the correct access rights:
```
chown -R 1000:1000 data/
```
Create Bolt database file with `insert_bolt` container:
```
docker run -ti --name insert_bolt -v `pwd`/data:/go/src/github.com/spacepatcher/softrace/data:delegated insert_bolt
```
Creating a database file which is located in `data/bolt/bolt.db` takes several hours (in some cases, with low R/W it will take more than a day, so be patient). Total size of the database file by the end of the process is about 38 gigabytes.
### Start API service
To start API service run:
```
docker-compose up
```
By default the API service is available on `127.0.0.1:8001`.
### Update data
To update the database file data, you need to put new exemplars of files downloaded from Current RDS Hash Sets page in `data/nsrl_rds/rds_modern/`. After that, you should remove the existing database file and generate a new one.