Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/mitre/heimdall_tools

DEPRECATED: A set of utilities for converting and working with compliance data for viewing in the heimdall applications
https://github.com/mitre/heimdall_tools

heimdall heimdall-lite inspec mitre-corporation mitre-heimdall mitre-heimdall-lite

Last synced: 6 months ago
JSON representation

DEPRECATED: A set of utilities for converting and working with compliance data for viewing in the heimdall applications

Awesome Lists containing this project

README 

          
# Heimdall Tools has been deprecated. See the new [SAF CLI](https://github.com/mitre/saf)



# Heimdall Tools



![Overall Status](https://github.com/mitre/heimdall_tools/workflows/heimdall_tools/badge.svg)

![Heimdall Tools Build](https://github.com/mitre/heimdall_tools/workflows/Build%20and%20release%20gem/badge.svg)



HeimdallTools supplies several methods to convert output from various tools to "Heimdall Data Format"(HDF) format to be viewable in Heimdall. The current converters are:



1.  [**aws_config_mapper**](#aws_config_mapper) - assess, audit, and evaluate AWS resources

1.  [**burpsuite_mapper**](#burpsuite_mapper) - commercial dynamic analysis tool

1.  [**dbprotect_mapper**](#dbprotect_mapper) - database vulnerability scanner

1.  [**fortify_mapper**](#fortify_mapper) - commercial static code analysis tool

1.  [**jfrog_xray_mapper**](#jfrog_xray_mapper) - package vulnerability scanner

1.  [**nessus_mapper**](#nessus_mapper) - commercial security scanner (supports compliance and vulnerability scans from Tenable.sc and Tenable.io)

1.  [**netsparker_mapper**](#netsparker_mapper) - web application security scanner

1.  [**nikto_mapper**](#nikto_mapper) - open-source web server scanner

1.  [**sarif_mapper**](#sarif_mapper) - static analysis results interchange format

1. [**scoutsuite_mapper**](#scoutsuite_mapper) - multi-cloud security auditing tool

1. [**snyk_mapper**](#snyk_mapper) - commercial package vulnerability scanner

1. [**sonarqube_mapper**](#sonarqube_mapper) - open-source static code analysis tool

1. [**xccdf_results_mapper**](#xccdf_results_mapper) - extensible configuration checklist description results format

1. [**scc_mapper**](#xccdf_results_mapper) - scap compliance checker format

1. [**zap_mapper**](#zap_mapper) - OWASP ZAP - open-source dynamic code analysis tool



## Want to recommend a mapper for another tool? Please use these steps:



1.  Create an [issue](https://github.com/mitre/heimdall_tools/issues/new), and email saf@groups.mitre.org citing the issue link so we can help

2.  Provide a sample output, preferably the most detailed the tool can provide, and also preferably in a machine-readable format, such as xml, json, or csv - whichever is natively available. If it is sensitive we'll work that in #3. (If it's an API only, we'll also just talk about it in #3)

3.  Let's arrange a time to take a close look at the data it provides to get an idea of all it has to offer. We'll suggest an initial mapping of the HDF core elements.

4.  Note: if the tool doesn't provide a NIST SP 800-53 reference, we've worked on mappings to other references such as CWE or OWASP Top 10.

5.  If the tool doesn't provide something for #4, or another core element such as impact, we'll help you identify a custom mapping approach.

6.  We'll help you decide how to preserve any other information (non-core elements) the tool provides to ensure that all of the original tool's intent comes through for the user when the data is viewed in Heimdall.

7.  Finally, We'll provide final peer review and support merging your pull request.

    We appreciate your contributions, but we're here to help!



## How to Install Heimdall Tools:



Ruby 2.4 or higher (check using "ruby -v")



If installation of Ruby is required, perform these steps:



## Linux Installation of Ruby



### Installation on RHEL-based systems



To install Ruby using RVM (Ruby Version Manager):



`sudo yum install curl gpg gcc gcc-c++ make patch autoconf automake bison libffi-devel libtool patch readline-devel sqlite-devel zlib-devel openssl-devel`



`sudo gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB`



`curl -sSL https://get.rvm.io | bash -s stable`



`source ~/.rvm/scripts/rvm`



Finally, install the latest version of Ruby (stable), currently 2.6.x:



`rvm install 2.6`



`rvm use 2.6 --default`



Verify the installed version number:



`ruby -v`



`ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]`



### Installation on Ubuntu-based systems



# Installation of Heimdall Tools:



`gem install heimdall_tools`



## Command line Usage



On the Command Line, `heimdall_tools help` will print a listing of all the command with a short description.

For detailed help on any command, run `heimdall_tools help [COMMAND]`. Help can also be called with the `-h, --help` flags after any command, like `heimdall_tools fortify_mapper -h`.



For Docker usage, replace the `heimdall_tools` command with the correct Docker command below for your operating system:



-   **On Linux and Mac:** `docker run -it -v$(pwd):/share mitre/heimdall_tools`

-   **On Windows CMD:** `docker run -it -v%cd%:/share mitre/heimdall_tools`



Note that all of the above Docker commands will mount your current directory on the Docker container. Ensure that you have navigated to the directory you intend to convert files in before executing the command.



## aws_config_mapper



aws_config_mapper pulls Ruby AWS SDK data to translate AWS Config Rule results into HDF format json to be viewable in Heimdall



### AWS Config Rule Mapping:



  The mapping of AWS Config Rules to 800-53 Controls was sourced from [this link](https://docs.aws.amazon.com/config/latest/developerguide/operational-best-practices-for-nist-800-53_rev_4.html).



### Authentication with AWS:



  [Developer Guide for configuring Ruby AWS SDK for authentication](https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html)



    USAGE: heimdall_tools aws_config_mapper [OPTIONS] -o

    

    FLAGS:

        -o --output        : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example: heimdall_tools aws_config_mapper -o aws_config_results_hdf.json



## burpsuite_mapper



burpsuite_mapper translates an BurpSuite Pro exported XML results file into HDF format json to be viewable in Heimdall



    USAGE: heimdall_tools burpsuite_mapper [OPTIONS] -x  -o

    

    FLAGS:

        -x                : path to BurpSuitePro exported XML results file.

        -o --output        : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example: heimdall_tools burpsuite_mapper -x burpsuite_results.xml -o scan_results.json



## dbprotect_mapper



dbprotect_mapper translates DBProtect report in `Check Results Details` format XML to HDF format JSON be viewed on Heimdall.



    USAGE: heimdall_tools dbprotect_mapper [OPTIONS] -x  -o

    

    FLAGS:

        -x            : path to DBProtect report XML file.

        -o --output        : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example: heimdall_tools dbprotect_mapper -x check_results_details_report.xml -o db_protect_hdf.json



## fortify_mapper



fortify_mapper translates an Fortify results FVDL file into HDF format json to be viewable in Heimdall



    USAGE: heimdall_tools fortify_mapper [OPTIONS] -f  -o

    

    FLAGS:

    	-f --fvdl          : path to Fortify Scan FVDL file.

    	-o --output        : path to output scan-results json.

    	-V --verbose                     : verbose run [optional].

    

    example: heimdall_tools fortify_mapper -f audit.fvdl -o scan_results.json



## jfrog_xray_mapper



jfrog_xray_mapper translates an JFrog Xray results JSON file into HDF format JSON to be viewable in Heimdall



    USAGE: heimdall_tools jfrog_xray_mapper [OPTIONS] -j  -o

    

    FLAGS:

        -j            : path to xray results JSON file.

        -o --output        : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example: heimdall_tools jfrog_xray_mapper -j xray_results.json -o xray_results_hdf.json



## nessus_mapper



nessus_mapper translates a Nessus-exported XML results file into HDF format json to be viewable in Heimdall.

Supports compliance and vulnerability scans from Tenable.sc and Tenable.io.



Note: A separate HDF JSON file is generated for each host reported in the Nessus Report.



    USAGE: heimdall_tools nessus_mapper [OPTIONS] -x  -o

    

    FLAGS:

        -x           : path to Nessus-exported XML results file.

        -o --output_prefix       : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example: heimdall_tools nessus_mapper -x nessus-results.xml -o test-env



## netsparker_mapper



netsparker_mapper translates an Netsparker XML results file into HDF format JSON to be viewable in Heimdall.



The current iteration only works with Netsparker Enterprise Vulnerabilities Scan.



    USAGE: heimdall_tools netsparker_mapper [OPTIONS] -x  -o

    

    FLAGS:

        -x       : path to netsparker results XML file.

        -o --output        : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example: heimdall_tools netsparker_mapper -x netsparker_results.xml -o netsparker_hdf.json



## nikto_mapper



nikto_mapper translates an Nikto results JSON file into HDF format JSON to be viewable in Heimdall



Note: Current this mapper only support single target Nikto Scans.



    USAGE: heimdall_tools nikto_mapper [OPTIONS] -x  -o

    

    FLAGS:

        -j           : path to Nikto results JSON file.

        -o --output_prefix       : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example: heimdall_tools nikto_mapper -j nikto_results.json -o nikto_results.json



## sarif_mapper



sarif_mapper translates a SARIF JSON file into HDF format JSON to be viewable in Heimdall



    USAGE: heimdall_tools sarif_mapper [OPTIONS] -j  -o

    

    FLAGS:

        -j           : path to SARIF results JSON file.

        -o --output_prefix       : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example: heimdall_tools sarif_mapper -j sarif_results.json -o sarif_results_hdf.json



## scoutsuite_mapper



scoutsuite_mapper translates Scout Suite results from Javascript to HDF-formatted JSON so as to be viewable on Heimdall



Note: Currently this mapper only supports AWS.



    USAGE: heimdall_tools scoutsuite_mapper -i  -o

    

    FLAGS:

        -i --input -j --javascript  : path to Scout Suite results Javascript file.

        -o --output                 : path to output scan-results json.

    

    example: heimdall_tools scoutsuite_mapper -i scoutsuite_results.js -o scoutsuite_hdf.json



## snyk_mapper



snyk_mapper translates an Snyk results JSON file into HDF format json to be viewable in Heimdall



Note: A separate HDF JSON is generated for each project reported in the Snyk Report.



    USAGE: heimdall_tools snyk_mapper [OPTIONS] -x  -o

    

    FLAGS:

        -j           : path to Snyk results JSON file.

        -o --output_prefix       : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example: heimdall_tools snyk_mapper -j snyk_results.json -o output-file-prefix



## sonarqube_mapper



sonarqube_mapper pulls SonarQube results, for the specified project, from the API and outputs in HDF format Json to be viewed on Heimdall



    USAGE: heimdall_tools sonarqube_mapper [OPTIONS] -n  -u  -o

    

    FLAGS:

        -n --name          : Project Key of the project in SonarQube

        -u --api_url            : url of the SonarQube Server API. Typically ends with /api.

        --auth               : username:password or token [optional].

        -o --output        : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example:

    

    heimdall_tools sonarqube_mapper -n sonar_project_key -u http://sonar:9000/api -o scan_results.json

    

    heimdall_tools sonarqube_mapper -n sonar_project_key -u http://sonar:9000/api --auth admin:admin -o scan_results.json



## xccdf_results_mapper



Note: SCC outputs scan results in XCCDF-Results format.



xccdf_results_mapper translates an XCCDF_Results XML scan into HDF format json to be viewable in Heimdall.



    USAGE: heimdall_tools xccdf_results_mapper [OPTIONS] -x  -o

    

    FLAGS:

        -x            : path to XCCDF-Results XML file.

        -o --output        : path to output scan-results json.

    

    example: heimdall_tools xccdf_results_mapper -x xccdf_results.xml -o scan_results.json



## zap_mapper



zap_mapper translates OWASP ZAP results Json to HDF format Json be viewed on Heimdall



    USAGE: heimdall_tools zap_mapper [OPTIONS] -j  -n  -o

    

    FLAGS:

        -j --json              : path to OWASP ZAP results JSON file.

        -n --name             : URL of the site being evaluated.

        -o --output        : path to output scan-results json.

        -V --verbose                     : verbose run [optional].

    

    example: heimdall_tools zap_mapper -j zap_results.json -n site_name -o scan_results.json



## version



Prints out the gem version



    USAGE: heimdall_tools version



# Development



## Submitting a PR



### A complete PR should include 7 core elements:



-   A signed PR ( aka `git commit -a -s` )

-   Code for the new functionality

-   Updates to the CLI

-   New unit tests for the functionality

-   Updates to the docs and examples in `README.md` and `./docs/*`

-   (if needed) Example / Template files ( `metadata.yml`,`example.yml`, etc )

    -   Scripts / Scaffolding code for the Example / Template files ( `generate_map` is an example )

-   Example Output of the new functionality if it produces an artifact



### Overview of our PR process



1.  open an issue on the main inspec_tools website noting the issues your PR will address

2.  fork the repo

3.  checkout your repo

4.  cd to the repo

5.  git co -b ``

6.  bundle install

7.  `hack as you will`

8.  test via rake

9.  ensure unit tests still function and add unit tests for your new feature

10. add new docs to the `README.md` and to `./docs/examples`

11. update the CLI as needed and add in `usage` example

12. (if needed) create and document any example or templates

13. (if needed) create any supporing scripts

14. (opt) gem build inspec_tools.gemspec

15. (opt) gem install inspec_tools

16. (opt) test via the installed gem

17. git commit -a -s ``

18. Open a PRs aginst the MITRE inspec_tools repo



# Publishing a Release



If you are a maintainer, it is very easy to cut a release of this gem:



1.  Click on "Releases" and there should be a draft pending.

2.  Make sure the Tag version and Release title match!

3.  Add any additional notes can be added in the Description box.

4.  Click "Publish release".

5.  Release notes will be posted and a new gem will be pushed to Rubygems & Github Packages with the version you specified on step 2.



# Testing



This gem was developed using the [CLI Template](https://github.com/tongueroo/cli-template), a generator tool that builds a starter CLI project.



There are a set of unit tests. Run `rake test` to run the tests.



To release a new version, update the version number in `version.rb` according to the [Semantic Versioning Policy](https://semver.org/). Then, run `bundle exec rake release` which will create a git tag for the specified version, push git commits and tags, and push the `.gem` file to [github.com](https://github.com/mitre/heimdall_tools).



# License and Author



### Authors



-   Author:: Rony Xavier [rx294](https://github.com/rx294)

-   Author:: Dan Mirsky [mirskiy](https://github.com/mirskiy)



### NOTICE



© 2018 The MITRE Corporation.



Approved for Public Release; Distribution Unlimited. Case Number 18-3678.



### NOTICE



MITRE hereby grants express written permission to use, reproduce, distribute, modify, and otherwise leverage this software to the extent permitted by the licensed terms provided in the LICENSE.md file included with this project.



### NOTICE



This software was produced for the U. S. Government under Contract Number HHSM-500-2012-00008I, and is subject to Federal Acquisition Regulation Clause 52.227-14, Rights in Data-General.



No other use other than that granted to the U. S. Government, or to those acting on behalf of the U. S. Government under that Clause is authorized without the express written permission of The MITRE Corporation.



For further information, please contact The MITRE Corporation, Contracts Management Office, 7515 Colshire Drive, McLean, VA 22102-7539, (703) 983-6000.