Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/ernw/nmap-parse-output

Converts/manipulates/extracts data from a Nmap scan output.
https://github.com/ernw/nmap-parse-output

Last synced: 2 months ago
JSON representation

Converts/manipulates/extracts data from a Nmap scan output.

Awesome Lists containing this project

README

        

# nmap-parse-output

Converts/manipulates/extracts data from a nmap scan output.

Needs [xsltproc](http://xmlsoft.org/XSLT/xsltproc.html) as dependency.

**Demo**

![Demo](demo.gif)

**Table of Contents**
- [nmap-parse-output](#nmap-parse-output)
- [Examples](#examples)
- [Usage](#usage)
- [Changelog](#changelog)
- [Contribute](#contribute)
- [Adding new Commands](#adding-new-commands)
- [Installation](#installation)
- [Bash Completion](#bash-completion)
- [ZSH Completion](#zsh-completion)
- [Distro package](#distro-prackage)

## Examples

Write HTML output to scan.html:

$ ./nmap-parse-output scan.xml html > scan.html

Generates a list of all HTTP(s) ports:

$ ./nmap-parse-output scan.xml http-ports
http://192.168.0.1:8081
https://192.168.0.1:8443

List all names of detected services and get a list of hosts with the port for the service http-proxy:

$ ./nmap-parse-output scan.xml service-names
http
https
http-proxy
ms-wbt-server
smtp
$ ./nmap-parse-output scan.xml service http-proxy
192.168.0.24:8085
192.168.0.25:9000

Exclude some hosts from a scan and generate a list of ports:

$ ./nmap-parse-output scan.xml exclude '192.168.1.1,192.168.1.20' | nmap-parse-output - ports
22,80,443,8080

Filter `scan-all.xml` to include only hosts scanned in `scan-subnet.xml` and write the output to `filtered-scan.xml`:

$ ./nmap-parse-output scan-all.xml include $(./nmap-parse-output.sh scan-subnet.xml hosts | tr "\n" ",") > filtered-scan.xml

Generate a subreport with results only from the networks 10.0.1.0/24 and 10.80.0.0/24 from `big-scan-result.xml`:

$ ./nmap-parse-output big-scan-result.xml \
include $(nmap -Pn -n -sL 10.0.1.0/24 10.80.0.0/24 -oX - | nmap-parse-output - all-hosts | tr "\n" ',') \
> networks-10.0.1.0-24-and-10.80.0.0-24.xml

Add comments to a scan, mark specific ports red, and generate a HTML report with the annotations:

$ ./nmap-parse-output scan.xml comment-ports '8080,10.0.20.4:443' 'this port should be filtered'
| ./nmap-parse-output - mark-ports '8080,10.0.20.4:443' red
| ./nmap-parse-output - comment-hosts '10.0.20.1' 'look further into this host'
| ./nmap-parse-output - html > test.html

Remove all ports found in `scan-before.xml` from `scan-after.xml` and write the output to `filtered-scan.xml`:

$ ./nmap-parse-output scan-after.xml exclude-ports $(./nmap-parse-output.sh scan-before.xml host-ports | tr "\n" ",") > filtered-scan.xml

Group hosts by open TCP ports:

$ ./nmap-parse-output scan.xml group-by-ports
- Open TCP ports 22,443 on:
- 192.168.1.73 (web.local)
- 192.168.1.74 (web2.local)
- Open TCP ports 8080 on:
- 192.168.1.135
- No open TCP ports on:
- 192.168.1.71 (db.local)
- 192.168.1.81

Rerun TCP scan for all alive hosts but only scan open ports found in the previous scan:

$ for host in ./nmap-parse-output scan.xml hosts; do nmap -sSVC -p $(./nmap-parse-output scan.xml include $host | nmap-parse-output - ports) -oA $host.xml -vv -Pn; done

## Usage

Usage: ./nmap-parse-output [options]... [command-parameters]...

Converts/manipulates/extracts data from nmap scan XML output.

Options:
-u, --unfinished-scan try to read an unfinished scan output

Extract Data Commands:
all-hosts
Generates a line break separated list of all hosts. Can be used to perform an additional scan on this hosts.
Can be useful to generate a list of IPs for masscan with nmap (masscan has a more limited support for IP lists):
nmap -Pn -n -sL -iL input.lst -oX all-ips.xml; nmap-parse-output all-ips.xml all-hosts

banner [service-name]
Extracts a list of all ports with a specific service (e.g. http, ms-wbt-server, smtp) in host:port format.
Note: This command is intended for the masscan XML output only.

blocked-ports
Extracts all ports in host:port format, which either admin-prohibited or tcpwrapped.

group-by-ports
Groups hosts by open TCP ports and generates a human-readable list in the markdown format.

group-by-product
Groups open ports by product name and generates a human-readable list in the markdown format. The command ignores ports with empty product name.

group-by-service
Groups open ports by service name and generates a human-readable list in the markdown format.

host-ports-protocol
Extracts a list of all *open* ports in host:port format and marks the protocol type (tcp, udp)

host-ports
Extracts a list of all *open* ports in host:port format.

hosts-to-port [port]
Extracts a list of all hosts that have the given port open in 'host (hostname)' format.

hosts
Generates a line break separated list of all hosts with open ports. Can be used to perform an additional scan on this hosts.

http-info
Extracts a list of extra information about the HTTP ports from a script scan in the following format:
host:port service-name service-tunnel product http-server-header http-title redirect_url

http-ports
Generates a line separated list of all HTTP(s) ports. Script scan output is recommended.
Currently, the following services are detected as HTTP: http, https, http-alt, https-alt, http-proxy, sip, rtsp, soap, vnc-http, caldav (potentially incomplete) and to get a more reliable result all ports which which have output in the http-methods script.
On ports where HTTPs and HTTP is responding, the HTTPs port is preferred.

http-title
Extracts a list of HTTP HTML titles in the following format:
host:port HTML title

nmap-cmdline
Shows the parameters passed to nmap of the runned scan

port-info [port]
Extracts a list of extra information about the given port in the following format:
port;service-name;http-server-header;http title

ports-reachable
Generates a comma-separated list of all reachable ports (open and closed, unfiltered). Can be used to verify if ports reachable from another host or generate port lists for specific environments.

ports
Generates a comma-separated list of all open ports. Can be used to verify if open ports reachable from another host or generate port lists for specific environments. Closed/filtered ports are not shown in the list.

product
Extracts all detected product names.

search-product
Search in product names (case sensitive).

service-names
Extracts all detected service names (on open ports).

service [service-name]
Extracts a list of all *open* ports with a specific service (e.g. http, ms-wbt-server, smtp) in host:port format.

show-comments
Shows comments added by the comment-* commands.

ssl-common-name
Extracts a list of TLS/SSL ports with the commonName and Subject Alternative Name in the following format:
host:port commonName X509v3 Subject Alternative Name

tls-ports
Extracts a list of all TLS ports in host:port format. Works only reliable after a script scan. Can be used to do a testssl.sh scan.
Example testssl.sh command (generates a text and HTML report for each host):
for f in `cat ~/ssl-hosts.txt`; do ./testssl.sh --logfile ~/testssl.sh-results/$f.log --htmlfile ~/testssl.sh-results/$f.html $f; done


Manipulate Scan Commands:
comment-hosts [hosts] [comment]
Comments a list of hosts in scan result. Expects a comma-separated list as input. The comment will be displayed in the HTML report.
Example:
nmap-parse-output scan.xml comment-hosts '10.0.0.1,192.168.10.1' 'allowed services' | nmap-parse-output - html > report.html
You can comment hosts from another scan, too:
nmap-parse-output scan.xml comment-hosts $(./nmap-parse-output.sh scan-subnet.xml hosts | tr "\n" ",") 'this host was scanned in subnet, too.'

comment-ports [ports] [comment]
Comments a list of ports or hosts with port (in address:port format) in scan result. Expects a comma-separated list as input. The comment will be displayed in the HTML report.
Example:
nmap-parse-output scan.xml comment-ports '80,10.0.0.1:8080' 'allowed services' | nmap-parse-output - html > report.html
You can comment services, too:
nmap-parse-output scan.xml comment-ports $(./nmap-parse-output.sh scan.xml service http | tr "\n" ",") 'this is a http port'

exclude-ports [ports]
Excludes a list of ports or ports of a specific host (in address:port format) from a scan result. Expects a comma-separated list as input.
You can pipe the output, for instance:
nmap-parse-output scan.xml exclude '80,443,192.168.0.2:80' | nmap-parse-output - service-names

exclude [hosts]
Excludes a list of hosts from scan result by its IP address. Expects a comma-separated list as input.
You can pipe the output, for instance:
nmap-parse-output scan.xml exclude '192.168.1.1,192.168.1.20' | nmap-parse-output - service-names

include-ports [ports]
Filter a scan by a list of ports or ports of a specific host (in address:port format) so that only the specified ports are in the output. Expects a comma-separated list as input.
You can pipe the output, for instance:
nmap-parse-output scan.xml include-ports '80,443,192.168.0.2:8080' | nmap-parse-output - http-title

include [hosts]
Filter a scan by a list of hosts so that only the specified hosts are in the output.
Filter a list of hosts from scan result by its IP address. Expects a comma-separated list as input.
You can pipe the output, for instance:
nmap-parse-output scan.xml include '192.168.1.1,192.168.1.20' | nmap-parse-output - service-names

mark-ports [ports] [color]
Marks a list of ports or hosts with port (in address:port format) with the given color in scan result. Expects a comma-separated list as input. The comment will be displayed in the HTML report.
Example:
nmap-parse-output scan.xml mark-ports '80,10.0.0.1:8080' red | nmap-parse-output - html > report.html

reachable
Removes all hosts where all ports a filtered. Can be used to generate a smaller HTML report.
Example usage to generate HTML report:
nmap-parse-output scan.xml reachable | nmap-parse-output - html > scan.html


Convert Scan Commands:
html-bootstrap
Converts the XML output into a fancy HTML report based on Bootstrap.
Note: This HTML report requests JS/CSS libs from CDNs. However, the generated file uses the no-referrer meta tag and subresource integrity to protect the confidentiality.

html
Converts a XML output into a HTML report

to-json
Converts a nmap scan output to JSON


Misc Commands:

[v1.5.1]

## Changelog

* v1.5.1
* More reliable SSL/TLS detection
* More reliable HTTP(s) port detection
* Added group-by-service command
* Added group-by-product command
* Added http-info command
* Added http-server-header to port-info command
* Added hostname to group-by-ports command
* v1.4.6
* Added search-product command
* Added group-by-ports command
* Added examples for generating subreports, group-by-ports command, and rerun scans
* v1.4.5
* Show only service-names of open ports
* Splitted ports command in: ports and ports-reachable
* Added show-comments command
* v1.4.4
* Fixed bug in [bash completion](_nmap-parse-output) when installed system-wide
* Improved documentation
* v1.4.3
* Fixed bug in [include-ports command](nmap-parse-output-xslt/include-ports.xslt)
* v1.4.2
* Added [nmap-bootstrap-xsl](https://github.com/honze-net/nmap-bootstrap-xsl) as [html-bootstrap command](nmap-parse-output-xslt/html-bootstrap.xslt)
* Added [nmap-cmdline command](nmap-parse-output-xslt/nmap-cmdline.xslt)
* Added [host-ports-protocol command](nmap-parse-output-xslt/host-ports-protocol.xslt)
* v1.4.1
* Improved error handling
* Bugfix in ports command
* v1.4.0
* Support for unfinished scans
* Commands are categorized as convert, manipulate, extract and misc now
* v1.3.0
* First public release

## Contribute

### Adding new Commands

Commands are written as [XSLT](https://en.wikipedia.org/wiki/XSLT). See [nmap-parse-output-xslt/](nmap-parse-output-xslt/) if you want to add new commands. A good way is mostly copying an existing script that does something similar.

The documentation printed in the help page can be written with the ```` tag (XML namespace: http://xmlns.sven.to/npo). A command can have one of the following categories: convert, manipulate or extract. You can set it with the ```` tag. It is not necessary to set a category, uncategorized commands are will be shown as a misc command in the help page. Commands with an invalid category will not be shown on the help page.

Parameters will be passed as variables named ``$param1``, ``$param2`` and so on. An post processing command can be added with the ```` tag.

Example XSLT file:





extract
sort | uniq








,




More information about XSLT and writing new commands can be found here:
- http://xmlsoft.org/XSLT/
- http://www.w3.org/TR/xslt
- http://www.exslt.org/
- http://xmlsoft.org/XSLT/xsltproc.html

## Installation

Requierements:
* bash
* [xsltproc](http://xmlsoft.org/XSLT/xsltproc.html)

Check out the repository and run it:

git clone https://github.com/ernw/nmap-parse-output.git
cd nmap-parse-output
./nmap-parse-output

You can create a link to the ``nmap-parse-output`` script in your local bin directory or add the directory into your path if you want to execute it directly. Add the following line into your ``~/.bash_profile`` or your ``~/.zshrc`` file:

export PATH="$PATH:$PATH_TO_YOUR_NMAP_PARSE_OUTPUT_DIR"

### Bash Completion

Bash completion can be enabled by adding the following line to your `~/.bash_profile` or `.bashrc`:

source ~/path/to/nmap-parse-output/_nmap-parse-output

### ZSH Completion

ZSH completion can be enabled by adding the following line to your `~/.zshrc`:

autoload bashcompinit && bashcompinit && source ~/path/to/nmap-parse-output/_nmap-parse-output

### Distro Packages

[![Packaging status](https://repology.org/badge/vertical-allrepos/nmap-parse-output.svg)](https://repology.org/project/nmap-parse-output/versions)