{"id":13845747,"url":"https://github.com/gh0x0st/pythonizing_nmap","last_synced_at":"2025-07-10T19:32:25.831Z","repository":{"id":62624210,"uuid":"366840788","full_name":"gh0x0st/pythonizing_nmap","owner":"gh0x0st","description":"A detailed guide showing you different ways you can incorporate Python into your workflows around Nmap.","archived":false,"fork":false,"pushed_at":"2021-11-19T15:37:33.000Z","size":1609,"stargazers_count":150,"open_issues_count":0,"forks_count":42,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-05T16:51:07.628Z","etag":null,"topics":["enumeration","nmap","nmap-scripts","nmap-xml","offensive-security","oscp","oscp-journey","oscp-prep","penetration-testing","python3","security-tools","sqlite3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gh0x0st.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-12T20:08:21.000Z","updated_at":"2024-10-18T06:58:37.000Z","dependencies_parsed_at":"2022-11-03T19:47:15.375Z","dependency_job_id":null,"html_url":"https://github.com/gh0x0st/pythonizing_nmap","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/gh0x0st/pythonizing_nmap","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gh0x0st%2Fpythonizing_nmap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gh0x0st%2Fpythonizing_nmap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gh0x0st%2Fpythonizing_nmap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gh0x0st%2Fpythonizing_nmap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gh0x0st","download_url":"https://codeload.github.com/gh0x0st/pythonizing_nmap/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gh0x0st%2Fpythonizing_nmap/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264639922,"owners_count":23642319,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["enumeration","nmap","nmap-scripts","nmap-xml","offensive-security","oscp","oscp-journey","oscp-prep","penetration-testing","python3","security-tools","sqlite3"],"created_at":"2024-08-04T17:03:34.982Z","updated_at":"2025-07-10T19:32:25.431Z","avatar_url":"https://github.com/gh0x0st.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Pythonizing Nmap\nWhen I started to get into this field, I tried my best to stick to manual workflows to get the lay of the land. As time passed by and I gained more experience the more I found that I needed to find better ways to be more efficient with my time. One place where I spent too much time was performing my initial enumeration with Nmap on larger scale assessments, staying organized as well as writing reports with the information I have collected. \n\nI believe that automation is crucial for some aspects of a penetration test and Python is a tool to help us facilitate this. Allow me to show you various ways you can enhance your workflows by incorporating Python into your Nmap processes.\n\n## Boots on The Ground\n\n1. Opening Remarks on Nmap Wrappers\n2. Using Subprocess with Nmap\n3. Parsing Nmap XML\n4. Importing Nmap XML into SQLite Databases\n5. Python Nmap Wrapper Scripts\n6. Generating Reports from Nmap XML\n7. Wrapping Up\n\n## Opening Remarks on Nmap Wrappers\n\nKeep in mind that there is no one size fits all when it comes to Nmap scans. Before you run any sort of Nmap wrapper you should always look at the parameters that are in play and craft them to meet your needs and applicable scenarios. For your convenience here are the individual nmap commands I have incorporated in these scripts. \n\nI like to keep the structure of my nmap commands consistent in a TARGET PORT OMIT SCAN SPEED VERBOSITY OUTPUT format.\n\n| Stage | Nmap Command | Requires Root\n| -------------- | :--------- | :--------- | \n| Host Discovery - ICMP Echo | nmap TARGET -n -sn -PE -vv -oX OUTPUT | Yes\n| Host Discovery - ICMP Netmask | nmap TARGET -n -sn -PM -vv -oX OUTPUT | Yes\n| Host Discovery - ICMP Timestamp | nmap TARGET -n -sn -PP -vv -oX OUTPUT | Yes\n| Host Discovery - Port Scanning | nmap TARGET -PS21,22,23,25,80,113,443 -PA80,113,443 -n -sn -T4 -vv -oX OUTPUT | Yes\n| Port Scanning (Top 1000) | nmap TARGET --top-ports 1000 -n -Pn -sS -T4 --min-parallelism 100 --min-rate 64 -vv -oX OUTPUT | Yes\n| Service Detection | nmap TARGET -p PORTS -n -Pn -sV --version-intensity 6 --script banner -T4 -vv -oX OUTPUT | No\n| OS Detection | nmap TARGET -n -Pn -O -T4 --min-parallelism 100 --min-rate 64 -vv -oX OUTPUT | Yes\n| SSL Ciphers | nmap TARGET -p PORTS -n -Pn --script ssl-enum-ciphers -T4 -vv -oX OUTPUT | No\n| SSL Certs | nmap TARGET -p PORTS -n -Pn --script ssl-cert -T4 -vv -oX OUTPUT | No\n| Port Scanning (1-65535) | nmap TARGET -p- -n -Pn -sS -T4 --min-parallelism 100 --min-rate 128 -vv -oX OUTPUT | Yes\n\n## Using Subprocess with Nmap\n\nThe `subprocess` (https://docs.python.org/3/library/subprocess.html) library allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This library will make it easy for us to make calls to Nmap as well as manage the output effectively. Because we are going to use `subprocess` to call a program with parameters, we must pass our arguments as a list. What makes this tricky is each parameter will need to be its own element. However, Python makes this easy for us by using the `shlex` (https://docs.python.org/3/library/shlex.html) library. \n\nThis library takes in a string and it will split each space delimiter parameter as its own element in the list. I frequently see scripts do this manually but it's not necessary. There may be some reading this and wonder why we are using a library when we can just use the built-in `split()` method from a string. These two approaches do nearly the same thing. The difference being is the `split()` method will create a list based on the delimiter and `shlex.split()` will create a delimited list intelligently based on how the shell interprets the input. \n\nWhat this means is if you have any parameters passed to Nmap that contains spaces within quotes, then `split()` will break your input when delimiting on spaces whereas `shlex.split()` will break it down appropriately. In a nut shell, if you do not plan on using spaces where you shouldn't, `split()` will work just fine, but out of my own habit, I incorporate `shlex.split()` to build my arguments for `subprocess`. \n\nYou can see an example of what this looks like below:\n\n![Alt text](https://github.com/gh0x0st/pythonizing_nmap/blob/main/Screenshots/shlex-vs-split.png?raw=true \"shlex-vs-split\")\n\nReading STDOUT and STDERR is also relatively easy to do if you care about capturing both within your scripts. You can declare the values of the stdout/stderr arguments as `subprocess.PIPE`. Finally, you can read the data passed from stdout and stderr by using `communicate()` and declare them in variables respectively.\n\n![Alt text](https://github.com/gh0x0st/pythonizing_nmap/blob/main/Screenshots/subprocess-stdout-stderr.png?raw=true \"subprocess-stdout-stderr\")\n\nI invite you to look at the man page for subprocess to see if there's any other tricks that you could find useful as it expands a lot further than what I have provided here.\n\nhttps://docs.python.org/3/library/subprocess.html#subprocess.Popen.communicate\n\n## Parsing Nmap XML \n\nOne of my favorite features of Nmap is the ability to output our scan results to XML files. This enables us to parse through them to generate reports or use the output to generate input parameters for other Nmap operations. Let's look at an example of the XML output:\n\n![Alt text](https://github.com/gh0x0st/pythonizing_nmap/blob/main/Screenshots/xml-parse-sample.png?raw=true \"xml-parse-sample\")\n\n### Parse for Live Hosts\n\nThe first case where this will be useful for us is to determine which hosts from our host discovery probes are considered up. To facilitate this task in python we'll take advantage of the `xml.etree.ElementTree` (https://docs.python.org/3/library/xml.etree.elementtree.html) library. Take a look at the note in https://github.com/gh0x0st/pythonizing_nmap/blob/main/XML%20Parser%20Scripts/README.md for an alternate library if you do not free comfortble with using ElementTree.\n\nOur helper function will take in the path of an XML file we designate and parse out the hosts that are flagged as being 'up'. \n\nSince I use all possible discovery probes I use `parseDiscoverXml()` to take in the results from all the Xml files, then I use a second helper function to remove any duplicates and output them space delimited so I can use those at the target input values for future Nmap calls.\n\n```Python\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\n\ndef parseDiscoverXml(in_xml):\n    live_hosts = []\n    xml_tree = ET.parse(in_xml)\n    xml_root = xml_tree.getroot()\n    for host in xml_root.findall('host'):\n        ip_state = host.find('status').get('state')\n        if ip_state == \"up\":\n            live_hosts.append(host.find('address').get('addr'))\n    return live_hosts\n\n\ndef convertToNmapTarget(hosts):\n    hosts = list(dict.fromkeys(hosts))\n    return \" \".join(hosts)\n\n\ndef main():\n    hosts = parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_echo_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_netmask_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_timestamp_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/tcp_syn_host_discovery.xml')\n\n    print(f\"Flagged Hosts: {len(hosts)}\")\n    print(f\"Unique Hosts: {len(list(dict.fromkeys(hosts)))}\")\n    print(f\"Nmap Format Example: {convertToNmapTarget(hosts)}\")\n\nif __name__ == '__main__':\n    main()\n```\n\n![Alt text](https://github.com/gh0x0st/pythonizing_nmap/blob/main/Screenshots/parse-live-hosts.png?raw=true \"parse-live-hosts\")\n\n### Parse for Accessible Ports\n\nNow that we have a way to easily construct a list of available hosts, we can move onto port scanning. After this operation is finished, we'll need a way to programmatically parse the ports that considered available to our attacker machine. Our port scanning stages scripts will produce files called `top_1000_portscan.xml` / `full_portscan.xml` respectively. \n\nWhat we will do with this file is parse through every host in the `hosts` element and for each host we will loop through every port in the `ports` element. After it flags a port that's found to be open it'll keep all the results in a list with each element in a \"\u003cIP\u003e \u003cPORT\u003e,\u003cPORT\u003e\" format. When we start our service scanning, we'll split the results so we can designate our target host and target hosts respectively in future Nmap calls. This allows to programmatically generate our Nmap commands with the necessary target ip addresses and ports.\n    \n```Python\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\n\ndef parseDiscoverPorts(in_xml):\n    results = []\n    port_list = ''\n    xml_tree = ET.parse(in_xml)\n    xml_root = xml_tree.getroot()\n    for host in xml_root.findall('host'):\n        ip = host.find('address').get('addr')\n        ports = host.findall('ports')[0].findall('port')\n        for port in ports:\n            state = port.find('state').get('state')\n            if state == 'open':\n                port_list += port.get('portid') + ','\n        port_list = port_list.rstrip(',')\n        if port_list:\n            results.append(f\"{ip} {port_list}\")\n        port_list = ''\n    return results\n\n\ndef main():\n    targets = parseDiscoverPorts('/home/tristram/Scans/Stage_2/top_1000_portscan.xml')\n    for target in targets:\n        element = target.split()\n        target_ip = element[0]\n        target_ports = element[1]\n        print(f'Nmap Format Example: nmap {target_ip} -p {target_ports}')\n\nif __name__ == '__main__':\n    main()  \n```\n\n![Alt text](https://github.com/gh0x0st/pythonizing_nmap/blob/main/Screenshots/parse-accessible-ports.png?raw=true \"parse-accessible-ports\")\n\n### Parsing XML in Memory\n\nThe previous examples showed you how you can parse XML files that are on disk, but you are also able to parse XML without relying on XML on disk by changing a few approaches. Specifically, we'll tell Nmap to output XML to stdout and we will store that in a variable. The output itself will be stored in the first element in the `tuple` as a `bytes-like object`. We will just need to make a few changes but can borrow nearly the entire function we created before.\n\nThe only changes we need to make will be to remove `ET.parse` and `xml_tree.getroot()` and replace with `ET.fromstring`  which parses XML from a string directly into an Element, which is the root element of the parsed tree.\n\nPersonally, I do not use this approach as much as I like to have the XML files on disk so I can use with other operations. Keep in mind that if you wanted to write your tool that works with everything in memory then you should keep an eye on your system resources. Some Nmap scans can produce quite large output files and you do not want to bog down your system or lose data in the event of a system crash.\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport subprocess\nimport shlex\n\n\ndef nmapMemory(target):\n    args = shlex.split(f\"/usr/bin/nmap {target} -T4 -Pn -n -vv -sS -min-parallelism 100 --min-rate 64 --top-ports 1000 -oX -\")\n    return subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n\n\ndef parseDiscoverPortsMemory(in_xml):\n    results = []\n    port_list = ''\n    #xml_tree = ET.parse(in_xml)\n    #xml_root = xml_tree.getroot() \n    xml_root = ET.fromstring(in_xml.decode('utf-8'))\n    for host in xml_root.findall('host'):\n        ip = host.find('address').get('addr')\n        ports = host.findall('ports')[0].findall('port')\n        for port in ports:\n            state = port.find('state').get('state')\n            if state == 'open':\n                port_list += port.get('portid') + ','\n        port_list = port_list.rstrip(',')\n        if port_list:\n            results.append(f\"{ip} {port_list}\")\n        port_list = ''\n    return results\n\n\ndef main():\n    target = '127.0.0.1'\n    xml_output = nmapMemory(target)[0]\n\n    targets = parseDiscoverPortsMemory(xml_output)\n    for target in targets:\n        element = target.split()\n        target_ip = element[0]\n        target_ports = element[1]\n        print(f'Scanning: {target_ip} against ports {target_ports}')    \n\n\nif __name__ == '__main__':\n    main()\n```\n\n## Importing Nmap XML into SQLite Databases\n\nSince we have learned previously how to parse Nmap XML using Python we can also take those results and import them into SQLite Databases. From there you could use that database to build input parameters, generate reports or keep historical information from past engagements. The `sqlite3` (https://docs.python.org/3/library/sqlite3.html) library does virtually all the hard work for us. Keep in mind that this library requires us to work with `tuples` when you receive results back from the database. They work just like lists except you cannot change the element values. \n\nLet's look at how this can be done with a simple use case scenario for storing the results from a host discovery scan and whether an IP is up or not based on the results from an ICMP Echo scan.\n\n1. Creating the Database File\n2. Inserting Data into a Table\n3. Selecting Content from a Table\n\n### Creating the Database File\n\nThis is where we'll create the actual database file on disk. Within the create_db function you can setup your tables and the values you want to store. If you want some ideas on what sort of tables could work for you then consider peaking at section 6 of this post before moving on as those sections produce CSV tables with various amounts of information.\n\n```PYTHON\n#!/usr/bin/python3\n\nimport sqlite3\n\ndef create_connection(db_file):\n    conn = None\n    try:\n        conn = sqlite3.connect(db_file)\n    except Exception as e:\n        print(e)\n    return conn\n\n\ndef create_db(conn):\n    createHostDiscoveryTable=\"\"\"CREATE TABLE IF NOT EXISTS HostDiscovery (\n            id integer PRIMARY KEY,\n            IP text NOT NULL,\n            Status text NOT NULL,\n            ICMP_Echo text NOT NULL);\"\"\"\n    try:\n        c = conn.cursor()\n        c.execute(createHostDiscoveryTable)\n    except Exception as e:\n        print(e)\n\n\ndef main():\n    db_file = 'PythonizingNmap.db'\n    conn = create_connection(db_file)\n    create_db(conn)\n\n\nif __name__ == '__main__':\n    main()\n```\n\n![Alt text](https://github.com/gh0x0st/pythonizing_nmap/blob/main/Screenshots/create-database.png?raw=true \"create-database\")\n\n### Inserting Data into a Table\n\nNow that our table is created, we can define our insert_content function to insert our parsed XML data directly into the `HostDiscovery` table. Granted I hardcoded some values here this would be a good function to parameterize to make it more dynamic. Keep note that we are inserting our data as a `tuple`.\n\n```PYTHON\n#!/usr/bin/python3\n\nimport sqlite3\nimport xml.etree.ElementTree as ET\n\n\ndef create_connection(db_file):\n    conn = None\n    try:\n        conn = sqlite3.connect(db_file)\n    except Exception as e:\n        print(e)\n    return conn\n\n\ndef insert_content(conn, content):\n    sql = ''' INSERT INTO HostDiscovery(IP,Status,ICMP_Echo)\n              VALUES(?,?,?) '''\n    cur = conn.cursor()\n    cur.execute(sql, content)\n    return cur.lastrowid\n\n\ndef main():\n    # Database Connection\n    db_file = 'PythonizingNmap.db'\n    conn = create_connection(db_file)\n\n    # Parse XML\n    in_xml_echo = '/home/tristram/Scans/Stage_1/icmp_echo_host_discovery.xml'\n\n    # Load ICMP Echo XML\n    xml_tree_echo = ET.parse(in_xml_echo)\n    xml_root_echo = xml_tree_echo.getroot()\n\n    # Load ICMP Echo XML\n    for host in xml_root_echo.findall('host'):\n        echo_ip = host.find('address').get('addr')\n        echo_state = host.find('status').get('state')\n        echo_reason = host.find('status').get('reason')\n        \n        # Insert results into database\n        insert_content(conn, (echo_ip, echo_state, echo_reason))\n        conn.commit()\n\n\nif __name__ == '__main__':\n    main()\n```\n\n![Alt text](https://github.com/gh0x0st/pythonizing_nmap/blob/main/Screenshots/insert-content.png?raw=true \"insert-content\")\n\n### Selecting Content from a Table\n\nAfter our data is inserted into the database what you can do from here is up to you! You could use this to store results from past engagements or even use it as a working database where you can build other automated workflows that utilize information selected from the database itself. In this example here we are selecting all the hosts that are considered 'up'.\n\n```PYTHON\n#!/usr/bin/python3\n\nimport sqlite3\n\ndef create_connection(db_file):\n    conn = None\n    try:\n        conn = sqlite3.connect(db_file)\n    except Exception as e:\n        print(e)\n    return conn\n\n\ndef select_content(conn):\n    sql = \"\"\"SELECT IP \n              FROM HostDiscovery \n              WHERE Status = 'up' \n              \"\"\"\n    cur = conn.cursor()\n    cur.execute(sql)\n    rows = cur.fetchall()\n    return rows\n\n\ndef main():\n    db_file = 'PythonizingNmap.db'\n    conn = create_connection(db_file)\n    live_hosts = select_content(conn)\n    for host in live_hosts:\n        print(f'Live: {host[0]}')\n\n\nif __name__ == '__main__':\n    main()\n```\n\n![Alt text](https://github.com/gh0x0st/pythonizing_nmap/blob/main/Screenshots/select-content.png?raw=true \"select-content\")\n\n## Python Nmap Wrapper Scripts\n\nNow that we've gone through parsing the XML files from Nmap we can use this approach to programmatically generate input parameters for other Nmap operations where we need to designate target IPs and/or ports. I have included below some thoughts around a staged approach to Nmap enumeration. Keep in mind that the code snippets provided are intended to act as blueprints for you to build upon. \n\nAs you read these examples you will find cases where we scan individual IPs at a time, resulting in multiple XML output files and others where I have a single scan targeting all the IPs resulting in a single XML output file. I did this intentionally for you to weight the benefits of parsing through individual XML files vs a single XML file. One option allows you to pass in a single file into your functions where the others require you use a for loop. If you use individually XML files it would be easier to review the results for a specific machine vs picking out the bits you want a in a larger file.\n\n### Stage 1 - Host Discovery\n\nWith this step the objective is to determine whether something exists at a particular IP based on the response to your probes. You'll typically encounter straight ICMP restrictions at the firewall, but there are cases where there's misconfigurations or even intended configurations where specific ICMP types are permitted. Because of this I like to take advantage of `ICMP ECHO`, `ICMP TIMESTAMP` and `ICMP NETMASK` probes by sending them individually.\n\nOutside of ICMP probes, another approach you will likely have to take is to run a port scan with a small subset of ports to solicit a response from the firewall. In these cases, a `RESET` or `SYN-ACK` from the firewall denotes a live host at that IP address. I combine both half open scans and ack scans (https://nmap.org/book/host-discovery-strategies.html) with a very small subset of ports to try. By combining all five of these probes together you can craft yourself a scripted host discovery solution to enhance your chances of discovering a live host.\n\nGranted during this stage all you want is to know is whether a host is up. However, I like to expand on this a little more by reporting how each host responds to each of the probes. You may identify hosts that allow ICMP and if the client believes they are blocking ICMP across the board from the internet it could be helpful for them to be aware.\n\n```PYTHON\n#!/usr/bin/python3\n\nimport shlex\nimport subprocess\nimport os\nimport sys\n\n\ndef sendIcmpEcho(target, out_xml):\n    out_xml = os.path.join(out_xml,'icmp_echo_host_discovery.xml')\n    nmap_cmd = f\"/usr/bin/nmap {target} -n -sn -PE -vv -oX {out_xml}\"                     \n    sub_args = shlex.split(nmap_cmd)\n    subprocess.Popen(sub_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n    makeInvokerOwner(out_xml)\n\n\ndef sendIcmpNetmask(target, out_xml):\n    out_xml = os.path.join(out_xml,'icmp_netmask_host_discovery.xml')\n    nmap_cmd = f\"/usr/bin/nmap {target} -n -sn -PM -vv -oX {out_xml}\"\n    sub_args = shlex.split(nmap_cmd)\n    subprocess.Popen(sub_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n    makeInvokerOwner(out_xml)\n\n\ndef sendIcmpTimestamp(target, out_xml):\n    out_xml = os.path.join(out_xml,'icmp_timestamp_host_discovery.xml')\n    nmap_cmd = f\"/usr/bin/nmap {target} -n -sn -PP -vv -oX {out_xml}\"\n    sub_args = shlex.split(nmap_cmd)\n    subprocess.Popen(sub_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n    makeInvokerOwner(out_xml)\n\n\ndef sendTcpSyn(target, out_xml):\n    out_xml = os.path.join(out_xml,'tcp_syn_host_discovery.xml')\n    nmap_cmd = f\"/usr/bin/nmap {target} -PS21,22,23,25,80,113,443 -PA80,113,443 -n -sn -T4 -vv -oX {out_xml}\"\n    sub_args = shlex.split(nmap_cmd)\n    subprocess.Popen(sub_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n    makeInvokerOwner(out_xml)\n\n\ndef makeInvokerOwner(path):\n    uid = os.environ.get('SUDO_UID')\n    gid = os.environ.get('SUDO_GID')\n    if uid is not None:\n        os.chown(path, int(uid), int(gid))\n\n\ndef is_root():\n    if os.geteuid() == 0:\n        return True\n    else:\n        return False\n\n\ndef main():\n    if not is_root():\n        print('[!] The discovery probes in this script requires root privileges')\n        sys.exit(1)\n    \n    target = '127.0.0.1'\n\n    sendIcmpEcho(target, os.getcwd())\n    sendIcmpNetmask(target, os.getcwd())\n    sendIcmpTimestamp(target, os.getcwd())\n    sendTcpSyn(target, os.getcwd())\n\nif __name__ == '__main__':\n    main()\n```\n\n### Stage 2 - Port Scanning (Top 1000)\n\nI do not find services running on non-standard ports too often in production. Because of this I focus on the ports that have a higher ratio as defined in the nmap-services file. This will help save you time while finding the ports that are likely to be accessible. Based on the nmap author's research (https://nmap.org/book/performance-port-selection.html), scanning the top 1000 ports will catch roughly 93% of the TCP ports. The statistics here are in your favor and you'll find most of the ports within a reasonable amount of time. \n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport subprocess\nimport shlex\nimport os\nimport sys\n\n\ndef parseDiscoverXml(in_xml):\n    live_hosts = []\n    xml_tree = ET.parse(in_xml)\n    xml_root = xml_tree.getroot()\n    for host in xml_root.findall('host'):\n        ip_state = host.find('status').get('state')\n        if ip_state == \"up\":\n            live_hosts.append(host.find('address').get('addr'))\n    return live_hosts\n\n\ndef convertToNmapTarget(hosts):\n    hosts = list(dict.fromkeys(hosts))\n    return \" \".join(hosts)\n\n\ndef tcpSynPortScan(target, out_xml,):\n    out_xml = os.path.join(out_xml,'top_1000_portscan.xml')\n    nmap_cmd = f\"/usr/bin/nmap {target} --top-ports 1000 -n -Pn -sS -T4 --min-parallelism 100 --min-rate 64 -vv -oX {out_xml}\"\n    sub_args = shlex.split(nmap_cmd)\n    subprocess.Popen(sub_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n    makeInvokerOwner(out_xml)\n\n\ndef makeInvokerOwner(path):\n    uid = os.environ.get('SUDO_UID')\n    gid = os.environ.get('SUDO_GID')\n    if uid is not None:\n        os.chown(path, int(uid), int(gid))\n\n\ndef is_root():\n    if os.geteuid() == 0:\n        return True\n    else:\n        return False\n\n\ndef main():\n    if not is_root():\n        print('[!] TCP/SYN scans requires root privileges')\n        sys.exit(1)\n    \n    hosts = parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_echo_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_netmask_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_timestamp_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/tcp_syn_host_discovery.xml')\n\n    target = convertToNmapTarget(hosts)\n    tcpSynPortScan(target, os.getcwd())\n\nif __name__ == '__main__':\n    main()\n```\n\n### Stage 3 - Service Detection\n\nService scanning is something that will catch inexperienced pen testers off guard when they discover that a simple service scan, they run all the time on CTFs just alerted a blue team to their presence an hour in on their assessment. Allow me to provide you an example of what I’m talking about and look at an example from the nmap-service-probes file:\n\n![Alt text](https://github.com/gh0x0st/pythonizing_nmap/blob/main/Screenshots/nmap-service-probes.png?raw=true \"nmap-service-probes\")\n\nIf you come across any server that uses ports 515,1028,1068,1503,1720,1935,2040,3388,3389 then nmap, with the default options, will eventually use the TerminalServer probes. Here's the problem. If you have a client that uses a Cisco IPS for example that sits in front of that server and it sees `\\x03\\0\\0\\x0b\\x06\\xe0\\0\\0\\0\\0\\0|` destined to any port that isn't 3389, then it's going to flag you thinking you're trying to connect to RDP on a non-standard port. Because of this as a rule of thumb I put a hard stop on letting nmap try to service probe anything on those ports so I block those off the bat in the config file on line 29 `Exclude T:9100-9107,T:515,T:1028,T:1068,T:1503,T:1720,T:1935,T:2040,T:3388`.\n\n_NOTE: There is a `--exclude-ports` parameter but I like to show people that there are configurable options within the config files_\n\nThe problem doesn't stop there though. If you run into a port that nmap cannot figure out, it will try every possible probe up the intensity level, which by default is 7 (https://nmap.org/book/man-version-detection.html). If you look at the snippet below, there is another terminal server probe that is set to rarity 7, so those probes would be included. To prevent that from happening, I set my intensity version to 5 or 6 via `--version-intensity` depending on how paranoid I am.\n\nIf you have access to lab network with some sort of IDS/IPS it would be great practice for you to see what type of scans trigger alerts and what you can do to prevent them from happening.\n\n![Alt text](https://github.com/gh0x0st/pythonizing_nmap/blob/main/Screenshots/nmap-service-probes-2.png?raw=true \"nmap-service-probes-2\")\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport subprocess\nimport shlex\nimport os\n\n\ndef parseDiscoverPorts(in_xml):\n    results = []\n    port_list = ''\n    xml_tree = ET.parse(in_xml)\n    xml_root = xml_tree.getroot()\n    for host in xml_root.findall('host'):\n        ip = host.find('address').get('addr')\n        ports = host.findall('ports')[0].findall('port')\n        for port in ports:\n            state = port.find('state').get('state')\n            if state == 'open':\n                port_list += port.get('portid') + ','\n        port_list = port_list.rstrip(',')\n        if port_list:\n            results.append(f\"{ip} {port_list}\")\n        port_list = ''\n    return results\n\n\ndef serviceScan(target_ip, target_ports, out_xml):\n    out_xml = os.path.join(out_xml,f'{target_ip}_services.xml')\n    nmap_cmd = f\"/usr/bin/nmap {target_ip} -p {target_ports} -n -Pn -sV --version-intensity 6 --script banner -T4 -vv -oX {out_xml}\"\n    sub_args = shlex.split(nmap_cmd)\n    subprocess.Popen(sub_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n\n\ndef main():\n    in_xml = '/home/tristram/Scans/Stage_2/top_1000_portscan.xml'\n    targets = parseDiscoverPorts(in_xml)\n    for target in targets:\n        element = target.split()\n        target_ip = element[0]\n        target_ports = element[1]\n        print(f'Scanning: {target_ip} against ports {target_ports}')\n        serviceScan(target_ip, target_ports, os.getcwd())\n        \n\nif __name__ == '__main__':\n    main()\n```\n\n### Stage 4 - OS Detection\n\nI'm a bit torn on using the OS discovery scan over the internet. Sometimes it does not provide me anything useful and other times it provides me a gold mine with unsupported operating systems. I will run this scan just to see and will try to verify through other types of enumeration, such as identifying os requirements for the running software if I'm able. If I'm on the network probing a device, I'll typically use this all the time if I'm on the internal network but over the internet it all depends on if I have anything else to work off from first.\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport subprocess\nimport shlex\nimport os\nimport sys\n\n\ndef parseDiscoverXml(in_xml):\n    live_hosts = []\n    xml_tree = ET.parse(in_xml)\n    xml_root = xml_tree.getroot()\n    for host in xml_root.findall('host'):\n        ip_state = host.find('status').get('state')\n        if ip_state == \"up\":\n            live_hosts.append(host.find('address').get('addr'))\n    return live_hosts\n\n\ndef convertToNmapTarget(hosts):\n    hosts = list(dict.fromkeys(hosts))\n    return \" \".join(hosts)\n\n\ndef osScan(targets, out_xml):\n    out_xml = os.path.join(out_xml,f'osdetection.xml')\n    nmap_cmd = f\"/usr/bin/nmap {targets} -n -Pn -O -T4 --min-parallelism 100 --min-rate 64 -vv -oX {out_xml}\"\n    sub_args = shlex.split(nmap_cmd)\n    subprocess.Popen(sub_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n    makeInvokerOwner(out_xml)\n\n\ndef makeInvokerOwner(path):\n    uid = os.environ.get('SUDO_UID')\n    gid = os.environ.get('SUDO_GID')\n    if uid is not None:\n        os.chown(path, int(uid), int(gid))\n\n\ndef is_root():\n    if os.geteuid() == 0:\n        return True\n    else:\n        return False    \n\n\ndef main():\n    if not is_root():\n        print('[!] TCP/IP fingerprinting (for OS scan) requires root privileges.')\n        sys.exit(1)\n    \n    hosts = parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_echo_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_netmask_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_timestamp_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/port_host_discovery.xml')\n\n    target = convertToNmapTarget(hosts)\n\n    osScan(target, os.getcwd())\n        \n\nif __name__ == '__main__':\n    main()\n```\n\n### Stage 5 - SSL Ciphers\n\nFor the most part I try to keep NSE scripts for more targeted enumeration, apart from `ssl-enum-ciphers` and `ssl-certs`. The NSE script `ssl-enum-ciphers` is particularly useful for when your target under regulatory requirements and aren't supposed to be using unsafe TLS configurations. Some of the NSE scripts can be noisy so weigh the benefit of what you are trying to learn about a target vs the risk of being busted. \n\nThe results of this NSE script exports nicely into XML and I'll show you how you can convert these results into a CSV format so you can easily move into a report further down.\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport subprocess\nimport shlex\nimport os\n\n\ndef parseDiscoverPorts(in_xml):\n    results = []\n    port_list = ''\n    xml_tree = ET.parse(in_xml)\n    xml_root = xml_tree.getroot()\n    for host in xml_root.findall('host'):\n        ip = host.find('address').get('addr')\n        ports = host.findall('ports')[0].findall('port')\n        for port in ports:\n            state = port.find('state').get('state')\n            if state == 'open':\n                port_list += port.get('portid') + ','\n        port_list = port_list.rstrip(',')\n        if port_list:\n            results.append(f\"{ip} {port_list}\")\n        port_list = ''\n    return results\n\n\ndef sslCipherScan(target_ip, target_ports, out_xml):\n    out_xml = os.path.join(out_xml,f'{target_ip}_ssl_ciphers.xml')\n    nmap_cmd = f\"/usr/bin/nmap {target_ip} -p {target_ports} -n -Pn --script ssl-enum-ciphers -T4 -vv -oX {out_xml}\"\n    sub_args = shlex.split(nmap_cmd)\n    subprocess.Popen(sub_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n\n\ndef main():\n    in_xml = '/home/tristram/Scans/Stage_2/syn_port_scan.xml'\n    targets = parseDiscoverPorts(in_xml)\n    for target in targets:\n        element = target.split()\n        target_ip = element[0]\n        target_ports = element[1]\n        print(f'Scanning: {target_ip} against ports {target_ports}')\n        sslCipherScan(target_ip, target_ports, os.getcwd())\n        \n\nif __name__ == '__main__':\n    main()\n```\n\n### Stage 6 - SSL Certs\n\nI like to include this step because from time to time misconfigured or poorly crafted SSL certificates can reveal quite a bit of information. For example, if you identify a web server that's accessible to the internet and it has a certificate signed by an internal CA then there is a good chance that web server is behind reverse proxy or a server on the private network being NAT'd to the internet which could lead to a damaging foothold if you can identify an exploitable condition.\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport subprocess\nimport shlex\nimport os\n\n\ndef parseDiscoverPorts(in_xml):\n    results = []\n    port_list = ''\n    xml_tree = ET.parse(in_xml)\n    xml_root = xml_tree.getroot()\n    for host in xml_root.findall('host'):\n        ip = host.find('address').get('addr')\n        ports = host.findall('ports')[0].findall('port')\n        for port in ports:\n            state = port.find('state').get('state')\n            if state == 'open':\n                port_list += port.get('portid') + ','\n        port_list = port_list.rstrip(',')\n        if port_list:\n            results.append(f\"{ip} {port_list}\")\n        port_list = ''\n    return results\n\n\ndef sslCertScan(target_ip, target_ports, out_xml):\n    out_xml = os.path.join(out_xml,f'{target_ip}_ssl_certs.xml')\n    nmap_cmd = f\"/usr/bin/nmap {target_ip} -p {target_ports} -n -Pn --script ssl-cert -T4 -vv -oX {out_xml}\"\n    sub_args = shlex.split(nmap_cmd)\n    subprocess.Popen(sub_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n\n\ndef main():\n    in_xml = '/home/tristram/Scans/Stage_2/syn_port_scan.xml'\n    targets = parseDiscoverPorts(in_xml)\n    for target in targets:\n        element = target.split()\n        target_ip = element[0]\n        target_ports = element[1]\n        print(f'Scanning: {target_ip} against ports {target_ports}')\n        sslCertScan(target_ip, target_ports, os.getcwd())\n        \n\nif __name__ == '__main__':\n    main()\n```\n\n### Stage 7 - Port Scanning (1-65535)\n\nI intentionally run this step last because it takes a long time if you have a lot of hosts. Based on the stats from the first port scan we'll only have a 7% chance of finding anything new so the return on investment of is particularly low. However, this stage is still something worth digging into a little bit. Obviously, we want our scans to run as fast as possible but we're too noisy we might trip an alarm, especially since we're scanning the entire TCP port range. \n\nIf you have a lot of time to spare, consider the low and slow approach. If you're not concerned about alerts, then play around with the timing and performance parameters (https://nmap.org/book/man-performance.html). I've found `--min-parallelism 100 --min-rate 128` to be a good sweet spot between speed and reliably.\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport subprocess\nimport shlex\nimport os\nimport sys\n\n\ndef parseDiscoverXml(in_xml):\n    live_hosts = []\n    xml_tree = ET.parse(in_xml)\n    xml_root = xml_tree.getroot()\n    for host in xml_root.findall('host'):\n        ip_state = host.find('status').get('state')\n        if ip_state == \"up\":\n            live_hosts.append(host.find('address').get('addr'))\n    return live_hosts\n\n\ndef convertToNmapTarget(hosts):\n    hosts = list(dict.fromkeys(hosts))\n    return \" \".join(hosts)\n\n\ndef tcpSynPortScan(target, out_xml,):\n    out_xml = os.path.join(out_xml,'65535_portscan.xml')\n    nmap_cmd = f\"/usr/bin/nmap {target} -p- -n -Pn -sS -T4 --min-parallelism 100 --min-rate 128 -vv -oX {out_xml}\"\n    sub_args = shlex.split(nmap_cmd)\n    subprocess.Popen(sub_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE).communicate()\n    makeInvokerOwner(out_xml)\n\n\ndef makeInvokerOwner(path):\n    uid = os.environ.get('SUDO_UID')\n    gid = os.environ.get('SUDO_GID')\n    if uid is not None:\n        os.chown(path, int(uid), int(gid))\n\n\ndef is_root():\n    if os.geteuid() == 0:\n        return True\n    else:\n        return False\n\n\ndef main():\n    if not is_root():\n        print('[!] TCP/SYN scans requires root privileges')\n        sys.exit(1)\n    \n    hosts = parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_echo_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_netmask_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/icmp_timestamp_host_discovery.xml')\n    hosts += parseDiscoverXml('/home/tristram/Scans/Stage_1/port_host_discovery.xml')\n\n    target = convertToNmapTarget(hosts)\n    tcpSynPortScan(target, os.getcwd())\n\nif __name__ == '__main__':\n    main()\n```\n\n## Generating Reports from Nmap XML\n\nDepending on the size of your engagement the process of transcribing your notes into a report can be quite tedious. Thankfully this is another place where Python comes to the rescue. We can take the same XML files we were working with before to generate CSV files that we can then use to import into the report format of our choosing. \n\nThe scripts for this can be a little confusing with the all the loops so I added comments to help describe each step. Keep a mental note that if there are multiple tables shown in a section that means that script will create that many tables.\n\n### Detected Hosts\n\n| IP | Status | ICMP Echo | ICMP Netmask | ICMP Timestamp | Port\n| :--- | :---| :---| :---| :---| :---| \n| 192.168.0.100|down|no-response|no-response|no-response|no-response\n| 192.168.0.101|up|echo-reply|no-response|timestamp-reply|reset\n| 192.168.0.102|up|echo-reply|no-response|no-response|syn-ack\n| 192.168.0.103|down|no-response|no-response|no-response|no-response\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport csv\n\ndef main():\n    # File Paths\n    in_xml_port = '/home/tristram/Scans/Stage_1/tcp_syn_host_discovery.xml'\n    in_xml_echo = '/home/tristram/Scans/Stage_1/icmp_echo_host_discovery.xml'\n    in_xml_netmask = '/home/tristram/Scans/Stage_1/icmp_netmask_host_discovery.xml'\n    in_xml_timestamp = '/home/tristram/Scans/Stage_1/icmp_timestamp_host_discovery.xml'\n\n    # Load Port XML\n    xml_tree_port = ET.parse(in_xml_port)\n    xml_root_port = xml_tree_port.getroot()\n\n    # Load ICMP Echo XML\n    xml_tree_echo = ET.parse(in_xml_echo)\n    xml_root_echo = xml_tree_echo.getroot()\n\n    # Load ICMP Netmask XML\n    xml_tree_netmask = ET.parse(in_xml_netmask)\n    xml_root_netmask = xml_tree_netmask.getroot()\n\n    # Load ICMP Timestamp XML\n    xml_tree_timestamp = ET.parse(in_xml_timestamp)\n    xml_root_timestamp = xml_tree_timestamp.getroot()\n\n    # CSV File\n    with open('detected_hosts.csv', 'w') as file:\n        writer = csv.writer(file)\n        # CSV Headers\n        writer.writerow(['IP', 'Status', 'ICMP Echo', 'ICMP Netmask', 'ICMP Timestamp', 'Port'])\n\n        # Load SYN Port XML\n        for host in xml_root_port.findall('host'):\n            host_status = 'down'\n            master_ip = host.find('address').get('addr')\n            port_state = host.find('status').get('state')\n            port_reason = host.find('status').get('reason')\n\n            # Load ICMP Echo XML\n            for host in xml_root_echo.findall('host'):\n                echo_ip = host.find('address').get('addr')\n                echo_state = host.find('status').get('state')\n                echo_reason = host.find('status').get('reason')\n                \n                # Load ICMP Netmask\n                if master_ip == echo_ip:\n                    for host in xml_root_netmask.findall('host'):\n                        netmask_ip = host.find('address').get('addr')\n                        netmask_state = host.find('status').get('state')\n                        netmask_reason = host.find('status').get('reason')\n                        \n                        # Load ICMP Timestamp\n                        if master_ip == netmask_ip: \n                            for host in xml_root_timestamp.findall('host'):\n                                timestamp_ip = host.find('address').get('addr')\n                                timestamp_state = host.find('status').get('state')\n                                timestamp_reason = host.find('status').get('reason')\n                                if master_ip == timestamp_ip:\n                                    if port_state == 'up' or echo_state == 'up' or netmask_state == 'up' or timestamp_state == 'up':\n                                        host_status = 'up'\n                                    \n                                    # Write results to row\n                                    writer.writerow([master_ip, host_status, echo_reason, netmask_reason, timestamp_reason, port_reason])\n\nif __name__ == '__main__':\n    main()\n```\n\n### Detected Hosts Without Ports\n\n| IP | Port| Service\n| :--- | :---| :---|\n| 192.168.0.104|no-response|no-response|no-response\n| 192.168.0.105|no-response|no-response|no-response\n| 192.168.0.106|no-response|no-response|no-response\n| 192.168.0.107|no-response|no-response|no-response\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport csv\n\ndef main():\n    # Path to directory with host XML files\n    in_xml = '/home/tristram/Scans/Stage_2/top_1000_portscan.xml'\n\n    # CSV Data\n    with open('detected_hosts_no_ports.csv', 'w') as file:\n        writer = csv.writer(file)\n\n        # CSV Headers\n        writer.writerow(['IP', 'Port', 'Service'])\n\n        # Load Top 1000 Port Scan\n        xml_tree = ET.parse(in_xml)\n        xml_root = xml_tree.getroot()\n        \n        # Cycle through each host\n        for host in xml_root.findall('host'):\n            ip_address = host.findall('address')[0].attrib['addr']\n            ports_element = host.findall('ports')\n            port_child = ports_element[0].findall('port')\n            open_ports = []\n\n            # Within each host cycle through the ports\n            for port in port_child:\n                if port.findall('state')[0].attrib['state'] == 'open':\n                    port_id = port.attrib['portid']\n                    open_ports.append(port_id)\n\n            # Write results to row\n            if len(open_ports) == 0:\n                writer.writerow([ip_address, 'no-response', 'no-response'])\n\n\nif __name__ == '__main__':\n    main()\n```\n\n### Detected Hosts With Ports + Services\n\n| IP | Port| Service\n| :--- | :---| :---|\n| 192.168.0.108|443|https\n| 192.168.0.109|443|https\n| 192.168.0.110|25|tcpwrapped\n| 192.168.0.111|443|https\n| 192.168.0.112|25|Microsoft Exchange smtpd\n| |443|https\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport csv\nimport os\n\ndef main():\n    # Path to directory with host XML files\n    in_path = '/home/tristram/Scans/Stage_3/'\n\n    # CSV Data\n    with open('detected_hosts_with_ports.csv', 'w') as file:\n        writer = csv.writer(file)\n\n        # CSV Headers\n        writer.writerow(['IP', 'Port', 'Service'])\n        for file in sorted(os.listdir(in_path)):\n            if file.endswith(\".xml\"):\n                if \"no_ports.xml\" in file:\n                    writer.writerow([file.split('_no_ports.xml')[0], 'no-response', 'no-response'])\n                else:\n                    # Load Service Scan\n                    in_xml = os.path.join(in_path, file)\n                    xml_tree = ET.parse(in_xml)\n                    xml_root = xml_tree.getroot()\n\n                    # Cycle through each host\n                    for host in xml_root.findall('host'):\n                        ip_once = True\n                        ip_address = host.findall('address')[0].attrib['addr']\n                        ports_element = host.findall('ports')\n                        port_child = ports_element[0].findall('port')\n                        open_ports = []\n\n                        # Within each host cycle through the ports\n                        for port in port_child:\n                            if port.findall('state')[0].attrib['state'] == 'open':\n                                port_id = port.attrib['portid']\n                                service_name = port.find('service').get('product')\n                                if service_name == None:\n                                    service_name = port.find('service').get('name')\n                                open_ports.append([port_id, service_name])\n                        \n                        # Within each port cycle through the open ports\n                        if len(open_ports):\n                            for op in open_ports:\n                                # Ensure we only notate the IP once to keep it clean\n                                if ip_once == True:\n                                    # Write results to row\n                                    writer.writerow([ip_address, op[0], op[1]])\n                                    ip_once = False\n                                else:\n                                    # Write results to row\n                                    writer.writerow([None, op[0], op[1]])\n                        else:\n                            # Write results to row if none\n                            writer.writerow([ip_address, 'none', 'none'])\n\nif __name__ == '__main__':\n    main()\n```\n\n### Detected Hosts With Guessed Operating Systems\n\n| IP | Port\n| :--- | :---|\n| 192.168.0.113|Unknown\n| 192.168.0.114|D-Link DCS-6620G webcam or Linksys BEFSR41 EtherFast router\n| 192.168.0.115|Linux 4.9\n| 192.168.0.116|Linux 2.6.32\n| 192.168.0.117|FreeBSD 9.0-RELEASE - 10.3-RELEASE\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport csv\n\ndef main():\n    # Path top the os scan file\n    in_xml = '/home/tristram/Scans/Stage_4/osdetection.xml'\n\n    # CSV Data\n    with open('detected_hosts_os.csv', 'w') as file:\n        writer = csv.writer(file)\n        # CSV Headers\n        writer.writerow(['IP', 'OperatingSystem'])\n\n        # Load OS Scan XML\n        xml_tree = ET.parse(in_xml)\n        xml_root = xml_tree.getroot()\n\n        # Cycle through each host\n        for host in xml_root.findall('host'):\n            ip_address = host.findall('address')[0].attrib['addr']\n            try:\n                os_element = host.findall('os')\n                os_name = os_element[0].findall('osmatch')[0].attrib['name']\n            except IndexError:\n                os_name = 'Unknown'\n            \n            # Write results to row\n            writer.writerow([ip_address, os_name])\n            \n\nif __name__ == '__main__':\n    main()\n```\n\n### Detected Hosts TLS Protocols\n\n#### TLSv1.0\n\n| IP | Port | Protocol\n| :--- | :---| :--- |\n| 192.168.0.118|443|TLSv1.0\n| 192.168.0.119|25|TLSv1.0\n| 192.168.0.120|443|TLSv1.0\n| 192.168.0.121|443|TLSv1.0\n| 192.168.0.122|5061|TLSv1.0\n\n#### TLSv1.1\n\n| IP | Port | Protocol\n| :--- | :---| :--- |\n| 192.168.0.123|443|TLSv1.1\n| 192.168.0.124|25|TLSv1.1\n| 192.168.0.125|443|TLSv1.1\n| 192.168.0.126|443|TLSv1.1\n| 192.168.0.127|5061|TLSv1.1\n\n#### SSLv3.0\n\n| IP | Port | Protocol\n| :--- | :---| :--- |\n| 192.168.0.128|443|SSLv3.0\n| 192.168.0.129|25|SSLv3.0\n| 192.168.0.130|443|SSLv3.0\n| 192.168.0.131|443|SSLv3.0\n| 192.168.0.132|5061|SSLv3.0\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport csv\nimport os\n\n\ndef main():\n    # Protocol Report Lists\n    tls_v10_report = []\n    tls_v11_report = []\n    ssl_v3_report = []\n\n    # Flagged Lists\n    flagged_tls_v10 = ''\n    flagged_tls_v11 = ''\n    flagged_ssl_v3 = ''\n\n    # File Path\n    in_path= '/home/tristram/Scans/Stage_5/'\n\n    # Cycle through each XML file\n    for file in os.listdir(in_path):\n        # Load each XML file\n        if file.endswith(\".xml\"):\n            in_xml = os.path.join(in_path, file)\n            xml_tree = ET.parse(in_xml)\n            xml_root = xml_tree.getroot()\n            \n            # Cycle through each host\n            for host in xml_root.findall('host'):\n                ip = host.find('address').get('addr')\n                ports_element = host.findall('ports')\n                port_element = ports_element[0].findall('port')\n                \n                # Cycle through each port\n                for scanned_port in port_element:\n                    port_id = scanned_port.get('portid')\n                    script_element = scanned_port.find('script')\n                    if script_element:\n                        # Cycle through each protocol\n                        protocol_element = script_element.findall('table')\n                        for tls_protocol in protocol_element: \n                            protocol_version = tls_protocol.attrib['key']\n                            \n                            # Stage data for TLSv1.0 Table\n                            if protocol_version in 'TLSv1.0':\n                                flagged_tls_v10 += protocol_version + ','\n                            \n                            # Stage data for TLSv1.1 Table\n                            if protocol_version in 'TLSv1.1':\n                                flagged_tls_v11 += protocol_version + ','\n                            \n                            # Stage data for SSLv3 Table\n                            if protocol_version in 'SSLv3':\n                                flagged_ssl_v3 += protocol_version + ','\n\n                        # Load TLSv1.0 Data\n                        if flagged_tls_v10:\n                            flagged_tls_v10 = flagged_tls_v10.strip(',').split(',')\n                            tls_v10_report.append([ip,port_id,flagged_tls_v10 ])\n                            flagged_tls_v10 = ''\n                        \n                        # Load TLSv1.1 Data\n                        if flagged_tls_v11:\n                            flagged_tls_v11 = flagged_tls_v11.strip(',').split(',')\n                            tls_v11_report.append([ip,port_id,flagged_tls_v11 ])\n                            flagged_tls_v11 = ''\n                        \n                        # Load SSLv3.0 Data\n                        if flagged_ssl_v3:\n                            flagged_ssl_v3 = flagged_ssl_v3.strip(',').split(',')\n                            ssl_v3_report.append([ip,port_id,flagged_ssl_v3 ])\n                            flagged_ssl_v3 = ''\n\n\n            # CSV Data for TLSv1.0\n            with open('detected_tls_v10.csv', 'w') as csvFile:\n                writer = csv.writer(csvFile)\n\n                # CSV Headers\n                writer.writerow([\"IP\", \"Port\", \"Protocol\"])\n\n                # Cycle through each staged element in list\n                for server in tls_v10_report:\n                    row_ip = server[0]\n                    row_port = server[1]\n                    \n                    # Write results to row\n                    writer.writerow([row_ip,row_port,'TLSv1.0'])\n\n            # CSV Data for TLSv1.1\n            with open('detected_tls_v11.csv', 'w') as csvFile:\n                writer = csv.writer(csvFile)\n\n                # CSV Headers\n                writer.writerow([\"IP\", \"Port\", \"Protocol\"])\n                for server in tls_v11_report:\n                    row_ip = server[0]\n                    row_port = server[1]\n                    \n                    # Write results to row\n                    writer.writerow([row_ip,row_port,'TLSv1.1'])\n\n            # CSV Data for SSLv.3\n            with open('detected_ssl_v3.csv', 'w') as csvFile:\n                writer = csv.writer(csvFile)\n\n                # CSV Headers\n                writer.writerow([\"IP\", \"Port\", \"Protocol\"])\n                for server in ssl_v3_report:\n                    row_ip = server[0]\n                    row_port = server[1]\n                    # Write results to row\n                    writer.writerow([row_ip,row_port,'SSLv3.0'])\n\nif __name__ == '__main__':\n    main()\n```\n\n### Detected SSL Certificates\n\n| IP | Port | CommonName | IssuerCommon | CertStart | CertEnd\n| :--- | :--- | :--- | :--- | :--- | :--- |\n| 192.168.0.133|443|stay.example.com|DigiCert Global CA G2|2020-05-06|2021-05-06\n| 192.168.0.134|443|off.example.com|DigiCert Global CA G2|2019-11-06|2020-11-05\n| 192.168.0.135|443|ronins.example.com|DigiCert Global CA G2|2020-05-04|2021-05-05\n| 192.168.0.136|443|lawn.example.com|DigiCert Global CA G2|2019-11-15|2020-11-14\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport csv\nimport os\nimport re\n\n\ndef main():\n    # Regular Expressions\n    reg_date = r'\\d{4}-\\d{2}-\\d{2}'\n\n    # File Path\n    in_path= '/home/tristram/Scans/Stage_6/'\n\n    # Reports\n    cert_info_list = ''\n    cert_info_report = []\n\n    # Cycle through each XML file\n    for file in os.listdir(in_path):\n        if file.endswith(\".xml\"):\n            # Load XML\n            in_xml = os.path.join(in_path, file)\n            xml_tree = ET.parse(in_xml)\n            xml_root = xml_tree.getroot()\n            \n            # Cycle through each host\n            for host in xml_root.findall('host'):\n                ip = host.find('address').get('addr')\n                ports_element = host.findall('ports')\n                port_element = ports_element[0].findall('port')\n                \n                # Check Every Port\n                for scanned_port in port_element:\n                    port_id = scanned_port.get('portid')\n                    script_element = scanned_port.find('script')\n                    \n                    # SSL Cert Top Level\n                    if script_element:\n                        ssl_element = script_element.findall('table')\n\n                        # Cycle through each certificate\n                        for ssl in ssl_element:\n                            # Subject Field\n                            if ssl.attrib.get('key') == 'subject':\n                                for data in ssl:\n                                    if data.attrib.get('key') == 'commonName':\n                                        host_common_name = data.text\n                            # Issuer Field\n                            if ssl.attrib.get('key') == 'issuer':\n                                for data in ssl:\n                                    if data.attrib.get('key') == 'commonName':\n                                        issuer_common_name = data.text\n                            \n                            # Validity Field\n                            if ssl.attrib.get('key') == 'validity':\n                                for data in ssl:\n                                    if data.attrib.get('key') == 'notBefore':\n                                        cert_start = data.text\n                                        cert_start = re.findall(reg_date, cert_start)[0]\n                                    if data.attrib.get('key') == 'notAfter':\n                                        cert_end = data.text\n                                        cert_end = re.findall(reg_date, cert_end)[0]\n                        cert_info_list = f\"{ip},{port_id},{host_common_name},{issuer_common_name},{cert_start},{cert_end}\"\n\n                # Load Data\n                cert_info_list = cert_info_list.split(',')\n                cert_info_report.append(cert_info_list)\n\n                # Reset results for next host\n                cert_info_list = ''\n\n            # CSV Data\n            with open('detected_ssl_certs.csv', 'w') as csvFile:\n                writer = csv.writer(csvFile)\n                \n                # CSV Headers\n                writer.writerow([\"IP\", \"Port\",\"CommonName\",\"IssuerCommon\",\"CertStart\", \"CertEnd\"])\n                for ci in cert_info_report:\n                    if len(ci) == 6:\n                        row_ip = ci[0]\n                        row_port = ci[1]\n                        row_cn = ci[2]\n                        row_ion = ci[3]\n                        row_cs = ci[4]\n                        row_ce = ci[5]\n                        # Write results to row\n                        writer.writerow([row_ip,row_port,row_cn, row_ion, row_cs, row_ce])\n\n\nif __name__ == '__main__':\n    main()\n```\n\n### Detected Cipher Suites\n\nThese reports are built to flag insecure cipher suites like that of virtually any vulnerability scanner. The risk levels were determined by the grade threshold output by Nmap's ssl-enum-ciphers NSE script (https://nmap.org/nsedoc/scripts/ssl-enum-ciphers.html). My own preference is to treat F, E and D as high risk and C as a moderate risk, but you can tweak that within the script itself. \n\n_NOTE: I included extra scripts for other ciphers within the repo but to keep things relatively clean I will include just a few examples below. _\n\n#### Detected High Risk Ciphers\n\n| IP | Port| Cipher Suite\n| :--- |:--- |:--- |\n|192.168.0.154|443|\"TLS_RSA_WITH_NULL_SHA (F)\n| ||TLS_RSA_WITH_NULL_MD5 (F)\"\n|192.168.0.155|443|\"TLS_RSA_EXPORT1024_WITH_RC2_CBC_56_MD5 (D)\n| ||TLS_RSA_WITH_NULL_SHA (F)\n| ||TLS_RSA_EXPORT1024_WITH_RC4_56_SHA (D)\n| ||TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5 (E)\n| ||TLS_RSA_EXPORT_WITH_DES40_CBC_SHA (E)\n| ||TLS_RSA_WITH_NULL_MD5 (F)\n| ||TLS_RSA_EXPORT_WITH_RC4_40_MD5 (E)\n| ||TLS_RSA_EXPORT1024_WITH_RC4_56_MD5 (D)\"\n\n#### Detected Moderate Risk Ciphers\n\n| IP | Port| Cipher Suite\n| :--- |:--- |:--- |\n| 192.168.0.156|443|\"TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (C)\n| ||TLS_RSA_WITH_RC4_128_MD5 (C)\n| ||TLS_RSA_WITH_3DES_EDE_CBC_SHA (C)\n| ||TLS_RSA_WITH_RC4_128_SHA (C)\"\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport csv\nimport os\n\n\ndef main():\n    # Cipher Risk Lists\n    ciphers_list = []\n    flagged_ciphers = ''\n\n    # Grade Threshold Lists\n    high_risk_grades = ['D','E','F']\n    moderate_risk_grades = ['C']\n\n    high_risk_ciphers = []\n    moderate_risk_ciphers = []\n\n    high_risk_flagged_list = ''\n    moderate_risk_flagged_list = ''\n\n    # Path to directory with host XML files\n    in_path= '/home/tristram/Scans/Stage_5/'\n    for file in os.listdir(in_path):\n        if file.endswith(\".xml\"):\n            # Load XML\n            in_xml = os.path.join(in_path, file)\n            xml_tree = ET.parse(in_xml)\n            xml_root = xml_tree.getroot()\n\n            # Cycle through each host\n            for host in xml_root.findall('host'):\n                ip = host.find('address').get('addr')\n                ports_element = host.findall('ports')\n                port_element = ports_element[0].findall('port')\n                \n                # Cycle through every port\n                for scanned_port in port_element:\n                    port_id = scanned_port.get('portid')\n                    script_element = scanned_port.find('script')\n                    if script_element:\n                        script_element = script_element.findall('table')    \n                        \n                        # Cycle through script element\n                        for tls_protocol in script_element: \n                            \n                            # Cycle through TLS protocol\n                            for protocol in tls_protocol:\n                                if protocol.attrib.get('key') == 'ciphers':\n                                    \n                                    # Cycle through each cipher\n                                    for entry in protocol:\n                                        for en in entry:\n                                            if en.attrib.get('key') == 'name':\n                                                name = en.text\n                                            if en.attrib.get('key') == 'strength':\n                                                grade = en.text\n                                                # Risk based on grade\n                                                if grade in high_risk_grades:\n                                                    high_risk_flagged_list += f\"{name} ({grade})\" + ','\n                                                if grade in moderate_risk_grades:\n                                                    moderate_risk_flagged_list += f\"{name} ({grade})\" + ','\n                                                \n                # Stage flagged data for current host\n                if high_risk_flagged_list:    \n                    high_risk_flagged_list = list(set(high_risk_flagged_list.strip(',').split(',')))\n                    high_risk_ciphers.append([ip,port_id,high_risk_flagged_list])\n\n                if moderate_risk_flagged_list:    \n                    moderate_risk_flagged_list = list(set(moderate_risk_flagged_list.strip(',').split(',')))\n                    moderate_risk_ciphers.append([ip,port_id,moderate_risk_flagged_list])\n\n                # Reset results for next host\n                high_risk_flagged_list = ''\n                moderate_risk_flagged_list = ''   \n\n           # Create High Risk Report\n            with open('high_risk_tls_ciphers.csv', 'w') as file:\n                writer = csv.writer(file)\n                \n                # CSV Headers\n                writer.writerow([\"IP\", \"Port\",\"Cipher Suite\"])\n                for hrc in high_risk_ciphers:\n                    row_ip = hrc[0]\n                    row_port = hrc[1]\n                    row_cs = '\\r\\n'.join(hrc[2])\n\n                    # Write results to row\n                    writer.writerow([row_ip,row_port,row_cs])\n\n            # Create Moderate Risk Report\n            with open('moderate_risk_tls_ciphers.csv', 'w') as file:\n                writer = csv.writer(file)\n                \n                # CSV Headers\n                writer.writerow([\"IP\", \"Port\",\"Cipher Suite\"])\n                for mrc in moderate_risk_ciphers:\n                    row_ip = mrc[0]\n                    row_port = mrc[1]\n                    row_cs = '\\r\\n'.join(mrc[2])\n\n                    # Write results to row\n                    writer.writerow([row_ip,row_port,row_cs])\n\n            \nif __name__ == '__main__':\n    main()\n```\n\n#### Detected 3DES Ciphers\n\n| IP | Port| Cipher Suite\n| :--- |:--- |:--- |\n| 192.168.0.137|443|\"TLS_RSA_WITH_3DES_EDE_CBC_SHA\n| ||TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA\"\n| 192.168.0.138|443|TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA\n| 192.168.0.139|443|\"TLS_RSA_WITH_3DES_EDE_CBC_SHA\n| ||TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA\"\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport csv\nimport os\n\n\ndef main():\n    # Cipher Risk Lists\n    ciphers_list = []\n    flagged_ciphers = ''\n\n    # Path to directory with host XML files\n    in_path= '/home/tristram/Scans/Stage_5/'\n    for file in os.listdir(in_path):\n        if file.endswith(\".xml\"):\n            # Load XML\n            in_xml = os.path.join(in_path, file)\n            xml_tree = ET.parse(in_xml)\n            xml_root = xml_tree.getroot()\n\n            # Cycle through each host\n            for host in xml_root.findall('host'):\n                ip = host.find('address').get('addr')\n                ports_element = host.findall('ports')\n                port_element = ports_element[0].findall('port')\n                \n                # Cycle through every port\n                for scanned_port in port_element:\n                    port_id = scanned_port.get('portid')\n                    script_element = scanned_port.find('script')\n                    if script_element:\n                        script_element = script_element.findall('table')    \n                        \n                        # Cycle through script element\n                        for tls_protocol in script_element: \n                            \n                            # Cycle through TLS protocol\n                            for protocol in tls_protocol:\n                                if protocol.attrib.get('key') == 'ciphers':\n                                    \n                                    # Cycle through each cipher\n                                    for entry in protocol:\n                                        for en in entry:\n                                            if en.attrib.get('key') == 'name':\n                                                name = en.text\n                                            if en.attrib.get('key') == 'strength':\n                                                # Check for targeted cipher\n                                                if '3DES' in name:\n                                                    flagged_ciphers += name + ','\n                                                \n                # Stage flagged data for current host\n                if flagged_ciphers:    \n                    flagged_ciphers = list(set(flagged_ciphers.strip(',').split(',')))\n                    ciphers_list.append([ip,port_id,flagged_ciphers])\n\n                # Reset results for next host\n                flagged_ciphers = ''\n\n            # Create NULL Cipher Report\n            with open('detected_3des_ciphers.csv', 'w') as file:\n                writer = csv.writer(file)\n                \n                # CSV Headers\n                writer.writerow([\"IP\", \"Port\",\"Cipher Suite\"])\n                for entry in ciphers_list:\n                    row_ip = entry[0]\n                    row_port = entry[1]\n                    row_cs = '\\r\\n'.join(entry[2])\n                \n                    # Write results to row\n                    writer.writerow([row_ip,row_port,row_cs])\n\n            \nif __name__ == '__main__':\n    main()\n```\n\n#### Detected RC4 Ciphers\n\n| IP | Port| Cipher Suite\n| :--- |:--- |:--- |\n|192.168.0.148|443|\"TLS_RSA_WITH_RC4_128_MD5\n| ||TLS_RSA_WITH_RC4_128_SHA\"\n|192.168.0.149|443|\"TLS_RSA_EXPORT1024_WITH_RC4_56_SHA\n| ||TLS_RSA_WITH_RC4_128_MD5\n| ||TLS_RSA_WITH_RC4_128_SHA\n| ||TLS_RSA_EXPORT1024_WITH_RC4_56_MD5\n| ||TLS_RSA_EXPORT_WITH_RC4_40_MD5\"\n\n```PYTHON\n#!/usr/bin/python3\n\nimport xml.etree.ElementTree as ET\nimport csv\nimport os\n\n\ndef main():\n    # Cipher Risk Lists\n    ciphers_list = []\n    flagged_ciphers = ''\n\n    # Path to directory with host XML files\n    in_path= '/home/tristram/Scans/Stage_5/'\n    for file in os.listdir(in_path):\n        if file.endswith(\".xml\"):\n            # Load XML\n            in_xml = os.path.join(in_path, file)\n            xml_tree = ET.parse(in_xml)\n            xml_root = xml_tree.getroot()\n\n            # Cycle through each host\n            for host in xml_root.findall('host'):\n                ip = host.find('address').get('addr')\n                ports_element = host.findall('ports')\n                port_element = ports_element[0].findall('port')\n                \n                # Cycle through every port\n                for scanned_port in port_element:\n                    port_id = scanned_port.get('portid')\n                    script_element = scanned_port.find('script')\n                    if script_element:\n                        script_element = script_element.findall('table')\n                        # Cycle through script element    \n                        for tls_protocol in script_element:\n\n                            # Cycle through TLS protocol\n                            for protocol in tls_protocol:\n                                if protocol.attrib.get('key') == 'ciphers':\n                                    \n                                    # Cycle through each cipher\n                                    for entry in protocol:\n                                        for en in entry:\n                                            if en.attrib.get('key') == 'name':\n                                                name = en.text\n                                            if en.attrib.get('key') == 'strength':\n                                                # Check for targeted cipher\n                                                if 'RC4' in name:\n                                                    flagged_ciphers += name + ','\n                                                \n                # Stage flagged data for current host\n                if flagged_ciphers:    \n                    flagged_ciphers = list(set(flagged_ciphers.strip(',').split(',')))\n                    ciphers_list.append([ip,port_id,flagged_ciphers])\n\n                # Reset results for next host\n                flagged_ciphers = ''    \n\n            # Create NULL Cipher Report\n            with open('detected_rc4_ciphers.csv', 'w') as file:\n                writer = csv.writer(file)\n                \n                # CSV Headers\n                writer.writerow([\"IP\", \"Port\",\"Cipher Suite\"])\n                for entry in ciphers_list:\n                    row_ip = entry[0]\n                    row_port = entry[1]\n                    row_cs = '\\r\\n'.join(entry[2])\n                \n                    # Write results to row\n                    writer.writerow([row_ip,row_port,row_cs])\n\n            \nif __name__ == '__main__':\n    main()\n```\n\n## Wrapping Up\n\nThere was a lot of information presented here as well as a lot of Python code. It is my hope that you found it useful and perhaps sparked some inspirational fires for you to think about designing your own enumeration tools or other automated workflows. I invite you to look at your own processes and see if the information you have learned here can be used to help enhance your own processes.\n\nTristram\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgh0x0st%2Fpythonizing_nmap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgh0x0st%2Fpythonizing_nmap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgh0x0st%2Fpythonizing_nmap/lists"}