{"id":13701990,"url":"https://github.com/e1abrador/sub.Monitor","last_synced_at":"2025-05-05T04:30:39.356Z","repository":{"id":173108612,"uuid":"650219193","full_name":"e1abrador/sub.Monitor","owner":"e1abrador","description":"Self-hosted passive subdomain continous monitoring tool.","archived":false,"fork":false,"pushed_at":"2024-01-30T17:13:32.000Z","size":57,"stargazers_count":154,"open_issues_count":1,"forks_count":16,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-11-13T09:40:25.418Z","etag":null,"topics":["attack-surface-management","bug-bounty","bugbounty","bugbounty-tools","bugbountytips","infosec","monitoring-tool","python","python3","red-team-tools","security","subdomain-enumeration","subdomain-scanner"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/e1abrador.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-06-06T15:43:15.000Z","updated_at":"2024-11-01T19:45:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"31ab1572-bb5d-486f-8dfd-ae3526001dc6","html_url":"https://github.com/e1abrador/sub.Monitor","commit_stats":null,"previous_names":["e1abrador/sub.monitor"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e1abrador%2Fsub.Monitor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e1abrador%2Fsub.Monitor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e1abrador%2Fsub.Monitor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e1abrador%2Fsub.Monitor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/e1abrador","download_url":"https://codeload.github.com/e1abrador/sub.Monitor/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252439365,"owners_count":21747993,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attack-surface-management","bug-bounty","bugbounty","bugbounty-tools","bugbountytips","infosec","monitoring-tool","python","python3","red-team-tools","security","subdomain-enumeration","subdomain-scanner"],"created_at":"2024-08-02T21:00:29.087Z","updated_at":"2025-05-05T04:30:39.075Z","avatar_url":"https://github.com/e1abrador.png","language":"Python","funding_links":["https://www.buymeacoffee.com/e1abrador","https://www.buymeacoffee.com/e1abrador'"],"categories":["[↑](#-table-of-contents) SubDomain's"],"sub_categories":["Tools"],"readme":"\u003ch1 align=\"center\"\u003e\n sub.Monitor\n\u003cbr\u003e\n\u003c/h1\u003e\n\n\u003cpre align=\"center\"\u003e\n\u003cb\u003e\n   Fast \u0026 user-friendly subdomain monitoring tool for continous attack surface management.\n\u003c/b\u003e\n\u003c/pre\u003e\n\n![image](https://github.com/e1abrador/sub.Monitor/assets/74373745/b7bfe315-fa26-4f55-b0af-5a3db4995dc0)\n\n## Why?\n\nWhy opt for sub.Monitor? This program offers easy setup due to its utilization of SQLite for storing all identified domains (removing the necessity for prior database management). Moreover, it boasts high customizability, with users needing to append only a few lines of code to the script using their chosen tools. In contrast to other existing solutions relying on databases such as MySQL or MongoDB, initializing the script can turn into a tedious task.\n\n## Features\n- **Fast**\n- **Easy to use**\n- **Easy to install**\n- **Easy to customize**\n- **Continuously save subdomains in the local database (with the possibility of dumping them all)**\n- **Telegram/slack/discord notifications**\n\n## Help Menu\n**sub.Monitor** flags:\n\n```console\noptions:\n  --add ADD             Domain to scan\n  --out-scope OUT_SCOPE\n                        File with out-of-scope domains\n  --file FILE           File with known subdomains\n  -D DOMAIN, --domain DOMAIN\n                        Domain to scan\n  -H HOURS, --hours HOURS\n                        Hours between scans\n  --dump                Dump all subdomains for a specific domain\n  --list                List all root domains in the database\n  -df DF                File with domains to scan\n  --inscope             Dump only in-scope domains\n  --notinscope          Dump only out-of-scope domains\n  --dumpall             Dump all subdomains with their details\n  --info                Show discovery date for subdomains\n  -help, -?             Show this help message and exit\n  ````\n  \n  ## Previous needed configurations\n  \n  You need to write the configuration (api) path files into **config.ini** file.\n  \n- [Subfinder](https://github.com/projectdiscovery/subfinder/tree/main#post-installation-instructions) api configuration file.\n- [Amass](https://github.com/owasp-amass/amass/blob/master/examples/config.ini) api configuration file.\n- Notify api configuration file.\n- You will need to specify the binary and config paths on **sub.monitor-config.ini** file.\n- Finally, execute ``pip3 install -r requirements.txt``\n\nYou can easily implement your own tools on the script, just modify the lines of code:\n\n````python\ndef run_tool(tool, domain, output_file):\n    print(f'[{datetime.datetime.now()}] - Running {tool} on {domain}')\n    if tool == 'assetfinder':\n        assetfinder_binary = config.get('Binary paths', 'assetfinder')\n        cmd = f'echo {domain} | {assetfinder_binary} -subs-only | grep -E \"{domain}$\" |grep -v \"*\" | grep -v \"@\"'\n    elif tool == 'subfinder':\n        subfinder_binary = config.get('Binary paths', 'subfinder')\n        subfinder_api = config.get('Api', 'subfinder_api')\n        cmd = f'{subfinder_binary} -d {domain} -silent -pc {subfinder_api} -all'\n    elif tool == 'amass':\n        amass_binary = config.get('Binary paths', 'amass')\n        amass_api = config.get('Api', 'amass_api')\n        cmd = f'{amass_binary} enum -passive -norecursive -noalts -d {domain} -config {amass_api}'\n    elif tool == 'my-custom-tool':\n        my-custom-tool-binary = config.get('Binary paths', 'my-custom-tool-binary-or-script')\n        cmd = f'{my-custom-tool-binary} -d {domain}'\n````\n\nIt's important to note that if you add any custom tool, the last output must be the domain results in order to sub.Monitor save them on the database, for example the way of using it would be:\n\n````bash\ncmd = f'bash customtool.sh {domain}''\n...\ncat customtool.sh\nmycustomtool -d $1 \u003e /tmp/$1.txt \u0026\u003e/dev/null; cat /tmp/$1.txt | unfurl domains | anew; rm -r /tmp/$1.txt\n````\n\nAlso add your tool name here:\n\n````python\n                for tool in ['subfinder', 'amass', 'assetfinder', 'my-custom-tool-name']:\n                    run_tool(tool, domain, output_file)\n````\n\nFinally, of course, you need to add the tool on the sub.monitor-config.ini file so that the python script can get the binary from that configuration file.\n\nThe only needed thing is that once the command is finished, it must show on the output all the domains discovered so the tool can save them on the logs file and inside the database\n\n## Work plan\n\nFirst of all **sub.Monitor** needs a list of already scanned domains:\n\n````console\n➜  cat ibm_sorted_subdomains.txt\ntest1.ibm.com\ntest2.ibm.com\ntest3.ibm.com\nsubtest.testX.ibm.com\n...\n````\n\nNow, the file containing the subdomains can be passed to **sub.Monitor** with the following command:\n\n````console\npython3 monitor.py --add ibm.com --file ibm_sorted_subdomains.txt\n          _    ___  ___            _ _\n          | |   |  \\/  |           (_) |\n ___ _   _| |__ | .  . | ___  _ __  _| |_ ___  _ __\n/ __| | | | '_ \\| |\\/| |/ _ \\| '_ \\| | __/ _ \\| '__|\n\\__ \\ |_| | |_) | |  | | (_) | | | | | || (_) | |\n|___/\\__,_|_.__/\\_|  |_/\\___/|_| |_|_|\\__\\___/|_|\n\n                    github.com/e1abrador/sub.Monitor\n\n[2023-06-06 18:16:26.002521] - 538 subdomains were added to the local database.\n````\n\nAlso there is a possibility to add an out-of-scope flag, for example, let's say that a Bug Bounty program has the following policy:\n\n````console\n*.ibm.com - in scope\nsuper-admin.ibm.com - out of scope\n*.super-admin.ibm.com - out of scope\n````\n\nIt is possible to create a file with this stuff:\n\n````console\ncat outscope.txt\nsuper-admin.ibm.com\n*.super-admin.ibm.com\n````\n\nNow, you can add manually all your discovered domains to the database (filtering the ones that are in scope):\n\n````console\npython3 monitor.py --add ibm.com --file ibm_sorted_subdomains.txt --out-scope outscope.txt\n          _    ___  ___            _ _\n          | |   |  \\/  |           (_) |\n ___ _   _| |__ | .  . | ___  _ __  _| |_ ___  _ __\n/ __| | | | '_ \\| |\\/| |/ _ \\| '_ \\| | __/ _ \\| '__|\n\\__ \\ |_| | |_) | |  | | (_) | | | | | || (_) | |\n|___/\\__,_|_.__/\\_|  |_/\\___/|_| |_|_|\\__\\___/|_|\n\n                    github.com/e1abrador/sub.Monitor\n\n[2023-06-06 18:16:26.002521] - 538 subdomains were added to the local database.\n````\n\nThis will add all your subdomains to the database, but it will mark all the ones that are out of scope with the flag ``[Out of scope]`` (we will see how to filter in-scope domains from the database on dumping domains section).\n\nTo confirm that the domain has been added to the database, execute:\n\n```console\npython3 monitor.py --list\n          _    ___  ___            _ _\n          | |   |  \\/  |           (_) |\n ___ _   _| |__ | .  . | ___  _ __  _| |_ ___  _ __\n/ __| | | | '_ \\| |\\/| |/ _ \\| '_ \\| | __/ _ \\| '__|\n\\__ \\ |_| | |_) | |  | | (_) | | | | | || (_) | |\n|___/\\__,_|_.__/\\_|  |_/\\___/|_| |_|_|\\__\\___/|_|\n\n                    github.com/e1abrador/sub.Monitor\n\nibm.com [9 subdomains added manually] [2 subdomains discovered] [11 total unique in database]\n\n````\n\nOnce it has been correctly loaded, the monitoring process can start. It is recommended to use TMUX in a VPS and leave it running for a long time. With the following command, the script will be running the subdomain enumeration tools and will compare the new results with the old results. If there's any new subdomain found, sub.Monitor will first add it to the local database (so it will not notify anymore about that discovery) and then will notify the user via slack/telegram/discord.\n\n````console\npython3 monitor.py -D ibm.com -H 12 --out-scope outscope.txt\n````\n\nIf any subdomain is found, sub.Monitor will show the following message on the output:\n\n````console\n➜ python3 monitor.py -D ibm.com -H 8 # To filter the subdomains that are in the current scope from the out-scope ones, you can use:\n                                      # python3 monitor.py -D ibm.com -H 8 --out-scope outscope.txt\n\n          _    ___  ___            _ _\n          | |   |  \\/  |           (_) |\n ___ _   _| |__ | .  . | ___  _ __  _| |_ ___  _ __\n/ __| | | | '_ \\| |\\/| |/ _ \\| '_ \\| | __/ _ \\| '__|\n\\__ \\ |_| | |_) | |  | | (_) | | | | | || (_) | |\n|___/\\__,_|_.__/\\_|  |_/\\___/|_| |_|_|\\__\\___/|_|\n\n                    github.com/e1abrador/sub.Monitor\n\n[2023-06-06 18:07:25.191169] - New subdomain xxxx.ibm.com discovered\n[2023-06-06 18:07:25.353156] - New subdomain xyxyxyxyxyx.ibm.com discovered\n[2023-06-06 18:07:25.641082] - New subdomain x1.xxxx.ibm.com discovered\n````\n\nIt is also possible to monitor more than 1 domain, with the following command:\n\n````console\npython3 monitor.py -df root-domains.txt -H 8\npython3 monitor.py -df roots.txt -H 8 --out-scope outscope.txt\n````\n\nAfter those messages are reported, on the same time the user will receive the notifications on telegram\n\n![image](https://github.com/e1abrador/sub.Monitor/assets/74373745/c67ceb5f-da77-4aed-ab28-73f32421273f)\n\nHow do I manage the notifications? I found a pretty easy (and easy to manage) way to see all newly discovered domains with Discord. I got this (blurred since all of them are private programs):\n\n![image](https://github.com/e1abrador/sub.Monitor/assets/74373745/e82a94ea-3310-48ad-a2aa-0f40a6b29637)\n\nThe idea is first to create a category regarding the program name:\n\n\\\u003e Yahoo\n\n--- yahoo.com\n  \n--- yahoo.net\n  \n--- etc ...\n  \n\\\u003e IBM\n\n--- ibm.com\n  \n--- whateverdomain.com\n  \n--- etc ...\n\nThis is what my notify config file looks like:\n\n![image](https://github.com/e1abrador/sub.Monitor/assets/74373745/1f060b91-01a2-4790-8663-d2535b58bc4e)\n\nI recommend doing this for each domain (it may be pretty tedious to set up this but it is a great way to manage the results). monitor.py script will send the results to the given webhook based on the ID (which must be the same as the domain name to scan).\n\nLet's say that the script has been running for 2 months and you want to get all the results (old subdomains and newly discovered ones). With sub.Monitor whether it is possible using the --dump flag:\n\n````console\npython3 monitor.py -D ibm.com --dump\n\n          _    ___  ___            _ _\n          | |   |  \\/  |           (_) |\n ___ _   _| |__ | .  . | ___  _ __  _| |_ ___  _ __\n/ __| | | | '_ \\| |\\/| |/ _ \\| '_ \\| | __/ _ \\| '__|\n\\__ \\ |_| | |_) | |  | | (_) | | | | | || (_) | |\n|___/\\__,_|_.__/\\_|  |_/\\___/|_| |_|_|\\__\\___/|_|\n\n                    github.com/e1abrador/sub.Monitor\n\nSubdomains for ibm.com:\nsubdomain1.ibm.com\nsubdomain2.ibm.com\n...\n````\n\nYou can also use the following command to see the day on which the domain was discovered:\n\n````console\npython3 monitor.py -D ibm.com --dump --info\n\n          _    ___  ___            _ _\n          | |   |  \\/  |           (_) |\n ___ _   _| |__ | .  . | ___  _ __  _| |_ ___  _ __\n/ __| | | | '_ \\| |\\/| |/ _ \\| '_ \\| | __/ _ \\| '__|\n\\__ \\ |_| | |_) | |  | | (_) | | | | | || (_) | |\n|___/\\__,_|_.__/\\_|  |_/\\___/|_| |_|_|\\__\\___/|_|\n\n                    github.com/e1abrador/sub.Monitor\n\nSubdomains for ibm.com:\ntest.ibm.com [discovered on 06/08/2023]\ntest2.ibm.com [discovered on 08/08/2023] [Out of scope]\n````\n\nAs it's highly probable that some domains are marked as out of scope using ``python3 monitor.py -d ibm.com --dump --info`` command, in order to show only the domains in-scope it is possible to use ``--inscope`` flag:\n\n````console\npython3 monitor.py -D ibm.com --dump --info --inscope\n\n          _    ___  ___            _ _\n          | |   |  \\/  |           (_) |\n ___ _   _| |__ | .  . | ___  _ __  _| |_ ___  _ __\n/ __| | | | '_ \\| |\\/| |/ _ \\| '_ \\| | __/ _ \\| '__|\n\\__ \\ |_| | |_) | |  | | (_) | | | | | || (_) | |\n|___/\\__,_|_.__/\\_|  |_/\\___/|_| |_|_|\\__\\___/|_|\n\n                    github.com/e1abrador/sub.Monitor\n\nSubdomains for ibm.com:\ntest.ibm.com [discovered on 06/08/2023]\ntest3.ibm.com [discovered on 08/08/2023]\n````\n\n  ## Thanks\n  \n  Thanks to:\n  \n  - Projectdiscovery for creating [subfinder](https://github.com/projectdiscovery/subfinder) and [notify](https://github.com/projectdiscovery/notify)!.\n  - Thanks to OWASP for their amazing project [amass](https://github.com/owasp-amass/amass/)!.\n  - Tomnomnom for creating [assetfinder](https://github.com/tomnomnom/assetfinder)!.\n\n## TODO\n\n- Implement the monitoring of more than 1 subdomain. [DONE] \n- Continuously read the domain files so new domains can be scanned without stopping the program. [DONE]\n- Implement out of scope filtering [DONE]\n  \nIf you have any idea of some new functionality open a PR at https://github.com/e1abrador/sub.Monitor/pulls.\n\nGood luck and good hunting!\nIf you really love the tool (or any others), or they helped you find an awesome bounty, consider [BUYING ME A COFFEE!](https://www.buymeacoffee.com/e1abrador) ☕ (I could use the caffeine!)\n\n⚪ e1abrador\n\nTwitter: https://twitter.com/e1abrador\n\n\u003ca href='https://www.buymeacoffee.com/e1abrador' target='_blank'\u003e\u003cimg height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi2.png?v=3' border='0' alt='Buy Me a Coffee at ko-fi.com' /\u003e\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fe1abrador%2Fsub.Monitor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fe1abrador%2Fsub.Monitor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fe1abrador%2Fsub.Monitor/lists"}