{"id":18243824,"url":"https://github.com/hamada-khairi/web-discovery","last_synced_at":"2025-04-08T18:27:19.085Z","repository":{"id":260993605,"uuid":"882927530","full_name":"Hamada-khairi/Web-Discovery","owner":"Hamada-khairi","description":"Web Discover tool","archived":false,"fork":false,"pushed_at":"2024-11-04T04:08:04.000Z","size":13,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-14T14:43:34.638Z","etag":null,"topics":["discovery","nmap","python","reconnaissance","web"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Hamada-khairi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-04T03:59:18.000Z","updated_at":"2025-01-23T21:38:23.000Z","dependencies_parsed_at":"2024-11-04T05:29:09.948Z","dependency_job_id":null,"html_url":"https://github.com/Hamada-khairi/Web-Discovery","commit_stats":null,"previous_names":["hamada-khairi/web-discovery"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hamada-khairi%2FWeb-Discovery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hamada-khairi%2FWeb-Discovery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hamada-khairi%2FWeb-Discovery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hamada-khairi%2FWeb-Discovery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Hamada-khairi","download_url":"https://codeload.github.com/Hamada-khairi/Web-Discovery/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247900845,"owners_count":21015138,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["discovery","nmap","python","reconnaissance","web"],"created_at":"2024-11-05T09:03:27.253Z","updated_at":"2025-04-08T18:27:19.060Z","avatar_url":"https://github.com/Hamada-khairi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Enterprise Network Discovery Tool\r\n\r\nA comprehensive, production-ready network scanning and service discovery tool built for enterprise environments. This tool provides detailed insights about network services, web servers, SSL certificates, and operating systems while maintaining logs and generating detailed reports.\r\n\r\n## 🚀 Features\r\n\r\n- **Comprehensive Service Discovery**\r\n  - Web server detection and analysis\r\n  - SSL/TLS certificate information\r\n  - Operating system detection\r\n  - Service identification\r\n  - Port state analysis\r\n\r\n- **Enterprise-Grade Reporting**\r\n  - JSON reports for programmatic processing\r\n  - CSV reports for spreadsheet analysis\r\n  - Detailed logging with rotation\r\n  - Progress tracking\r\n  - Summary statistics\r\n\r\n- **Performance Optimized**\r\n  - Multi-threaded scanning\r\n  - Configurable thread pools\r\n  - Timeout controls\r\n  - Error handling and recovery\r\n\r\n- **Security Focused**\r\n  - Non-intrusive scanning methods\r\n  - Configurable scan parameters\r\n  - Rate limiting capabilities\r\n  - SSL verification options\r\n\r\n## 📋 Prerequisites\r\n\r\n- Python 3.8 or higher\r\n- Administrative privileges (for some scanning features)\r\n- Network access to target systems\r\n\r\n## 🔧 Installation\r\n\r\n1. Clone the repository:\r\n```bash\r\ngit clone https://github.com/Hamada-khairi/Web-Discovery.git\r\ncd network-scanner\r\n```\r\n\r\n2. Create and activate a virtual environment:\r\n\r\nOn Windows:\r\n```bash\r\npython -m venv venv\r\n.\\venv\\Scripts\\activate.ps1\r\n```\r\n\r\nOn Linux/MacOS:\r\n```bash\r\npython -m venv venv\r\nsource venv/bin/activate\r\n```\r\n\r\n3. Install required packages:\r\n```bash\r\npip install -r requirements.txt\r\n```\r\n\r\n## 📄 Requirements.txt\r\n```\r\nrequests\u003e=2.28.0\r\npyyaml\u003e=6.0\r\ntyping\u003e=3.7.4\r\n```\r\n\r\n## ⚙️ Configuration\r\n\r\nCreate a `config.yaml` file in the project root:\r\n\r\n```yaml\r\nports:\r\n  web:\r\n    - 80\r\n    - 443\r\n    - 8080\r\n    - 8443\r\n    - 3000\r\n    - 4000\r\n    - 5000\r\n    - 8000\r\n    - 8888\r\n  additional:\r\n    - 21    # FTP\r\n    - 22    # SSH\r\n    - 25    # SMTP\r\n    - 53    # DNS\r\n    - 3306  # MySQL\r\n    - 5432  # PostgreSQL\r\n\r\ntimeout:\r\n  port: 1    # seconds\r\n  http: 3    # seconds\r\n\r\nthreads:\r\n  host: 50   # concurrent hosts\r\n  port: 10   # concurrent ports per host\r\n\r\nhttp:\r\n  paths:\r\n    - \"/\"\r\n    - \"/robots.txt\"\r\n    - \"/sitemap.xml\"\r\n  verify_ssl: false\r\n\r\noutput:\r\n  formats:\r\n    - json\r\n    - csv\r\n  directory: scan_results\r\n```\r\n\r\n## 🚀 Usage\r\n\r\nBasic scan of a network:\r\n```bash\r\npython app.py --config config.yaml 192.168.1.0/24\r\n```\r\n\r\nScan with custom config location:\r\n```bash\r\npython app.py --config /path/to/config.yaml 10.0.0.0/24\r\n```\r\n\r\n## 📊 Output\r\n\r\nThe tool generates multiple output files in the configured directory:\r\n\r\n```plaintext\r\nscan_results/\r\n├── scan_report_20240104_153021.json\r\n├── scan_report_20240104_153021.csv\r\n└── logs/\r\n    └── scan_20240104_153021.log\r\n```\r\n\r\n### JSON Report Structure:\r\n```json\r\n{\r\n  \"summary\": {\r\n    \"scan_start\": \"2024-01-04T15:30:21.123456\",\r\n    \"scan_end\": \"2024-01-04T15:35:42.654321\",\r\n    \"duration\": \"0:05:21.530865\",\r\n    \"total_hosts\": 12,\r\n    \"total_open_ports\": 47\r\n  },\r\n  \"results\": [\r\n    {\r\n      \"ip\": \"192.168.1.100\",\r\n      \"hostname\": \"webserver.local\",\r\n      \"timestamp\": \"2024-01-04T15:30:25.123456\",\r\n      \"os_detection\": \"Linux/Unix\",\r\n      \"ports\": [\r\n        {\r\n          \"port\": 80,\r\n          \"state\": \"open\",\r\n          \"service\": \"http\",\r\n          \"web_info\": {\r\n            \"title\": \"Welcome Page\",\r\n            \"headers\": {},\r\n            \"response_code\": 200\r\n          }\r\n        }\r\n      ]\r\n    }\r\n  ]\r\n}\r\n```\r\n\r\n## 🔍 Logging\r\n\r\nLogs are stored in the `logs` directory with the following format:\r\n```plaintext\r\n2024-01-04 15:30:21,123 - NetworkScanner - INFO - Starting network scan of 192.168.1.0/24\r\n2024-01-04 15:30:25,456 - NetworkScanner - INFO - Found active host: 192.168.1.100 (webserver.local)\r\n```\r\n\r\n## ⚠️ Security Considerations\r\n\r\n1. **Authorization**: Ensure you have permission to scan the target network\r\n2. **Rate Limiting**: Configure appropriate timeouts and thread counts\r\n3. **Network Impact**: Be aware of bandwidth and system resource usage\r\n4. **Sensitive Data**: Handle scan results securely\r\n5. **Credentials**: Never store sensitive credentials in configuration files\r\n\r\n## 🛠️ Troubleshooting\r\n\r\n1. **Permission Errors**\r\n   - Ensure the script runs with appropriate privileges\r\n   - Check firewall settings\r\n   - Verify network access\r\n\r\n2. **Slow Scans**\r\n   - Adjust thread counts in config\r\n   - Check network connectivity\r\n   - Verify timeout settings\r\n\r\n3. **Missing Results**\r\n   - Check log files for errors\r\n   - Verify target network is accessible\r\n   - Ensure correct CIDR notation\r\n\r\n## 📝 Contributing\r\n\r\n1. Fork the repository\r\n2. Create a feature branch\r\n3. Commit your changes\r\n4. Push to the branch\r\n5. Create a Pull Request\r\n\r\n## 📜 License\r\n\r\nThis project is licensed under the MIT License - see the LICENSE file for details.\r\n\r\n## 🤝 Support\r\n\r\nFor support and questions:\r\n- Create an issue in the repository\r\n- Contact the maintainer\r\n- Check the documentation\r\n\r\n## 🔄 Changelog\r\n\r\n### Version 1.0.0 (2024-01-04)\r\n- Initial release\r\n- Basic network scanning\r\n- Web service detection\r\n- Report generation\r\n\r\n### Version 1.1.0 (2024-01-10)\r\n- Added SSL certificate analysis\r\n- Improved OS detection\r\n- Enhanced error handling\r\n- Performance optimizations\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhamada-khairi%2Fweb-discovery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhamada-khairi%2Fweb-discovery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhamada-khairi%2Fweb-discovery/lists"}