{"id":31648251,"url":"https://github.com/trixsec/hexacloner","last_synced_at":"2025-10-07T06:59:56.762Z","repository":{"id":318302706,"uuid":"1070708842","full_name":"TrixSec/HexaCloner","owner":"TrixSec","description":null,"archived":false,"fork":false,"pushed_at":"2025-10-06T10:41:55.000Z","size":85,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-06T12:28:21.509Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TrixSec.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-06T10:31:36.000Z","updated_at":"2025-10-06T10:41:58.000Z","dependencies_parsed_at":"2025-10-06T12:28:24.683Z","dependency_job_id":"10a7df07-834e-489c-a0b2-38640d4d31aa","html_url":"https://github.com/TrixSec/HexaCloner","commit_stats":null,"previous_names":["trixsec/hexacloner"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/TrixSec/HexaCloner","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrixSec%2FHexaCloner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrixSec%2FHexaCloner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrixSec%2FHexaCloner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrixSec%2FHexaCloner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TrixSec","download_url":"https://codeload.github.com/TrixSec/HexaCloner/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TrixSec%2FHexaCloner/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278734417,"owners_count":26036404,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-07T06:59:53.622Z","updated_at":"2025-10-07T06:59:56.756Z","avatar_url":"https://github.com/TrixSec.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HexaCloner\r\n\r\nHexaCloner is an advanced, modular, and highly configurable website cloner written in Python. It supports full site mirroring, selective resource cloning, authentication, session resume, and a colorful, prompt-based CLI. Designed for both beginners and power users, HexaCloner works on Windows CMD, Linux, and Termux.\r\n\r\n## Features\r\n- **Full Website Cloning**: Download HTML, images, CSS, JS, and more, preserving the original directory structure.\r\n- **Selective Resource Cloning**: Choose to clone only HTML, images, CSS, JS, or all resources.\r\n- **URL Filtering \u0026 Depth Control**: Include/exclude URLs by regex and set maximum crawl depth.\r\n- **Threaded Downloads**: Fast, multi-threaded crawling and downloading.\r\n- **Resume \u0026 Session Export/Import**: Resume incomplete downloads and export/import session state for robust recovery.\r\n- **Authentication Support**: Clone protected/member-only sites with HTTP Basic Auth or cookies.\r\n- **Progress Bar \u0026 Logging**: Real-time progress bar (tqdm) and detailed, color-coded logs.\r\n- **Cross-Platform CLI**: Colorful, bold, and prompt-based interface for Windows, Linux, and Termux.\r\n\r\n## Installation\r\n1. Clone this repository or download the source code.\r\n2. Install dependencies:\r\n\t```sh\r\n\tpip install -r requirements.txt\r\n\t```\r\n\r\n## Usage\r\nRun the main script and follow the prompts:\r\n\r\n```sh\r\npython hexacloner.py\r\n```\r\n\r\n### CLI Options\r\n- **URL to clone**: The starting URL of the website.\r\n- **Number of threads**: Number of parallel download threads (default: 5).\r\n- **Resources to clone**: Choose from html, images, css, js, all (comma-separated).\r\n- **Include/Exclude URL pattern**: Regex to filter URLs.\r\n- **Max crawl depth**: Limit how deep the crawler goes.\r\n- **Authentication**: Enter username/password for HTTP Auth if needed.\r\n- **Session import/export**: Resume or save session state to a file.\r\n\r\n### Example\r\n```\r\npython hexacloner.py\r\n# Enter the URL to clone: https://example.com\r\n# Number of threads [default 5]: 10\r\n# Resources to clone (comma: html,images,css,js,all) [all]: html,images\r\n# Include URL pattern (regex, optional): ^https://example.com/blog\r\n# Exclude URL pattern (regex, optional): logout\r\n# Max crawl depth (int, optional): 3\r\n# Use HTTP Auth? (y/N): y\r\n# Username: user\r\n# Password: ******\r\n# Import previous session? (y/N): n\r\n# Export session after clone? (y/N): y\r\n# Session file to export: mysession.pkl\r\n```\r\n\r\n![Test Run](https://github.com/TrixSec/HexaCloner/blob/main/testrun/testrun.png)\r\n\r\n## Advanced Features\r\n- **Session Resume**: If interrupted, simply re-run with the same session file to continue.\r\n- **Progress Bar**: See real-time progress with tqdm.\r\n- **Detailed Logging**: Color-coded output for errors, warnings, and successes.\r\n\r\n## Requirements\r\n- Python 3.7+\r\n- requests\r\n- beautifulsoup4\r\n- termcolor\r\n- tqdm\r\n\r\n## License\r\nMIT License\r\n\r\n## Contributing\r\nPull requests and suggestions are welcome!\r\n\r\n## Disclaimer\r\nUse HexaCloner responsibly. Always respect website terms of service and robots.txt.\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrixsec%2Fhexacloner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftrixsec%2Fhexacloner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrixsec%2Fhexacloner/lists"}