{"id":20281796,"url":"https://github.com/geniuszly/GenWebDownloader","last_synced_at":"2025-05-07T18:30:57.438Z","repository":{"id":252599258,"uuid":"840907708","full_name":"geniuszly/GenWebDownloader","owner":"geniuszly","description":"is a Node.js application designed to download entire websites and their resources for offline viewing. It utilizes Puppeteer for browser automation and Axios for resource fetching, ensuring a complete and efficient download process.","archived":false,"fork":false,"pushed_at":"2024-08-11T03:59:20.000Z","size":6,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-06T18:50:30.395Z","etag":null,"topics":["automation","cybersecurity","ethical-hacking","file-download","file-transfer","genwebdownloader","http","network-security","network-tools","nodejs","penetration-testing","python","web","web-downloader","website","website-downloader"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/geniuszly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-11T03:58:32.000Z","updated_at":"2024-10-06T16:41:33.000Z","dependencies_parsed_at":"2024-08-11T05:37:23.711Z","dependency_job_id":null,"html_url":"https://github.com/geniuszly/GenWebDownloader","commit_stats":null,"previous_names":["geniuszlyy/genwebdownloader","geniuszly/genwebdownloader"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geniuszly%2FGenWebDownloader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geniuszly%2FGenWebDownloader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geniuszly%2FGenWebDownloader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/geniuszly%2FGenWebDownloader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/geniuszly","download_url":"https://codeload.github.com/geniuszly/GenWebDownloader/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252933950,"owners_count":21827608,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","cybersecurity","ethical-hacking","file-download","file-transfer","genwebdownloader","http","network-security","network-tools","nodejs","penetration-testing","python","web","web-downloader","website","website-downloader"],"created_at":"2024-11-14T14:06:58.895Z","updated_at":"2025-05-07T18:30:57.163Z","avatar_url":"https://github.com/geniuszly.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GenWebDownloader\nis a Node.js-based tool designed for downloading entire websites, including HTML, CSS, JavaScript, images, and other resources. This utility is particularly useful for creating local copies of websites for offline access or analysis.\n\n# EN\n**GenWebDownloader** is a Node.js-based tool designed for downloading entire websites, including HTML, CSS, JavaScript, images, and other resources. This utility is particularly useful for creating local copies of websites for offline access or analysis.\n\n## Features\n- **Complete Website Download**: Downloads HTML, CSS, JavaScript, images, and other resources.\n- **Dynamic Content Support**: Uses Puppeteer to render and download content generated by JavaScript.\n- **Resource Handling**: Correctly handles resources from different domains.\n- **Error Handling**: Logs errors for resources that fail to download.\n\n## Installation\nTo use GenWebDownloader, you need to have `Node.js` installed on your machine. Then, install the required dependencies:\n```bash\nnpm install axios cheerio puppeteer\n```\n\n## Usage\nTo download a website, run the following command in your terminal, replacing `http://example.com` with the URL of the website you wish to download:\n```bash\nnode GenWebDownloader.js http://example.com\n```\n\n## Example\n```bash\nnode GenWebDownloader.js https://example.com\n```\nThe website's contents will be saved in a folder named after the domain, located in the same directory as the script.\n\n![image](https://github.com/user-attachments/assets/2908d816-555b-47a3-b3bf-3f1dd4d50a3a)\n\n\n# RU\n**GenWebDownloader** — это инструмент на основе Node.js, предназначенный для скачивания целых веб-сайтов, включая HTML, CSS, JavaScript, изображения и другие ресурсы. Это приложение особенно полезно для создания локальных копий сайтов для офлайн-доступа или анализа.\n\n## Особенности\n- **Полная загрузка сайта**: Скачивает HTML, CSS, JavaScript, изображения и другие ресурсы.\n- **Поддержка динамического контента**: Использует Puppeteer для рендеринга и скачивания контента, создаваемого JavaScript.\n- **Обработка ресурсов**: Корректно обрабатывает ресурсы с разных доменов.\n- **Обработка ошибок**: Логирует ошибки для ресурсов, которые не удалось скачать.\n\n## Установка\nДля использования GenWebDownloader необходимо иметь установленный `Node.js`. Затем установите необходимые зависимости:\n```bash\nnpm install axios cheerio puppeteer\n```\n\n## Использование\nЧтобы скачать сайт, выполните следующую команду в терминале, заменив `http://example.com` на URL сайта, который вы хотите скачать:\n```bash\nnode GenWebDownloader.js http://example.com\n```\n\n## Пример\n```bash\nnode GenWebDownloader.js http://example.com\n```\nСодержимое сайта будет сохранено в папке с именем домена, расположенной в той же директории, что и скрипт.\n\n![image](https://github.com/user-attachments/assets/4c385861-2777-42b1-a8bd-5298b757a413)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeniuszly%2FGenWebDownloader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgeniuszly%2FGenWebDownloader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeniuszly%2FGenWebDownloader/lists"}