{"id":28428718,"url":"https://github.com/henrylok0/anydownload","last_synced_at":"2026-05-16T20:02:29.479Z","repository":{"id":295232154,"uuid":"989524205","full_name":"HenryLok0/AnyDownload","owner":"HenryLok0","description":"CLI \u0026 GUI website downloader for offline browsing. Static + Playwright render for SPAs, path discovery, local HTTP preview. npm install -g anydownload","archived":false,"fork":false,"pushed_at":"2026-05-15T18:42:22.000Z","size":264,"stargazers_count":21,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-05-15T19:20:57.101Z","etag":null,"topics":["cli","download","downloads","html","html-download","https","nodejs","puppeteer","web-scraper","website","website-backup","website-downloader","webstie"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HenryLok0.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-05-24T09:18:17.000Z","updated_at":"2026-05-15T18:42:26.000Z","dependencies_parsed_at":"2025-05-31T01:15:03.790Z","dependency_job_id":"90b81927-db60-4895-a6f9-e18205533231","html_url":"https://github.com/HenryLok0/AnyDownload","commit_stats":null,"previous_names":["henrylok0/website-downloader","henrylok0/anydownload"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/HenryLok0/AnyDownload","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HenryLok0%2FAnyDownload","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HenryLok0%2FAnyDownload/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HenryLok0%2FAnyDownload/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HenryLok0%2FAnyDownload/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HenryLok0","download_url":"https://codeload.github.com/HenryLok0/AnyDownload/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HenryLok0%2FAnyDownload/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33117350,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-16T18:38:32.183Z","status":"ssl_error","status_checked_at":"2026-05-16T18:38:29.903Z","response_time":115,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","download","downloads","html","html-download","https","nodejs","puppeteer","web-scraper","website","website-backup","website-downloader","webstie"],"created_at":"2025-06-05T13:00:20.524Z","updated_at":"2026-05-16T20:02:29.471Z","avatar_url":"https://github.com/HenryLok0.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AnyDownload\n\n[![Code Size](https://img.shields.io/github/languages/code-size/HenryLok0/AnyDownload?style=flat-square\u0026logo=github)](https://github.com/HenryLok0/AnyDownload)\n[![npm version](https://img.shields.io/npm/v/anydownload?style=flat-square)](https://www.npmjs.com/package/anydownload)\n\n[![MIT License](https://img.shields.io/github/license/HenryLok0/AnyDownload?style=flat-square)](LICENSE)\n[![Stars](https://img.shields.io/github/stars/HenryLok0/AnyDownload?style=flat-square)](https://github.com/HenryLok0/AnyDownload/stargazers)\n\nDownload websites for **offline browsing** — HTML, CSS, JavaScript, images, fonts, and common SPA bundles (React, Vite, Vue).\n\n---\n\n## Install\n\n### Desktop app (recommended for GUI users)\n\nGrab the **desktop build** for your OS from **[Releases](https://github.com/HenryLok0/AnyDownload/releases)** — Windows portable `.exe`, macOS `.dmg`, and Linux `.AppImage` are uploaded automatically when **`main`** is updated (GitHub Actions). No wizard-style setup on Windows: run the `.exe` directly.\n\n### CLI (npm)\n\n```bash\nnpm install -g anydownload\n```\n\n### From source\n\n`git clone` → `cd AnyDownload` → `npm install`\n\n---\n\n## Choose your scenario (copy \u0026 run)\n\n| I want to… | Command |\n|------------|---------|\n| Download one page + assets (default) | `anydownload example.com` |\n| Download a simple static site (fast, no browser) | `anydownload example.com --mode static` |\n| Download a React / Vite / SPA site | `anydownload example.com --mode render` |\n| Download and open preview when done (SPA) | `anydownload example.com --mode render --open` |\n| Download full site (depth 2) | `anydownload example.com --preset full` |\n| Mirror many pages (depth 5) | `anydownload example.com --preset mirror` |\n| Preview an existing download folder | `anydownload serve test` (auto-finds `test/example.com/`) |\n| Interactive wizard (URL, scope, engine — no browser pick) | `anydownload --wizard` |\n| Only CSS files | `anydownload example.com --type css` |\n| *(optional)* List URLs only (no download) | `anydownload example.com -p -o mysite` |\n\n---\n\n## Important: how to view offline sites\n\n### Do NOT double-click `index.html`\n\nModern sites (React, Vite, Next.js) use **ES modules**. Browsers block them on `file://` → you see a **white screen** even when files downloaded correctly.\n\n### DO use HTTP preview\n\n```bash\n# After download (render mode asks yes/no to open automatically)\nanydownload example.com --mode render\n\n# Or skip the prompt and open immediately\nanydownload example.com --mode render --open\n\n# Preview a folder you already downloaded (parent or host folder both work)\nanydownload serve test\nanydownload serve test/example.com\n```\n\nPreview runs at **http://127.0.0.1:8765/** (default) and always opens the site root `/`. Press **Ctrl+C** to stop the server.\n\n\u003e **Note:** `--mode render` finishes with **Open offline preview? (Y/n)**. Choosing **Yes** runs `anydownload serve \"\u003cfolder\u003e\"` and opens your browser.\n\n---\n\n## Mirrored HTML layout (framework docs)\n\n- Pages are mirrored to a folder tree aligned with URLs: `/` becomes `index.html`, `/learn` becomes `learn/index.html`, `/reference/react` becomes `reference/react/index.html`.\n- Scripts, stylesheets, and other same-origin URLs are rewritten **relative to the saved HTML file** so shared roots like `_next/static/...` still resolve offline at any depth.\n\n### Preview (`anydownload serve`)\n\n- Open `http://127.0.0.1:8765/learn/` **or** `http://127.0.0.1:8765/learn` once `learn/index.html` exists—the server probes `subdir/index.html` and `subdir.html` before falling back to the SPA bootstrap page.\n\n- Nested pages often request **`/subdir/_next/...`**, **`/subdir/static/...`**, **`/subdir/assets/...`**, or **`/subdir/logo.svg`** (relative links) while the mirror keeps those files at the site root; `anydownload serve` maps matching URLs back to the root bundles and public assets so nested routes still load.\n\n### Backward-compatible layout\n\n- Use **`--legacy-flat-pages`** if you rely on older behavior with every HTML file in the hostname root (`_underscore.html`). Default is hierarchical.\n\n---\n\n## Engine modes (`--mode`)\n\n| Mode | Use when | Browser install? |\n|------|----------|------------------|\n| `auto` (default) | Unknown site; picks static or render | Only if SPA detected |\n| `static` | Blogs, docs, plain HTML | **No** |\n| `render` | SPAs, React, Vue, Vite, heavy JS | **Yes** — Playwright + Chromium (auto on install) |\n\n```bash\nanydownload https://example.com --mode static\nanydownload https://spa-app.com --mode render\nanydownload https://example.com                    # auto\nanydownload https://spa-app.com -d                 # same as --mode render\n```\n\n**Render mode** saves assets from the browser’s network capture (not plain HTTP re-download). Use `--wait 5000` if images load late (e.g. backgrounds).\n\n```bash\nanydownload example.com --mode render --wait 5000 -v\n```\n\n---\n\n## Presets\n\n| Preset | What it does |\n|--------|----------------|\n| `page` (default) | One page + all assets on that page |\n| `full` | Same-hostname links, depth 2, sitemap |\n| `mirror` | Deep crawl, depth 5, sitemap |\n\n\u003e **Mirror** follows links on the **exact same hostname** only (not `sub.example.com`). For single-page portfolios (SPA), prefer **`page`** preset — mirror crawls many routes and is slow on portfolio sites.\n\n```bash\nanydownload https://example.com --preset full -o mysite\n```\n\n---\n\n## Common options\n\n| Option | Description | Default |\n|--------|-------------|---------|\n| `-o, --output \u003cdir\u003e` | Output parent folder (files go in `\u003cdir\u003e/\u003chostname\u003e/`) | `downloaded_site` |\n| `--mode \u003cmode\u003e` | `static` \\| `render` \\| `auto` | `auto` |\n| `--open` | Start HTTP preview + open browser | off |\n| `--serve` | Start HTTP preview after download | off |\n| `--serve-port \u003cn\u003e` | Preview port (download command) | `8765` |\n| `--wait \u003cms\u003e` | Extra wait after page load (**render** only) | `2000` |\n| `-v, --verbose` | Verbose logs | off |\n| `--preset \u003cname\u003e` | `page` \\| `full` \\| `mirror` | `page` |\n| `-r, --recursive` | Follow same-**hostname** links | preset |\n| `-m, --max-depth \u003cn\u003e` | Crawl / path-discovery depth | `1` |\n| `--type \u003ctype\u003e` | Filter assets: `all` \\| `image` \\| `css` \\| `js` \\| `html` \\| `media` \\| `font` | `all` |\n| `--sitemap` | Use sitemap when crawling + write `sitemap.xml.gz` | off |\n| `--delay \u003cms\u003e` | Delay between requests (path probe / download) | `500` |\n| `--concurrency \u003cn\u003e` | Parallel asset downloads | `5` |\n| `--filter \u003cregex\u003e` | Only URLs matching regex | — |\n| `-d, --dynamic` | Same as `--mode render` | off |\n| `--legacy-flat-pages` | Flat HTML in hostname root (`_learn.html` layout) | off |\n\n**Render-only:** `--browser` (`playwright` default, or `puppeteer`), `--browser-engine`, `--headless`\n\n**`serve` subcommand:** `-p, --port \u003cn\u003e` — preview server port (default `8765`)\n\nFull list: `anydownload --help`\n\n### Path discovery (optional, `-p`)\n\n**Auxiliary only** — does not download the site. Writes **`paths.txt`** under `\u003coutput\u003e/\u003chostname\u003e/` by combining sitemap, crawl, robots hints, JS/service-worker strings, and optional HTTP probes.\n\n```bash\nanydownload example.com -p -o mysite\n# → mysite/example.com/paths.txt\n```\n\n| Flag | When to use |\n|------|-------------|\n| `--path-deep` | Bundled ~2000-path wordlist + Wayback (slow; use `--delay`) |\n| `--path-seeds \u003cfile\u003e` | One guessed path per line (e.g. secret routes you know) |\n| `--path-probe-depth 2` | Also probe `/found-prefix/word` (capped) |\n| `--path-txt` / `./path.txt` | Replace bundled [`data/path-wordlist.txt`](data/path-wordlist.txt) |\n| `--path-no-render` | Faster scan without Playwright |\n\nExample (deeper scan): `anydownload example.com -p --path-deep --delay 500 -o mysite`. Paths with no public signal still need **`--path-seeds`**. Full flags: `anydownload --help`.\n\n---\n\n## Project limitations\n\nAnyDownload builds **offline-browsable mirrors**. It is **not** a universal “download anything from the internet” tool.\n\n### Works well\n\n- Public HTTP(S) pages and same-origin assets\n- Static sites, blogs, documentation\n- Many SPAs with `--mode render` + HTTP preview\n- HTML, CSS, JS (including `type=\"module\"`), images, fonts, preload/modulepreload\n- WASM, JSON, webp/avif, media when captured in render mode\n\n### Does not work (or unreliable)\n\n| Case | Why |\n|------|-----|\n| Login / paywall | No credentials unless you pass cookies |\n| DRM video (Netflix, etc.) | Encrypted streams |\n| CAPTCHA / bot protection | Needs human verification |\n| WebSocket / live streams | Not a static file |\n| `blob:` / `data:` URLs | Skipped by design |\n| Infinite scroll without scrolling | Content never loads |\n| Double-clicking `index.html` | `file://` breaks ES modules → white screen |\n| “Download every file on the internet” | Out of scope |\n\nThemes, locale switching, or client bundles that lazy-load translations from CDN may behave differently offline—even when `_next`/React chunks load locally. Interactive features aren’t guaranteed. External links (`https://`, other hostnames, e.g. GitHub) intentionally stay absolute so browsers can reach the live network when available.\n\nOptional failures (e.g. `favicon`, `banner.png`, cross-origin CDN fonts) may be reported in verbose mode but do not increment the failed count or block the main page.\n\n---\n\n## Output layout\n\nFiles are saved under **`\u003coutput-folder\u003e/\u003chostname\u003e/`**, not directly in the output folder root:\n\n```\ndownloaded_site/                          ← folder you pass with -o or wizard\n└── example.com/               ← hostname subfolder (always created)\n    ├── index.html\n    ├── paths.txt              ← when using -p / --path\n    ├── sitemap.xml.gz         ← when --sitemap is used\n    └── assets/\n        ├── index-xxxxx.js\n        └── index-xxxxx.css\n```\n\nIf you choose output `test`, the site lives at `test/example.com/`. A separate default `downloaded_site/` folder is only used when you omit `-o` entirely (not from wizard defaults leaking into CLI).\n\n---\n\n## Desktop App (Executable)\n\nAnyDownload can be compiled into a standalone desktop application (**Windows portable `.exe`**, **macOS `.dmg`**, **Linux `.AppImage`**) using Electron. The app bundles the Web GUI and Chromium directly, meaning users do not need Node.js or Playwright separately.\n\nOn **Windows**, the release is a **single portable executable**: download and double-click — no setup wizard (`NSIS`).\n\nTo build locally (match the OS you are running):\n\n```bash\n# Install dependencies\nnpm install\n\n# Start in development mode\nnpm run electron:start\n\n# Build for Windows (portable single .exe, no installer)\nnpm run electron:build:win\n\n# Build for macOS (.dmg — open disk image, drag/run the app or run from mounted volume)\nnpm run electron:build:mac\n\n# Build for Linux (.AppImage — chmod +x then run)\nnpm run electron:build:linux\n\n# Same as electron-builder for the CURRENT platform only (does not emit all three OSes locally)\nnpm run electron:build:all\n```\n\nBuilt binaries appear under **`dist-electron/`**. Release assets use predictable names (**`AnyDownload-Windows-\u003cversion\u003e.exe`**, **`AnyDownload-macOS-\u003cversion\u003e-\u003carch\u003e.dmg`**, **`AnyDownload-Linux-\u003cversion\u003e-\u003carch\u003e.AppImage`**).\n\n### GitHub Releases (automatic)\n\nEach **push to `main`** runs [.github/workflows/release-electron.yml](.github/workflows/release-electron.yml) and creates a **new GitHub Release** with **Windows portable `.exe`**, **macOS `.dmg`**, and **Linux `.AppImage`** attachments. Release **version tagging** bumps the **MINOR semver** (`x.y.z` → `x.(y+1).0`; e.g. `2.0.0` → `2.1.0`) using the **[latest Release](https://github.com/HenryLok0/AnyDownload/releases)** tag as baseline, or `package.json` `version` if no release exists yet.\n\nEach Release includes:\n\n- Windows portable `.exe` / macOS `.dmg` / Linux `.AppImage` produced in CI  \n- **`AnyDownload-\u003cversion\u003e-source.zip`** (`git archive` of the triggering commit — tracked sources only)\n\nGitHub also shows **Source code (zip)** and **Source code (tar.gz)** for the Release tag automatically in the Releases UI — those are maintained by GitHub alongside the desktop binaries.\n\n**macOS code signing / notarization:** CI produces an unsigned `.dmg` by default. Wide distribution typically requires Apple Developer **`CSC_`*** env secrets and optional notarization; that is advanced setup and not required for those builds to attach to the Release.\n\n---\n\n## Contributing\n\nWe welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n### Contributors\n\n\u003ca href=\"https://github.com/HenryLok0/AnyDownload/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=HenryLok0/AnyDownload\" /\u003e\n\u003c/a\u003e\n\n## License\n\nMIT License - see [LICENSE](LICENSE) for details.\n\n## Support\n\n- GitHub Issues: [Open an issue](https://github.com/HenryLok0/AnyDownload/issues)\n\n## Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=HenryLok0/AnyDownload\u0026type=Date)](https://star-history.com/#HenryLok0/AnyDownload\u0026Date)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhenrylok0%2Fanydownload","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhenrylok0%2Fanydownload","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhenrylok0%2Fanydownload/lists"}