{"id":15130346,"url":"https://github.com/yunatamos/source-maps-downloader","last_synced_at":"2026-01-20T14:07:08.218Z","repository":{"id":255648925,"uuid":"853317479","full_name":"yunatamos/source-maps-downloader","owner":"yunatamos","description":"This Node.js script automates the process of downloading and extracting source maps from websites. It uses Puppeteer to navigate web pages, captures all JavaScript files (including lazy-loaded ones), processes their source maps to reconstruct the original source files, and optionally records the browser session.","archived":false,"fork":false,"pushed_at":"2025-11-09T03:04:16.000Z","size":48,"stargazers_count":6,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-09T03:29:17.033Z","etag":null,"topics":["react","reactjs","resources","webpack"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yunatamos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-06T12:21:17.000Z","updated_at":"2025-11-09T03:04:19.000Z","dependencies_parsed_at":"2024-12-19T02:22:00.613Z","dependency_job_id":"1a0abb06-dfe1-42ac-9bb8-6223ab3c5432","html_url":"https://github.com/yunatamos/source-maps-downloader","commit_stats":null,"previous_names":["yunatamos/source-maps-downloader"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/yunatamos/source-maps-downloader","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yunatamos%2Fsource-maps-downloader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yunatamos%2Fsource-maps-downloader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yunatamos%2Fsource-maps-downloader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yunatamos%2Fsource-maps-downloader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yunatamos","download_url":"https://codeload.github.com/yunatamos/source-maps-downloader/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yunatamos%2Fsource-maps-downloader/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28604712,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T12:01:53.233Z","status":"ssl_error","status_checked_at":"2026-01-20T12:01:46.545Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["react","reactjs","resources","webpack"],"created_at":"2024-09-26T02:46:04.687Z","updated_at":"2026-01-20T14:07:08.213Z","avatar_url":"https://github.com/yunatamos.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Source Map Downloader\n\nAn intelligent Node.js tool that extracts source maps from modern web applications (React, Vue, Angular, etc.). Unlike simple network sniffers, this tool **parses JavaScript bundles to discover ALL chunks**, including lazy-loaded ones that may never be triggered during a normal browsing session.\n\n## Features\n\n- **Intelligent Chunk Discovery**: Parses main/runtime JS files to find ALL chunk references\n- **Comprehensive Coverage**: Finds lazy-loaded chunks that traditional network monitoring misses\n- **Multi-Framework Support**: Works with Webpack, Vite, Rollup, and other bundlers\n- **React-Optimized**: Especially effective at extracting React app source maps\n- **Network Monitoring**: Also captures JS files loaded via network requests\n- **Source Reconstruction**: Extracts original source files from source maps\n- **Optional Screen Recording**: Record browser session for debugging\n- **Organized Output**: Saves source maps, original sources, and compiled JS separately\n\n## Prerequisites\n\nBefore you begin, ensure you have met the following requirements:\n\n- You have installed Node.js (version 18 or later recommended)\n- You have a basic understanding of JavaScript and Node.js\n\n## Installing Source Map Downloader\n\nTo install the Source Map Downloader, follow these steps:\n\n1. Clone the repository or download the script.\n2. Navigate to the project directory.\n3. Install the required dependencies:\n\n```bash\nyarn install\n# or\nnpm install\n```\n\n## Using Source Map Downloader\n\nTo use the Source Map Downloader, follow these steps:\n\n1. Open a terminal/command prompt.\n2. Navigate to the directory containing the script.\n3. Run the script with the following command:\n\n```\nnode index.js --url https://example.com [--record]\n```\n\nReplace `https://example.com` with the URL of the website you want to download source maps from.\n\n### Command-line Options\n\n- `--url \u003cwebsite-url\u003e`: (Required) Specify the website URL to process.\n- `--record`: (Optional) Enable screen recording of the browser session.\n\n### Examples\n\n```bash\n# Basic usage - extract source maps from a React app\nnode index.js --url https://react-app.example.com\n\n# With screen recording\nnode index.js --url https://app.example.com --record\n\n# Extract from a production site\nnode index.js --url https://dashboard.company.com\n```\n\n## How it works\n\n1. **Browser Launch**: Opens the target URL using Puppeteer with a realistic user agent.\n2. **Network Monitoring**: Captures all JS files loaded via network requests.\n3. **Main File Discovery**: Identifies main/runtime JS files that contain chunk manifests (prioritizing files with keywords like 'runtime', 'main', 'vendor').\n4. **Chunk Parsing**: Analyzes main JS files using regex patterns to extract ALL chunk references:\n   - Webpack chunk loading patterns\n   - Dynamic import() statements\n   - Chunk manifests and mappings\n   - Static script references\n5. **Comprehensive Collection**: Combines discovered chunks with network-captured files.\n6. **Source Map Extraction**: For each JS file:\n   - Downloads the file\n   - Finds source map reference (`//# sourceMappingURL=...`)\n   - Downloads the source map\n   - Extracts all original source files\n7. **Organized Storage**: Saves everything in structured directories:\n   - `output/{hostname}/sourcemaps/` - Source map files\n   - `output/{hostname}/sources/` - Original source code\n   - `output/{hostname}/compiled/` - Compiled/bundled JS files\n8. **Optional Recording**: If `--record` is enabled, saves browser session video.\n\n## Directory Structure\n\n```\noutput/\n└── {hostname}/\n    ├── sourcemaps/     # Source map JSON files\n    │   ├── main.js.map\n    │   └── chunk-123.js.map\n    ├── sources/        # Original source code (reconstructed)\n    │   ├── src/\n    │   │   ├── App.jsx\n    │   │   └── components/\n    │   └── node_modules/\n    └── compiled/       # Compiled/bundled JavaScript\n        ├── main.js\n        └── chunk-123.js\n\nscreenRecordings/       # Browser session videos (if --record used)\n└── screen-recording-{hostname}-{timestamp}.mp4\n```\n\n## Notes\n\n- **Chunk Discovery**: This tool is especially powerful for React apps and other single-page applications that use code splitting. It will find chunks that may never load during normal browsing.\n- **Source Maps Required**: The tool can only extract source code if source maps are publicly accessible. Many production sites remove source maps.\n- **Multiple Bundlers**: Works with Webpack, Vite, Rollup, Parcel, and other modern bundlers.\n- **Output Organization**: Files are organized by hostname in the `output/` directory.\n- **Network + Parsing**: Combines network monitoring with intelligent JS parsing for comprehensive coverage.\n\n## Troubleshooting\n\nIf you encounter any issues:\n\n1. **No source maps found**: Many production websites remove source maps. Try development/staging environments.\n2. **Timeout errors**: Increase the timeout value in the code or try a faster network connection.\n3. **Missing chunks**: Some apps may use non-standard chunk naming. The regex patterns can be extended.\n4. **Screen recording issues**: Ensure ffmpeg is installed if using the `--record` option.\n5. **403/401 errors**: Some sites require authentication or block automated requests.\n\n### Common Issues\n\n**Q: The tool only finds a few chunks, but I know there are more**  \nA: Check if the site uses a custom bundler or non-standard chunk naming. You may need to add custom regex patterns to `extractChunkReferences()`.\n\n**Q: Source maps download but extraction fails**  \nA: Some source maps may be malformed or use unsupported formats. Check the console output for specific errors.\n\n**Q: Getting 404 errors for chunk files**  \nA: The chunk URLs might be relative and need different base URL resolution. Check the console to see which URLs are failing.\n\n## Contributing to Source Map Downloader\n\nTo contribute to Source Map Downloader, follow these steps:\n\n1. Fork this repository.\n2. Create a branch: `git checkout -b \u003cbranch_name\u003e`.\n3. Make your changes and commit them: `git commit -m '\u003ccommit_message\u003e'`\n4. Push to the original branch: `git push origin \u003cproject_name\u003e/\u003clocation\u003e`\n5. Create the pull request.\n\nAlternatively, see the GitHub documentation on [creating a pull request](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request).\n\n## License\n\nThis project uses the following license: [MIT License](\u003clink_to_license\u003e).\n\n## Contact\n\nIf you want to contact me, you can reach me at \u003cyunatamos@example.com\u003e.\n\n## Disclaimer\n\nThis tool is for educational purposes only. Always respect website terms of service and robots.txt files. Ensure you have permission before downloading source maps from any website.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyunatamos%2Fsource-maps-downloader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyunatamos%2Fsource-maps-downloader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyunatamos%2Fsource-maps-downloader/lists"}