{"id":18952441,"url":"https://github.com/devanshbatham/heaptruffle","last_synced_at":"2025-04-16T01:34:00.221Z","repository":{"id":186659813,"uuid":"675411968","full_name":"devanshbatham/heaptruffle","owner":"devanshbatham","description":"Mine URLs from Browser's Heap Snapshot for fun and profit ","archived":false,"fork":false,"pushed_at":"2023-08-09T09:02:33.000Z","size":319,"stargazers_count":64,"open_issues_count":0,"forks_count":16,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-29T05:12:20.698Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devanshbatham.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-06T20:28:56.000Z","updated_at":"2025-03-26T13:23:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"85f553b1-4031-4544-8011-a4b276ce375f","html_url":"https://github.com/devanshbatham/heaptruffle","commit_stats":null,"previous_names":["devanshbatham/heaptruffle"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devanshbatham%2Fheaptruffle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devanshbatham%2Fheaptruffle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devanshbatham%2Fheaptruffle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devanshbatham%2Fheaptruffle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devanshbatham","download_url":"https://codeload.github.com/devanshbatham/heaptruffle/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249182565,"owners_count":21226090,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T13:33:25.925Z","updated_at":"2025-04-16T01:33:59.934Z","avatar_url":"https://github.com/devanshbatham.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n    heaptruffle\n  \u003cbr\u003e\n\u003c/h1\u003e\n\n\u003ch4 align=\"center\"\u003eMine URLs from Browser's Heap Snapshot for fun and profit \u003c/h4\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#install\"\u003e🏗️ Install\u003c/a\u003e  \n  \u003ca href=\"#usage\"\u003e⛏️ Usage\u003c/a\u003e \n  \u003ca href=\"#how-it-works\"\u003e💡 How it Works\u003c/a\u003e  \n  \u003ca href=\"#inspiration\"\u003e⚡ Inspiration\u003c/a\u003e \n  \u003cbr\u003e\n\u003c/p\u003e\n\n\n![heaptruffle](https://github.com/devanshbatham/heaptruffle/blob/main/static/truffleheap.png?raw=true)\n\n# Installation\n\nFollow these steps to get `heaptruffle` up and running:\n\n1. **Clone the Repository**:\n   ```sh\n   git clone https://github.com/devanshbatham/heaptruffle\n   ```\n\n2. **Navigate to the Directory**:\n   ```sh\n   cd heaptruffle\n   ```\n\n3. **Build the Docker Image**:\n   ```sh\n   docker build -t heaptruffle .\n   ```\n\n4. **Make the script executable and move it to a directory in your PATH**:\n   ```sh\n   sudo chmod +x heaptruffle\n   sudo mv heaptruffle /usr/local/bin/heaptruffle\n   ```\n\nOnce done, you can invoke `heaptruffle` from any location in your terminal.\n\n# Usage\n\n### Using Docker:\n\n- To run heaptruffle on single URL\n  ```sh\n  docker run -it --rm -v \"$PWD\":/app/data --name heaptruffle-container heaptruffle --url http://example.com\n  ```\n\n- or, to run it on a file containing URLs. \n  ```sh\n  docker run -it --rm -v \"$PWD\":/app/data --name heaptruffle-container heaptruffle --list urls.txt\n  ```\n\n- Save the output to a file (output.txt):\n  ```sh\n  docker run -it --rm -v \"$PWD\":/app/data --name heaptruffle-container heaptruffle --url http://example.com --output /app/data/output.txt\n  ```\n\n- Increase concurrency to fetch URLs faster:\n  ```sh\n  docker run -it --rm -v \"$PWD\":/app/data --name heaptruffle-container heaptruffle --list urls.txt --concurrency 10\n  ```\n\n### Using heaptruffle alias (after installation):\n\n- To run heaptruffle:\n  ```sh\n  heaptruffle --url https://example.com\n  ```\n\n- or \n  ```sh\n  heaptruffle --list urls.txt\n  ```\n\n- Increase concurrency to fetch URLs faster:\n  ```sh\n  heaptruffle --list urls.txt --concurrency 10\n  ```\n\n- Save the output to a file (output.txt):\n  ```sh\n  heaptruffle --url https://example.com --output output.txt\n  ```\n\n- Use silent mode to suppress the ASCII banner:\n  ```sh\n  heaptruffle --url https://example.com --silent\n  ```\n\n## Options\n\n| Option          | Alias | Type     | Description                                                   |\n|-----------------|-------|----------|---------------------------------------------------------------|\n| `--url`         | `-u`  | `string` | URL address                                                   |\n| `--list`        | `-l`  | `string` | File containing list of URLs                                  |\n| `--concurrency` | `-c`  | `number` | Number of URLs to fetch concurrently (default: 5)             |\n| `--silent`      | `-s`  | `boolean`| Silent mode, does not display the ASCII banner (default: false)|\n| `--output`      | `-o`  | `string` | File to save the output                                        |\n\n# How it Works\n\nheaptruffle uses Puppeteer, a headless browser automation library, to load web pages and capture heap snapshots of the web pages' memory. These heap snapshots are then parsed using the `heapsnapshot-parser` library, allowing heaptruffle to extract URLs/endpoints from it.\n\nThe tool takes either a single URL or a file containing a list of URLs as input. It fetches each URL concurrently to speed up the process. For each URL, heaptruffle loads the web page, captures a heap snapshot, and then performs analysis to extract relevant paths from the snapshot. It identifies the URLs and paths accessed during the page's execution and outputs them to the console or a specified output file.\n\n## Inspiration\nThis tool was inspired by the project [extract-relative-url-heapsnapshot](https://github.com/smiegles/extract-relative-url-heapsnapshot) by [smiegles](https://github.com/smiegles). I just improved it in my way and extended its functionality (concurrency, support for multiple URLs, pretty output, the ability to save the results in a file, dockerization, error handling, an easy-to-use setup script, etc).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevanshbatham%2Fheaptruffle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevanshbatham%2Fheaptruffle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevanshbatham%2Fheaptruffle/lists"}