{"id":20597104,"url":"https://github.com/tradle/dedupe-deps","last_synced_at":"2025-07-24T01:07:10.787Z","repository":{"id":66294676,"uuid":"174760941","full_name":"tradle/dedupe-deps","owner":"tradle","description":"DEPRECATED rm duplicates in your node_modules","archived":false,"fork":false,"pushed_at":"2019-08-11T12:30:52.000Z","size":16,"stargazers_count":4,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-14T10:38:05.745Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tradle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-10T00:27:05.000Z","updated_at":"2019-08-11T12:30:53.000Z","dependencies_parsed_at":"2023-05-10T17:00:21.911Z","dependency_job_id":null,"html_url":"https://github.com/tradle/dedupe-deps","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/tradle/dedupe-deps","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tradle%2Fdedupe-deps","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tradle%2Fdedupe-deps/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tradle%2Fdedupe-deps/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tradle%2Fdedupe-deps/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tradle","download_url":"https://codeload.github.com/tradle/dedupe-deps/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tradle%2Fdedupe-deps/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266775383,"owners_count":23982273,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T08:20:20.655Z","updated_at":"2025-07-24T01:07:10.763Z","avatar_url":"https://github.com/tradle.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dedupe-deps\n\nDEPRECATED\n\nUPDATE: the current implementation has issues with files that have dependencies (`require`/`import` statements). Due to the structure of `node_modules/`, an identical file living in two locations can pull in different copies of dependencies.\n\nsee/follow a less opinionated one here: https://github.com/ExodusMovement/find-duplicates\n\n## TL;DR\n\nproblem: multiple copies of the same version of a dependency in both your fs and your bundle\n\nsolution: replace all duplicates with `module.exports = require('../path/to/cananocal/file.js')`, right in the filesystem, so bundlers (webpack/browserify/metro) don't need to worry about it\n\n## Problem\n\nBecause of the way package managers (npm/yarn) save deps in node_modules, you can end up with the same version of the same dependency occuring multiple times in your dependency tree. Bundlers like webpack, browserify and the React Native packager, then all have to solve that problem of preventing duplicates from being bundled. It's not that solutions don't exist, it's that they have to re-invented in every new ecosystem.\n\nSee for example this [webpack issue](https://github.com/webpack/webpack/issues/5593), opened in 2017...and still open.\n\n## Solution\n\nThe solution is somewhat dirty. Avert your eyes. We're going to deduplicate directly in the filesystem. Let's say we have multiple copies of super-awesome in our tree:\n\n```sh\n# version 1.1.0\nnode_modules/a/node_modules/super-awesome/index.js\n# also version 1.1.0\nnode_modules/b/node_modules/super-awesome/index.js\n```\n\nWe're going to choose the first one as the canonical super-awesome@1.1.0 and replace js files in the duplicate with `module.exports = require('../path/to/cananocal/file.js')`\n\n## Install\n\n```sh\nnpm install --save-dev dedupe-deps\n```\n\n## Usage\n\n\n```sh\n# run in project root\n# \"DEBUG=dedupe-deps\" is optional, if you want to see what files are being deduped\nDEBUG=dedupe-deps dedupe-deps --dir .\n```\n\nTo do a dry-run, add a `--dry-run` flag\n\nOr put it in your package.json, e.g.:\n\n```js\n  \"scripts\": {\n    \"dedupe-deps\": \"dedupe-deps --dir .\",\n    \"postinstall\": \"npm run dedupe-deps\"\n  }\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftradle%2Fdedupe-deps","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftradle%2Fdedupe-deps","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftradle%2Fdedupe-deps/lists"}