{"id":15408931,"url":"https://github.com/antongolub/lockfile","last_synced_at":"2025-04-19T03:12:33.602Z","repository":{"id":194521860,"uuid":"435501462","full_name":"antongolub/lockfile","owner":"antongolub","description":"Read and write lockfiles with reasonable losses","archived":false,"fork":false,"pushed_at":"2023-12-15T15:36:21.000Z","size":4156,"stargazers_count":4,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-23T08:16:04.436Z","etag":null,"topics":["lockfile","npm","yarn"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antongolub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-06T13:10:48.000Z","updated_at":"2024-03-27T17:35:17.000Z","dependencies_parsed_at":"2024-10-20T13:50:54.211Z","dependency_job_id":null,"html_url":"https://github.com/antongolub/lockfile","commit_stats":{"total_commits":147,"total_committers":2,"mean_commits":73.5,"dds":"0.013605442176870763","last_synced_commit":"aa3aa7dab7859fd6469dc5bee8ff57642d624e8c"},"previous_names":["antongolub/lockfile"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antongolub%2Flockfile","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antongolub%2Flockfile/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antongolub%2Flockfile/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antongolub%2Flockfile/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antongolub","download_url":"https://codeload.github.com/antongolub/lockfile/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249598181,"owners_count":21297464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lockfile","npm","yarn"],"created_at":"2024-10-01T16:36:01.388Z","updated_at":"2025-04-19T03:12:33.585Z","avatar_url":"https://github.com/antongolub.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# @antongolub/lockfile\n\u003e Read and write lockfiles with reasonable losses\n\n\u003cp\u003e\u003cimg alt=\"@antongolub/lockfile\" src=\"./pics/pic.png\" align=\"right\" width=\"300\"\u003e\nEach package manager brings its own philosophy of how to describe, store and control project dependencies.\nIt \u003ci\u003eseems\u003c/i\u003e acceptable for developers, but literally becomes a \u003cstrike\u003epain in *** ***\u003c/strike\u003e headache for isec, devops and release engineers.\nThis lib is a naive attempt to build a pm-independent, generic, extensible and reliable deps representation.\n\nThe `package.json` manifest contains its own deps requirements, the `lockfile` holds the deps resolution snapshot\u003csup\u003e*\u003c/sup\u003e,\nso both of them are required to build a dependency graph. We can try to convert this data into a normalized representation for further analysis and processing (for example, to fix vulnerabilities).\nAnd then, if necessary, try convert it back to the original/another format.\n\u003c/p\u003e\n\n## Status\nProof of concept. The API may change significantly ⚠️\n\n## Getting started\n### Install\n```shell\nyarn add @antongolub/lockfile@snapshot\n```\n\n## Usage\n_tl;dr_\n```ts\nimport fs from 'fs/promises'\nimport {parse, analyze} from '@antongolub/lockfile'\n\nconst lf = await fs.readFile('yarn.lock', 'utf-8')\nconst pkg = await fs.readFile('package.json', 'utf-8')\n\nconst snapshot = parse(lf, pkg) // Holds JSON-friendly TEntries[]\nconst idx = analyze(snapshot)   // An index to represent repo dep graphs\n\n// idx.entries\n// idx.prod\n// idx.edges\n```\n\n## API\n### JS/TS\n```ts\nimport { parse, format, analyze, convert } from '@antongolub/lockfile'\n\nconst lf = await fs.readFile('yarn.lock', 'utf-8')\nconst pkgJson = await fs.readFile('package.json', 'utf-8')\nconst snapshot = parse(lf, pkgJson)\n\nconst lf1 = format(snapshot)\nconst lf2 = format(snapshot, 'npm-1')         // Throws err: npm v1 meta does not support workspaces\n\nconst meta = await readMeta()                 // reads local package.jsons data to gather required data like `engines`, `license`, `bins`, etc\nconst meta2 = await fetchMeta(snapshot)       // does the same, but from the remote registry\nconst lf3 = format(snapshot, 'npm-3', {meta}) // format with options\n\nconst idx = analyze(snapshot)\nidx.edges\n// [\n//  [ '', '@antongolub/npm-test@4.0.1' ],\n//  [ '@antongolub/npm-test@4.0.1', '@antongolub/npm-test@3.0.1' ],\n//  [ '@antongolub/npm-test@3.0.1', '@antongolub/npm-test@2.0.1' ],\n//  [ '@antongolub/npm-test@2.0.1', '@antongolub/npm-test@1.0.0' ]\n// ]\n\nconst lf4 = await convert(lf, pkgJson, 'yarn-berry')\n```\n\n### CLI\n```shell\nnpx @antongolub/lockfile@snapshot \u003ccmd\u003e [options]\n\nnpx @antongolub/lockfile@snapshot parse --input=yarn.lock,package.json --output=snapshot.json\nnpx @antongolub/lockfile@snapshot format --input=snapshot.json --output=yarn.lock\n```\n\n| Command / Option | Description                                                                           |\n|------------------|---------------------------------------------------------------------------------------|\n| `parse`          | Parses lockfiles and package manifests into a snapshot                                |\n| `format`         | Formats a snapshot into a lockfile                                                    |\n| `convert`        | Converts a lockfile into another format. Shortcut for `parse` + `format`              |\n| `--input`        | A comma-separated list of files to parse: `snapshot.json` or `yarn.lock,package.json` |\n| `--output`       | A file to write the result to: `snapshot.json` or `yarn.lock`                         |\n| `--format`       | A lockfile format: `npm-1`, `npm-2`, `npm-3`, `yarn-berry`, `yarn-classic`            |\n\n### Terms\n`nmtree` — fs projection of deps, directories structure  \n`deptree` — bounds full dep paths with their resolved packages  \n`depgraph` — describes how resolved pkgs are related with each other\n\n### Lockfiles types\n| Package manager      | Meta format | Read | Write |\n|----------------------|-------------|------|-------|\n| npm \u003c7               | 1           | ✓    | ✓     |\n| npm \u003e=7              | 2           | ✓    |       |\n| npm \u003e=9              | 3           | ✓    |       | \n| yarn 1 (classic)     | 1           | ✓    | ✓     |\n| yarn 2, 3, 4 (berry) | 5, 6, 7     | ✓    | ✓     |\n\n### Dependency protocols\n| Type      | Supported | Example                                 | Description                                                    |\n|-----------|-----------|-----------------------------------------|----------------------------------------------------------------|\n| semver    | ✓         | `^1.2.3`                                | Resolves from the default registry                             |\n| tag       |           | `latest`                                | Resolves from the default registry                             |\n| npm       | ✓         | `npm:name@...`                          | Resolves from the npm registry                                 |\n| git       |           | `git@github.com:foo/bar.git`            | Downloads a public package from a Git repository               |\n| github    |           | `github:foo/bar`                        | Downloads a public package from GitHub                         |\n| github    | ✓         | `foo/bar`                               | Alias for the github: protocol                                 |\n| file      |           | `file:./my-package`                     | Copies the target location into the cache                      |\n| link      |           | `link:./my-folder`                      | Creates a link to the ./my-folder folder (ignore dependencies) |\n| patch     | _limited_ | `patch:left-pad@1.0.0#./my-patch.patch` | Creates a patched copy of the original package                 |\n| portal    |           | `portal:./my-folder`                    | Creates a link to the ./my-folder folder (follow dependencies) |\n| workspace | _limited_ | `workspace:*`                           | Creates a link to a package in another workspace               |\n\nhttps://v3.yarnpkg.com/features/protocols  \nhttps://yarnpkg.com/protocols  \nhttps://docs.npmjs.com/cli/v10/configuring-npm/package-json#dependencies\n\n### `TSnapshot`\n```ts\nexport type TSnapshot = Record\u003cstring, TEntry\u003e\n\nexport type TEntry = {\n  name:       string\n  version:    string\n  ranges:     string[]\n  hashes:     {\n    sha512?:  string\n    sha256?:  string\n    sha1?:    string\n    checksum?: string\n    md5?:     string\n  }\n  source:     {\n    type:     TSourceType // npm, workspace, gh, patch, etc\n    id:       string\n    registry?: string\n  }\n  // optional pm-specific lockfile meta\n  manifest?:              TManifest\n  conditions?:            string\n  dependencies?:          TDependencies\n  dependenciesMeta?:      TDependenciesMeta\n  devDependencies?:       TDependencies\n  optionalDependencies?:  TDependencies\n  peerDependencies?:      TDependencies\n  peerDependenciesMeta?:  TDependenciesMeta\n  bin?:                   Record\u003cstring, string\u003e\n  engines?:               Record\u003cstring, string\u003e\n  funding?:               Record\u003cstring, string\u003e\n}\n```\n\n### `TSnapshotIndex`\n```ts\nexport interface TSnapshotIndex {\n  snapshot: TSnapshot\n  entries:  TEntry[]\n  roots:    TEntry[]\n  edges:    [string, string][]\n  tree:       Record\u003cstring, {\n    key:      string\n    chunks:   string[]\n    parents:  TEntry[]\n    id:       string\n    name:     string\n    version:  string\n    entry:    TEntry\n    depth:    number // the lowest level where the dep@ver first time occurs\n  }\u003e\n  prod: Set\u003cTEntry\u003e\n  getEntryId ({name, version}: TEntry): string\n  getEntry (name: string, version?: string): TEntry | undefined,\n  getEntryByRange (name: string, range: string): TEntry | undefined\n  getEntryDeps(entry: TEntry): TEntry[]\n}\n```\n\n### Caveats\n* There is an infinite number of `nmtrees` that corresponds to the specified `deptree`, but among them there is a finite set of effective (sufficient) for the target criterion — for example, nesting, size, homogeneity of versions\n* npm1: `optional: true` label is not supported yet\n* yarn berry: no idea how to resolve and inject PnP patches https://github.com/yarnpkg/berry/tree/master/packages/plugin-compat\n* npm2 and npm3 requires `engines` and `funding` data, while yarn* or npm1 does not contain it\n* many `nmtree` projections may correspond to the specified `depgraph`\n* pkg.json `resolutions` and `overrides` directives are completely ignored for now\n* pkg aliases are not _fully_ supported yet [#2](https://github.com/antongolub/lockfile/issues/2#issuecomment-1786613893)\n\n### Snippets\nExtracts all deps by depth:\n```ts\nconst getDepsByDepth = (idx: TSnapshotIndex, depth = 0) =\u003e Object.values(idx.tree)\n  .filter(({depth: d}) =\u003e d === depth)\n  .map(({entry}) =\u003e entry)\n```\n\nGet the longest dep chain:\n```ts\nconst getLongestChain = (): TEntry[] =\u003e {\n  let max = 0\n  let chain: TEntry[] = []\n\n  for (const e of Object.values(idx.tree)) {\n    if (e.depth \u003e max) {\n      max = e.depth\n      chain = [...e.parents, e.entry]\n    }\n  }\n  return chain\n}\n\nconstole.log(\n  getLongestChain()\n    .map((e) =\u003e idx.getEntryId(e))\n    .join(' -\u003e ')\n)\n```\n\n### Inspired by\n* [synp](https://github.com/imsnif/synp)\n* [snyk-nodejs-lockfile-parser](https://github.com/snyk/nodejs-lockfile-parser)\n* [yarn](https://github.com/yarnpkg/yarn/blob/master/src/lockfile/parse.js)\n* [yarn-lockfile](https://github.com/yarnpkg/yarn/tree/master/packages/lockfile)\n\n### Refs\n* [yarn.lock](https://classic.yarnpkg.com/en/docs/yarn-lock/)\n* [package-lock-json](https://docs.npmjs.com/cli/v10/configuring-npm/package-lock-json)\n* [what-is-package-lock-json](https://snyk.io/blog/what-is-package-lock-json/)\n* [the-ultimate-guide-to-yarn-lock-lockfiles](https://www.arahansen.com/the-ultimate-guide-to-yarn-lock-lockfiles/)\n* [package-lock-json-the-complete-guide](https://medium.com/helpshift-engineering/package-lock-json-the-complete-guide-2ae40175ebdd)\n\u003cdetails\u003e\n  \u003csummary\u003emore\u003c/summary\u003e\n\n* [cvent/pnpm-lock-export](https://github.com/cvent/pnpm-lock-export)\n* [why-keep-package-lockjson](https://blog.npmjs.org/post/621733939456933888/npm-v7-series-why-keep-package-lockjson.html)\n* [pnpm/lockfile-utils](https://github.com/pnpm/pnpm/tree/main/lockfile/lockfile-utils)\n\u003c/details\u003e\n\n## License\n[MIT](./LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantongolub%2Flockfile","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantongolub%2Flockfile","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantongolub%2Flockfile/lists"}