{"id":20642404,"url":"https://github.com/ipld/js-unixfs","last_synced_at":"2025-04-16T01:41:06.747Z","repository":{"id":42472339,"uuid":"454586355","full_name":"ipld/js-unixfs","owner":"ipld","description":"UnixFS Directed Acyclic Graph for IPLD","archived":false,"fork":false,"pushed_at":"2024-01-31T16:02:39.000Z","size":11502,"stargazers_count":12,"open_issues_count":12,"forks_count":7,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-04-06T07:52:21.783Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ipld.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-02-01T23:34:41.000Z","updated_at":"2025-03-13T21:31:50.000Z","dependencies_parsed_at":"2024-01-31T17:43:34.067Z","dependency_job_id":null,"html_url":"https://github.com/ipld/js-unixfs","commit_stats":{"total_commits":59,"total_committers":4,"mean_commits":14.75,"dds":"0.23728813559322037","last_synced_commit":"7208f7ad0795b263c898696d891589133b1c6fde"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipld%2Fjs-unixfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipld%2Fjs-unixfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipld%2Fjs-unixfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipld%2Fjs-unixfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ipld","download_url":"https://codeload.github.com/ipld/js-unixfs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249182797,"owners_count":21226123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T16:08:59.254Z","updated_at":"2025-04-16T01:41:06.718Z","avatar_url":"https://github.com/ipld.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# @ipld/unixfs\n\nAn implementation of the [UnixFS spec][] in JavaScript designed for use with\n[multiformats][].\n\n[unixfs spec]: https://github.com/ipfs/specs/blob/master/UNIXFS.md\n[multiformats]: https://github.com/multiformats/js-multiformats\n\n## Overview\n\nThis library provides functionality similar to [ipfs-unixfs-importer][], but it had been designed around different set of use cases:\n\n1. Writing into Content Addressable Archives ([CAR][]).\n\n   In order to allow encoding file(s) into arbitrary number of CARs, library makes no assumbtions about how blocks will be consumed by returning [ReadableStream][] of blocks and leaving it up to caller to handle the rest.\n\n1. Incremental and resumable writes\n\n   Instead of passing a stream of files, user creates files, writes into them and when finished gets `Promise\u003cCID\u003e` for it. This removes need for mapping files back to their CIDs streamed on the other end.\n\n1. Complete control of memory and concurrency\n\n   By using writer style API users can choose how many files to write concurrently and change that decision based on other tasks application performs. User can also specify buffer size to be able to tweak read/write coordination.\n\n1. No indirect configuration\n\n   Library removes indirection by taking approach similar to [multiformats][] library. Instead of passing chunker and layout config options, you pass chunker / layout / encoder interface implementations.\n\n### Usage\n\nYou can encode a file as follows\n\n```js\nimport * as UnixFS from \"@ipld/unixfs\"\n\n// Create a redable \u0026 writable streams with internal queue that can\n// hold around 32 blocks\nconst { readable, writable } = new TransformStream(\n  {},\n  UnixFS.withCapacity(1048576 * 32)\n)\n// Next we create a writer with filesystem like API for encoding files and\n// directories into IPLD blocks that will come out on `readable` end.\nconst writer = UnixFS.createWriter({ writable })\n\n// Create file writer that can be used to encode UnixFS file.\nconst file = UnixFS.createFileWriter(writer)\n// write some content\nfile.write(new TextEncoder().encode(\"hello world\"))\n// Finalize file by closing it.\nconst { cid } = await file.close()\n\n// close the writer to close underlying block stream.\nwriter.close()\n\n// We could encode all this as car file\nencodeCAR({ roots: [cid], blocks: readable })\n```\n\nYou can encode (non sharded) directories with provided API as well\n\n```ts\nimport * as UnixFS from \"@ipld/unixfs\"\n\nexport const demo = async () =\u003e {\n  const { readable, writable } = new TransformStream()\n  const writer = UnixFS.createWriter({ writable })\n\n  // write a file\n  const file = UnixFS.createFileWriter(writer)\n  file.write(new TextEncoder().encode(\"hello world\"))\n  const fileLink = await file.close()\n\n  // create directory and add a file we encoded above\n  const dir = UnixFS.createDirectoryWriter(writer)\n  dir.set(\"intro.md\", fileLink)\n  const dirLink = await dir.close()\n\n  // now wrap above directory with another and also add the same file\n  // there\n  const root = UnixFS.createDirectoryWriter(fs)\n  root.set(\"user\", dirLink)\n  root.set(\"hello.md\", fileLink)\n\n  // Creates following UnixFS structure where intro.md and hello.md link to same\n  // IPFS file.\n  // ./\n  // ./user/intro.md\n  // ./hello.md\n  const rootLink = await root.close()\n  // ...\n  writer.close()\n}\n```\n\n### Configuration\n\nYou can configure DAG layout, chunking and bunch of other things by providing API compatible components. Library provides bunch of them but you can also bring your own.\n\n```js\nimport * as UnixFS from \"@ipld/unixfs\"\nimport * as Rabin from \"@ipld/unixfs/file/chunker/rabin\"\nimport * as Trickle from \"@ipld/unixfs/file/layout/trickle\"\nimport * as RawLeaf from \"multiformats/codecs/raw\"\nimport { sha256 } from \"multiformats/hashes/sha2\"\n\nconst demo = async blob =\u003e {\n  const { readable, writable } = new TransformStream()\n  const writer = UnixFS.createWriter({\n    writable,\n    // you can pass only things you want to override\n    settings: {\n      fileChunker: await Rabin.create({\n        avg: 60000,\n        min: 100,\n        max: 662144,\n      }),\n      fileLayout: Trickle.configure({ maxDirectLeaves: 100 }),\n      // Encode leaf nodes as raw blocks\n      fileChunkEncoder: RawLeaf,\n      smallFileEncoder: RawLeaf,\n      fileEncoder: UnixFS,\n      hasher: sha256,\n    },\n  })\n\n  const file = UnixFS.createFileWriter(writer)\n  // ...\n}\n```\n\n## License\n\nLicensed under either of\n\n- Apache 2.0, ([LICENSE-APACHE](LICENSE-APACHE) / http://www.apache.org/licenses/LICENSE-2.0)\n- MIT ([LICENSE-MIT](LICENSE-MIT) / http://opensource.org/licenses/MIT)\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.\n\n[ipfs-unixfs-importer]: https://www.npmjs.com/package/ipfs-unixfs-importer\n[readablestream]: https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream\n[car]: https://ipld.io/specs/transport/car/carv1/\n[`transformstream`]: https://developer.mozilla.org/en-US/docs/Web/API/TransformStream\n[`writablestream`]: https://developer.mozilla.org/en-US/docs/Web/API/WritableStream\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fipld%2Fjs-unixfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fipld%2Fjs-unixfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fipld%2Fjs-unixfs/lists"}