{"id":15152857,"url":"https://github.com/ipfs/go-ipfs-chunker","last_synced_at":"2025-09-30T01:31:16.390Z","repository":{"id":32581845,"uuid":"120324407","full_name":"ipfs/go-ipfs-chunker","owner":"ipfs","description":"go-ipfs-chunkers provides Splitter implementations for data before being ingested to IPFS","archived":true,"fork":false,"pushed_at":"2023-06-14T11:16:28.000Z","size":120,"stargazers_count":31,"open_issues_count":7,"forks_count":37,"subscribers_count":24,"default_branch":"master","last_synced_at":"2024-09-20T19:48:53.591Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ipfs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-05T15:38:17.000Z","updated_at":"2024-04-01T09:19:44.000Z","dependencies_parsed_at":"2024-06-18T14:08:04.059Z","dependency_job_id":null,"html_url":"https://github.com/ipfs/go-ipfs-chunker","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipfs%2Fgo-ipfs-chunker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipfs%2Fgo-ipfs-chunker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipfs%2Fgo-ipfs-chunker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipfs%2Fgo-ipfs-chunker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ipfs","download_url":"https://codeload.github.com/ipfs/go-ipfs-chunker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219865579,"owners_count":16554163,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-26T16:40:42.628Z","updated_at":"2025-09-30T01:31:16.068Z","avatar_url":"https://github.com/ipfs.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# go-ipfs-chunker\n\n\u003e go-ipfs-chunker implements data Splitters for go-ipfs.\n\n[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io)\n[![](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io/)\n[![standard-readme compliant](https://img.shields.io/badge/standard--readme-OK-green.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme)\n[![GoDoc](https://godoc.org/github.com/ipfs/go-ipfs-chunker?status.svg)](https://godoc.org/github.com/ipfs/go-ipfs-chunker)\n[![Build Status](https://travis-ci.org/ipfs/go-ipfs-chunker.svg?branch=master)](https://travis-ci.org/ipfs/go-ipfs-chunker)\n\n## ❗ This repo is no longer maintained.\n👉 We highly recommend switching to the maintained version at https://github.com/ipfs/boxo/tree/main/chunker.\n🏎️ Good news!  There is [tooling and documentation](https://github.com/ipfs/boxo#migrating-to-boxo) to expedite a switch in your repo. \n\n⚠️ If you continue using this repo, please note that security fixes will not be provided (unless someone steps in to maintain it).\n\n📚 Learn more, including how to take the maintainership mantle or ask questions, [here](https://github.com/ipfs/boxo/wiki/Copied-or-Migrated-Repos-FAQ).\n\n\n## Summary\n\n`go-ipfs-chunker` provides the `Splitter` interface. IPFS splitters read data from a reader an create \"chunks\". These chunks are used to build the ipfs DAGs (Merkle Tree) and are the base unit to obtain the sums that ipfs uses to address content.\n\nThe package provides a `SizeSplitter` which creates chunks of equal size and it is used by default in most cases, and a `rabin` fingerprint chunker. This chunker will attempt to split data in a way that the resulting blocks are the same when the data has repetitive patterns, thus optimizing the resulting DAGs.\n\n## Table of Contents\n\n- [Install](#install)\n- [Usage](#usage)\n- [License](#license)\n\n## Install\n\n`go-ipfs-chunker` works like a regular Go module:\n\n```\n\u003e go get github.com/ipfs/go-ipfs-chunker\n```\n\n## Usage\n\n```\nimport \"github.com/ipfs/go-ipfs-chunker\"\n```\n\nCheck the [GoDoc documentation](https://godoc.org/github.com/ipfs/go-ipfs-chunker)\n\n## License\n\nMIT © Protocol Labs, Inc.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fipfs%2Fgo-ipfs-chunker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fipfs%2Fgo-ipfs-chunker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fipfs%2Fgo-ipfs-chunker/lists"}