{"id":16009092,"url":"https://github.com/xyb/chunksum","last_synced_at":"2025-03-25T14:46:24.628Z","repository":{"id":62743097,"uuid":"562144024","full_name":"xyb/chunksum","owner":"xyb","description":"Print FastCDC rolling hash chunks and checksums.","archived":false,"fork":false,"pushed_at":"2022-11-27T12:49:05.000Z","size":52,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-30T13:41:39.602Z","etag":null,"topics":["checksum","chunking","content-defined-chunking","dedup","deduplication","fastcdc","rolling-hash"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xyb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-11-05T12:54:23.000Z","updated_at":"2022-11-05T14:36:26.000Z","dependencies_parsed_at":"2023-01-22T13:16:27.351Z","dependency_job_id":null,"html_url":"https://github.com/xyb/chunksum","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyb%2Fchunksum","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyb%2Fchunksum/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyb%2Fchunksum/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyb%2Fchunksum/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xyb","download_url":"https://codeload.github.com/xyb/chunksum/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245486261,"owners_count":20623239,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["checksum","chunking","content-defined-chunking","dedup","deduplication","fastcdc","rolling-hash"],"created_at":"2024-10-08T13:01:10.630Z","updated_at":"2025-03-25T14:46:24.538Z","avatar_url":"https://github.com/xyb.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# chunksum\n\nPrint FastCDC rolling hash chunks and checksums.\n\n[![test](https://github.com/xyb/chunksum/actions/workflows/test.yml/badge.svg)](https://github.com/xyb/chunksum/actions/workflows/test.yml)\n[![codecov](https://codecov.io/gh/xyb/chunksum/branch/main/graph/badge.svg?token=LR3ET0TBK2)](https://codecov.io/gh/xyb/chunksum)\n[![Maintainability](https://api.codeclimate.com/v1/badges/9bd0a3b4fcefb196b2f8/maintainability)](https://codeclimate.com/github/xyb/chunksum/maintainability)\n[![Latest version](https://img.shields.io/pypi/v/chunksum.svg)](https://pypi.org/project/chunksum/)\n[![Support python versions](https://img.shields.io/pypi/pyversions/chunksum)](https://pypi.org/project/chunksum/)\n\n```\nusage: chunksum [-h] [-n ALG_NAME] [-f CHUNKSUMS_FILE] [-i INCR_FILE] [-m]\n                [path ...]\n\nPrint FastCDC rolling hash chunks and checksums.\n\npositional arguments:\n  path                  path to compute chunksums\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -n ALG_NAME, --alg-name ALG_NAME\n                        chunksum algorithm name.\n  -f CHUNKSUMS_FILE, --chunksums-file CHUNKSUMS_FILE\n                        chunksum file path, `-' for standard output.\n  -i INCR_FILE, --incr-file INCR_FILE\n                        incremental updates file path\n  -m, --multi-process   same number of multi-processes as cpu\n\nalg-name:\n  Format \"fc[k|m|g][0-9][sha2|blake2b|blake2s][32]\".\n\n  For example, \"fck4sha2\", means using FastCDC(\"fc\") with an\n  average chunk size of 2**8=256KB(\"k8\") and using sha256(\"sha2\")\n  to calculate the checksum.\n\n  \"fcm4blake2b32\" means using FastCDC with an average chunk size\n  of 2**4=16MB(\"m4\") and using \"blake2b\" to calculate and output\n  a checksum of length \"32\" bytes(save storage).\n\n  For large files, you may using large chunk size, such as \"m4\",\n  to reduce the number of chunks.\n\n  (default: fck4sha2)\n\nchunksums-file and incr-file:\n  You can specify the previous chunksums file if you want to\n  resume a previous check, or if you want to find the incremental\n  updates (new files) of the directory.\n\nExamples:\n\n  $ chunksum /etc \u003e ~/etc.chunksums\n\n  $ chunksum -n fcm4blake2b32 -m ~/Videos\n\n  $ chunksum -n fcm4blake2b32 -f ~/Videos/chunksums ~/Videos\n\n  $ chunksum -n fcm4blake2b32 -f ~/chunksums -i ~/chunksums.incr ~/Videos\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxyb%2Fchunksum","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxyb%2Fchunksum","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxyb%2Fchunksum/lists"}