{"id":16009182,"url":"https://github.com/xyb/chunkdup","last_synced_at":"2025-07-28T12:31:14.683Z","repository":{"id":63222902,"uuid":"562409351","full_name":"xyb/chunkdup","owner":"xyb","description":"Find (partial content) duplicate files.","archived":false,"fork":false,"pushed_at":"2022-12-10T04:33:31.000Z","size":92,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-09T04:39:59.460Z","etag":null,"topics":["content-defined-chunking","dedup","deduplication","fastcdc","rolling-hash"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xyb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-11-06T09:09:33.000Z","updated_at":"2024-07-25T07:26:16.000Z","dependencies_parsed_at":"2023-01-25T20:20:36.576Z","dependency_job_id":null,"html_url":"https://github.com/xyb/chunkdup","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyb%2Fchunkdup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyb%2Fchunkdup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyb%2Fchunkdup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyb%2Fchunkdup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xyb","download_url":"https://codeload.github.com/xyb/chunkdup/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227905528,"owners_count":17837906,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["content-defined-chunking","dedup","deduplication","fastcdc","rolling-hash"],"created_at":"2024-10-08T13:01:25.903Z","updated_at":"2024-12-03T11:12:41.303Z","avatar_url":"https://github.com/xyb.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# chunkdup\n\nFind (partial content) duplicate files using [chunksum](https://github.com/xyb/chunksum) outputs.\n\n[![test](https://github.com/xyb/chunkdup/actions/workflows/test.yml/badge.svg)](https://github.com/xyb/chunkdup/actions/workflows/test.yml)\n[![codecov](https://codecov.io/gh/xyb/chunkdup/branch/main/graph/badge.svg?token=TVFUKMLFMX)](https://codecov.io/gh/xyb/chunkdup)\n[![Maintainability](https://api.codeclimate.com/v1/badges/0935f557916da1fdcddb/maintainability)](https://codeclimate.com/github/xyb/chunkdup/maintainability)\n[![Latest version](https://img.shields.io/pypi/v/chunkdup.svg)](https://pypi.org/project/chunkdup/)\n[![Support python versions](https://img.shields.io/pypi/pyversions/chunkdup)](https://pypi.org/project/chunkdup/)\n\n```\nusage: chunkdup [-h] [chunksums1] [chunksums2]\n\nFind (partial content) duplicate files.\n\npositional arguments:\n  chunksums1  path to chunksums\n  chunksums2  path to chunksums\n\noptional arguments:\n  -h, --help  show this help message and exit\n\nExamples:\n\n  $ chunksum dir1/ -f chunksums.dir1\n  $ chunksum dir2/ -f chunksums.dir2\n  $ chunkdup chunksums.dir1 chunksums.dir2\n```\n\n```\nusage: chunkdiff [-h] [-b BAR] [-w BARWIDTH] [-n] [-s CHUNKSUMS]\n                 [file1] [file2]\n\nShow the difference of two files.\n\npositional arguments:\n  file1                 path to file\n  file2                 path to file\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -b BAR, --bar BAR     the style of bar. default: oneline\n  -w BARWIDTH, --barwidth BARWIDTH\n                        the width of bar. default: 40\n  -n, --nocolor         do not colorize output. default: False\n  -s CHUNKSUMS, --chunksums CHUNKSUMS\n                        path to chunksums file\n\nExamples:\n\n  $ chunksum dir1/ -f chunksums.dir1\n  $ chunksum dir2/ -f chunksums.dir2\n\n  $ chunkdiff chunksums.dir1 chunksums.dir2 dir1/file1 dir2/file2\n\n  $ chunkdiff chunksums chunksums ./file1 ./file2\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxyb%2Fchunkdup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxyb%2Fchunkdup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxyb%2Fchunkdup/lists"}