{"id":40754942,"url":"https://github.com/oll3/bita","last_synced_at":"2026-01-21T16:09:01.181Z","repository":{"id":34053782,"uuid":"156584055","full_name":"oll3/bita","owner":"oll3","description":"Differential file synchronization over http","archived":false,"fork":false,"pushed_at":"2025-05-06T21:20:37.000Z","size":3074,"stargazers_count":285,"open_issues_count":6,"forks_count":12,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-10-26T08:11:27.165Z","etag":null,"topics":["differential-updates","download","file-sync","ota-update","rust","software-update","synchronization"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/bita","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oll3.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-11-07T17:30:56.000Z","updated_at":"2025-10-16T21:32:35.000Z","dependencies_parsed_at":"2024-05-22T20:48:29.190Z","dependency_job_id":"f6ba877e-40f8-476c-a14b-45874d2be1da","html_url":"https://github.com/oll3/bita","commit_stats":{"total_commits":383,"total_committers":9,"mean_commits":42.55555555555556,"dds":"0.044386422976501305","last_synced_commit":"48010433ae5e16d86fefbf05cd896fd1d2a6626b"},"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"purl":"pkg:github/oll3/bita","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oll3%2Fbita","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oll3%2Fbita/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oll3%2Fbita/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oll3%2Fbita/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oll3","download_url":"https://codeload.github.com/oll3/bita/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oll3%2Fbita/sbom","scorecard":{"id":705482,"data":{"date":"2025-08-11","repo":{"name":"github.com/oll3/bita","commit":"c26be0b2328d292ef341555d1a8a1f089bb33b4b"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.2,"checks":[{"name":"Code-Review","score":1,"reason":"Found 2/20 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 1 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/bita-ci.yml:1","Warn: no topLevel permission defined: .github/workflows/bitar-ci.yml:1","Warn: no topLevel permission defined: .github/workflows/cd.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/bita-ci.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bita-ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/bita-ci.yml:23: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bita-ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/bita-ci.yml:51: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bita-ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/bita-ci.yml:52: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bita-ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/bita-ci.yml:65: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bita-ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/bita-ci.yml:68: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bita-ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/bita-ci.yml:82: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bita-ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/bita-ci.yml:85: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bita-ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/bitar-ci.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bitar-ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/bitar-ci.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bitar-ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/bitar-ci.yml:43: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bitar-ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/bitar-ci.yml:46: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bitar-ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/bitar-ci.yml:60: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bitar-ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/bitar-ci.yml:63: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/bitar-ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/cd.yml:27: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/cd.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/cd.yml:28: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/cd.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/cd.yml:40: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/cd.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/cd.yml:66: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/cd.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/cd.yml:72: update your workflow using https://app.stepsecurity.io/secureworkflow/oll3/bita/cd.yml/main?enable=pin","Info:   0 out of   9 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of  10 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact v0.13.0 not signed: https://api.github.com/repos/oll3/bita/releases/193305431","Warn: release artifact v0.12.0 not signed: https://api.github.com/repos/oll3/bita/releases/156980919","Warn: release artifact v0.11.0 not signed: https://api.github.com/repos/oll3/bita/releases/94548595","Warn: release artifact v0.13.0 does not have provenance: https://api.github.com/repos/oll3/bita/releases/193305431","Warn: release artifact v0.12.0 does not have provenance: https://api.github.com/repos/oll3/bita/releases/156980919","Warn: release artifact v0.11.0 does not have provenance: https://api.github.com/repos/oll3/bita/releases/94548595"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 17 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-22T06:26:10.042Z","repository_id":34053782,"created_at":"2025-08-22T06:26:10.042Z","updated_at":"2025-08-22T06:26:10.042Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28635927,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T15:01:31.228Z","status":"ssl_error","status_checked_at":"2026-01-21T14:42:58.942Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["differential-updates","download","file-sync","ota-update","rust","software-update","synchronization"],"created_at":"2026-01-21T16:09:00.696Z","updated_at":"2026-01-21T16:09:01.173Z","avatar_url":"https://github.com/oll3.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![CI](https://github.com/oll3/bita/workflows/CI/badge.svg)](https://github.com/oll3/bita/actions?query=workflow%3ACI)\n[![](https://img.shields.io/crates/v/bita.svg)](https://crates.io/crates/bita)\n[![MIT licensed](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)\n\n## bita\n\n_bita_ is an HTTP-based file synchronization tool optimized to minimize bandwidth by reusing existing data without requiring pre-built patches or specialized servers.\n\n🔗 Host archives on any regular HTTP/HTTPS server \\\n♻️ Reuse existing data from any local file or block device \\\n💾 Write clone to a file or block device \\\n💫 Integrate into your own Rust project using the [bitar](bitar/README.md) library\n\n## Software updates\n\n_bita_ is a generic file synchronization tool but has been developed with software update of embedded/IoT systems in mind.\n\nSoftware update is a typical case where _bita_ may provide significant bandwidth reductions, where one can expect that a new software image will contain a lot of data already present on the system being updated. _bita_ can identify the parts (_chunks_) already present on the system and fetch the missing ones from remote, still outputing an exact clone of the archived source file.\n\nNo need to pre-build patch files for going to/from different release versions. No need to run any special file server.\nJust `bita compress` the release image, upload the archive to any HTTP file hosting site. And `bita clone` the archive using whatever local data is available on the system.\n\n![concept](images/concept.png?raw=true)\n\n### Compressing\n\nOn compression the input file is scanned for chunk boundaries using a rolling hash. With the default setting a suitable boundary should be found every ~64 KiB. A chunk is defined as the data contained between two boundaries. For each chunk a strong hash is generated (using blake2).\nThe chunk location (offset and size) in the input file and the strong hash is then stored in the dictionary. If chunk's strong hash has not been seen before the chunk data is also compressed (using brotli) and inserted into the output archive.\n\nThe final archive will contain a dictionary describing the order of chunks in the input file and the compressed chunks necessary to rebuild the input file. The archive will also contain the configuration used when scanning input for chunks.\n\n### Cloning\n\nOn clone the dictionary and chunker configuration is first fetched from the remote archive. Then the given seed file(s) are scanned for chunks present in the dictionary. Scanning is done using the same configuration as when building the archive.\nAny chunk found in a seed file will be copied into the output file at the location(s) specified by the dictionary.\nWhen all seeds has been consumed the chunks still missing, if any, is fetched from the remote archive, decompressed and inserted into the output file.\n\nTo keep the HTTP overhead low while cloning all adjacent chunks are fetched with a single request. And if possible the same connection is used for the whole clone operation.\n\n_bita_ can also use the output file as seed and reorganize chunks in place. Except using this for the obvious reason of saving bandwidth this will also let _bita_ avoid writing chunks that are already in place in the output file. This may be useful if writing to storage is either slow or we want to avoid tearing on the storage device.\n\nEach chunk, both fetched from seed and from archive, is verified by its strong hash before written to the output. _bita_ avoids using any extra storage space while cloning, the only file written to is the given output file.\n\n### Scanning for chunks\n\nThe process of splitting a file into chunks is heavily inspired by the one used in rsync. Where a window of bytes (default 64 for RollSum and 20 for BuzHash) is sliding through the input data, one byte at a time.\n\nFor every position of the window a short checksum is generated. If we're assuming that the checksum has an even distribution we can say that with some probability this checksum will be within a range of values at every `n` interval of bytes, where `n` represents the average target chunk size.\n\nWhen the checksum is within this range a chunk boundary has been found. A strong hash (blake2) is then generated for the data between the last boundary and this one. The strong hash is the one used to identify this chunk while the weaker rolling hash is never stored but only used for finding chunk boundaries.\n\nThe average target chunk size and the upper/lower limit of a chunk's size is runtime configurable.\n\n## Server requirements\n\nThe server serving _bita_ archives can be any HTTP/HTTPS server supporting [range requests](https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests), which should be most.\n\n## Install from crates.io\n\nInstall _bita_ using cargo:\n\n```console\nolle@home:~$ cargo install bita\n```\n\n## Build from source\n\n```console\nolle@home:~$ cargo build\n```\n\nBuild in release mode with rustls TLS backend:\n\n```console\nolle@home:~$ cargo build --release --no-default-features --features rustls-tls\n```\n\n## Example usage\n\nCreate a compressed archive `release_v1.1.ext4.cba` from file `release_v1.1.ext4`:\n\n```console\nolle@home:~$ bita compress -i release_v1.1.ext4 release_v1.1.ext4.cba\n```\n\nClone using block device `/dev/mmcblk0p1` as seed and `/dev/mmcblk0p2` as target:\n\n```console\nupgrader@device:~$ bita clone --seed /dev/mmcblk0p1 https://host/release_v1.1.ext4.cba /dev/mmcblk0p2\n```\n\nClone and use output (`/dev/mmcblk0p1`) as seed while cloning:\n\n```console\nupgrader@device:~$ bita clone --seed-output https://host/release_v1.1.ext4.cba /dev/mmcblk0p1\n```\n\nLocal archives can also be cloned:\n\n```console\nupgrader@device:~$ bita clone --seed-output local.cba local_output.file\n```\n\nClone file at `https://host/new.tar.cba` using stdin (-) and block device `/dev/sda1` as seed:\n\n```console\nolle@home:~$ gunzip -c old.tar.gz | bita clone --seed /dev/sda1 --seed - https://host/new.tar.cba new.tar\n```\n\nCompare two filesystem images to see how much content they share with different chunking parameters:\n\n```console\nolle@home:~$ bita diff release_v1.0.ext4 release_v1.1.ext4\nolle@home:~$ bita diff --hash-chunking BuzHash --avg-chunk-size 8KiB release_v1.0.ext4 release_v1.1.ext4\n```\n\n## Similar tools and inspiration\n\n- [casync](https://github.com/systemd/casync)\n- [zchunk](https://github.com/zchunk/zchunk)\n- [zsync](http://zsync.moria.org.uk)\n- [rsync](https://rsync.samba.org/)\n- [bup](https://github.com/bup/bup)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foll3%2Fbita","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foll3%2Fbita","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foll3%2Fbita/lists"}