{"id":23315456,"url":"https://github.com/ihoro/tar-stream-chunker","last_synced_at":"2026-06-30T09:31:23.626Z","repository":{"id":86874418,"uuid":"233269726","full_name":"ihoro/tar-stream-chunker","owner":"ihoro","description":"Splits stdin onto chunk files and writes resulting TAR archive to stdout ","archived":false,"fork":false,"pushed_at":"2020-09-03T08:12:55.000Z","size":80,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-09-11T17:57:52.785Z","etag":null,"topics":["backup","shell","tar","tarsnap","unix-way","utility"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ihoro.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-11T17:27:39.000Z","updated_at":"2022-06-05T10:32:52.000Z","dependencies_parsed_at":"2023-05-29T16:00:13.404Z","dependency_job_id":null,"html_url":"https://github.com/ihoro/tar-stream-chunker","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/ihoro/tar-stream-chunker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ihoro%2Ftar-stream-chunker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ihoro%2Ftar-stream-chunker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ihoro%2Ftar-stream-chunker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ihoro%2Ftar-stream-chunker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ihoro","download_url":"https://codeload.github.com/ihoro/tar-stream-chunker/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ihoro%2Ftar-stream-chunker/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34961543,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-30T02:00:05.919Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","shell","tar","tarsnap","unix-way","utility"],"created_at":"2024-12-20T15:37:50.628Z","updated_at":"2026-06-30T09:31:23.620Z","avatar_url":"https://github.com/ihoro.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"= tar-stream-chunker\n\nimage:https://github.com/ihoro/tar-stream-chunker.c/workflows/linux/badge.svg[link=\"https://github.com/ihoro/tar-stream-chunker.c/actions?query=workflow%3Alinux\"]\nimage:https://github.com/ihoro/tar-stream-chunker.c/workflows/macos/badge.svg[link=\"https://github.com/ihoro/tar-stream-chunker.c/actions?query=workflow%3Amacos\"]\nimage:https://img.shields.io/lgtm/alerts/g/ihoro/tar-stream-chunker.c.svg?logo=lgtm\u0026logoWidth=18[link=\"https://lgtm.com/projects/g/ihoro/tar-stream-chunker.c/alerts/\"]\nimage:https://img.shields.io/lgtm/grade/cpp/g/ihoro/tar-stream-chunker.c.svg?logo=lgtm\u0026logoWidth=18[link=\"https://lgtm.com/projects/g/ihoro/tar-stream-chunker.c/context:cpp\"]\nimage:https://img.shields.io/lgtm/grade/javascript/g/ihoro/tar-stream-chunker.c.svg?logo=lgtm\u0026logoWidth=18[link=\"https://lgtm.com/projects/g/ihoro/tar-stream-chunker.c/context:javascript\"]\nimage:https://img.shields.io/badge/FreeBSD-port-informational?logo=freebsd[link=\"https://www.freshports.org/archivers/tar-stream-chunker\"]\n\nSplits stdin onto chunk files and writes resulting TAR archive to stdout.\n\nIt can help to handle data stream of unknown size without hitting a disk. As long as creating TAR archive as a stream needs to know file size in advance, that is why the idea of the chunker is to split input stream onto chunks with known size.\n\nInitial motivation was to use it for backup streaming to tarsnap.com.\n\n== Build from sources\n\nExpecting you have a POSIX compliant system:\n\n`$ make`\n\n== Test suite\n\nEnd-to-end test suite requires node.js:\n\n`$ make e2e`\n\n== FreeBSD port/package\n\n- `pkg install tar-stream-chunker`\n\n- or `cd /usr/ports/archivers/tar-stream-chunker \u0026\u0026 make install clean`\n\n== Usage\n\n`$ tar_stream_chunker { --file-name | -f } \u003cfile-name\u003e { --chunk-size | -s } \u003cbytes\u003e`\n\nAnd redirect its stdout to your target and that's it.\n\n== Usage examples\n\nPostgreSQL dump what is streamed to tarsnap:\n\n`$ pg_dump | tar_stream_chunker -f dump.sql -s 500000000 | tarsnap -cvf dump.daily.$(date +%Y%m%d.%H%M%S) @-`\n\nLet's see what's inside a resulting TAR:\n```\n$ cat 3MB-file | tar_stream_chunker --file-name file --chunk-size 1000000 \u003e result.tar\n$ tar -tf result.tar\nfile.00001\nfile.00002\nfile.00003\n```\n\nAnd original input can be re-assembled as follows:\n```\n$ tar -xf result.tar; cat file.* \u003e original-3MB-file\n$ tar -xOf result.tar \u003e original-3MB-file\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fihoro%2Ftar-stream-chunker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fihoro%2Ftar-stream-chunker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fihoro%2Ftar-stream-chunker/lists"}