{"id":19302488,"url":"https://github.com/bitlytwiser/tinychunk","last_synced_at":"2026-06-18T11:32:03.993Z","repository":{"id":61624437,"uuid":"538559275","full_name":"BitlyTwiser/tinychunk","owner":"BitlyTwiser","description":"Tiny library for splicing byte streams into chunks.","archived":false,"fork":false,"pushed_at":"2022-12-25T17:33:49.000Z","size":9783,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-05T23:25:45.849Z","etag":null,"topics":["chunking-files","files","golang"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BitlyTwiser.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-09-19T15:05:11.000Z","updated_at":"2022-09-28T00:05:59.000Z","dependencies_parsed_at":"2023-01-30T22:30:48.028Z","dependency_job_id":null,"html_url":"https://github.com/BitlyTwiser/tinychunk","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BitlyTwiser%2Ftinychunk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BitlyTwiser%2Ftinychunk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BitlyTwiser%2Ftinychunk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BitlyTwiser%2Ftinychunk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BitlyTwiser","download_url":"https://codeload.github.com/BitlyTwiser/tinychunk/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240404680,"owners_count":19796062,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chunking-files","files","golang"],"created_at":"2024-11-09T23:22:23.733Z","updated_at":"2026-06-18T11:32:03.956Z","avatar_url":"https://github.com/BitlyTwiser.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tinyChunk\nTiny library for splicing byte streams into chunks.\n[![Go Report Card](https://goreportcard.com/badge/github.com/BitlyTwiser/tinychunk)](https://goreportcard.com/report/github.com/BitlyTwiser/tinychunk)\n\n## Installation:\n```go get github.com/BitlyTwiser/tinychunk```\n\n## Example usage and Testing:\n- For simple usage, one can look to the testing within the project to gleen how the chunker can operate:\n- Simply craft a function and pass said function as an argument itno the chunker. This, along with the data and the size of the chunks (in MB) will chunk your data and perform the given operations upon the chunked byte segments.\n\n```\npackage tinychunk_test\n\nimport (\n\t\"fmt\"\n\t. \"github.com/BitlyTwiser/tinychunk\"\n\t\"github.com/stretchr/testify/assert\"\n\t\"log\"\n\t\"os\"\n\t\"testing\"\n  \"github.com/google/uuid\"\n)\n\n// Simply write files, then delete them.\nfunc fileFunc(fileData []byte) error {\n  err := os.WriteFile(fmt.Sprintf(\"./assets/%v.txt\", uuid.NewString()), fileData, 0700)\n\n  if err != nil {\n    return err\n  }\n\n\treturn nil\n}\n\nfunc openFile(fileName string) []byte {\n\tfile, err := os.ReadFile(fmt.Sprintf(\"./assets/%v\", fileName))\n\n\tif err != nil {\n\t\tlog.Printf(\"Error opening file for test: %v\", err.Error())\n\t\tos.Exit(1)\n\t}\n\n\treturn file\n}\n\nfunc TestOneMegabyteChunking(t *testing.T) {\n\tt.Log(\"chunking 1MB file\")\n\tfile := openFile(\"testFile.txt\")\n\n\terr := Chunk(file, 1, fileFunc)\n\n\tassert.Nil(t, err)\n}\n\nfunc TestTwoMegabyteChunking(t *testing.T) {\n\tt.Log(\"chunking 2MB file\")\n\tfile := openFile(\"testFile.txt\")\n\n\terr := Chunk(file, 2, fileFunc)\n\n\tassert.Nil(t, err)\n}\n\nfunc TestThreeMegabyteChunking(t *testing.T) {\n\tt.Log(\"chunking 3MB file\")\n\tfile := openFile(\"testFile.txt\")\n\n\terr := Chunk(file, 3, fileFunc)\n\n\tassert.Nil(t, err)\n}\n\nfunc TestFourMegabyteChunking(t *testing.T) {\n\tt.Log(\"chunking 4MB file\")\n\tfile := openFile(\"testFile.txt\")\n\n\terr := Chunk(file, 4, fileFunc)\n\n\tassert.Nil(t, err)\n}\n```\n\n- After running the test, the \"test\" folder will hold the files resulting from the testing operation (result of ```ls -la test | sort -h```):\n```\n-rwx------ 1 steve steve 1048576 Sep 19 18:17 048929cb-9bed-4344-a4aa-c527569e205d.txt                                                                                  \n-rwx------ 1 steve steve 1048576 Sep 19 18:17 3ea9eeb4-2cb5-4923-9914-225afd1ec57b.txt                                                                                  \n-rwx------ 1 steve steve 1048576 Sep 19 18:17 41109604-7aba-458f-a86f-0b742ef270b9.txt                                                                                  \n-rwx------ 1 steve steve 1048576 Sep 19 18:17 41e14135-3e45-4e8f-8a94-ad0c0a30e43a.txt                                                                                  \n-rwx------ 1 steve steve 1048576 Sep 19 18:17 48b16133-a072-4f2e-b6cd-a9871fe52970.txt                                                                                  \n-rwx------ 1 steve steve 1048576 Sep 19 18:17 7ebe9cd6-adb2-4b1a-b7ae-9ea51dca1c86.txt                                                                                  \n-rwx------ 1 steve steve 1048576 Sep 19 18:17 b24f2b92-8266-44fd-a39e-fce8f5393e47.txt                                                                                  \n-rwx------ 1 steve steve 1048576 Sep 19 18:17 b50d9972-66d3-45db-b822-912a8f1fdfd9.txt                                                                                  \n-rwx------ 1 steve steve 1048576 Sep 19 18:17 f32c8c47-c096-42eb-b9db-1a6ed51f7acd.txt                                                                                  \n-rwx------ 1 steve steve 2097152 Sep 19 18:17 0431062f-d5d6-49bd-a230-70b28fa29d86.txt                                                                                  \n-rwx------ 1 steve steve 2097152 Sep 19 18:17 23d9d7fd-5382-419f-b4ec-0a538684a4da.txt                                                                                  \n-rwx------ 1 steve steve 2097152 Sep 19 18:17 70e265df-e474-43bb-87ca-bed52c8ba075.txt                                                                                  \n-rwx------ 1 steve steve 2097152 Sep 19 18:17 b7a719dc-9291-446c-920b-303f760f0a23.txt                                                                                  \n-rwx------ 1 steve steve 3145728 Sep 19 18:17 65a72369-76e3-446e-b05c-80266170e495.txt                                                                                  \n-rwx------ 1 steve steve 3145728 Sep 19 18:17 d92a6e72-ced5-42c6-9d51-40c33f6dc2d7.txt                                                                                  \n-rwx------ 1 steve steve 3145728 Sep 19 18:17 f710795d-c0ee-4a89-8bb5-9d9316807e93.txt                                                                                  \n-rwx------ 1 steve steve 4194304 Sep 19 18:17 394fb35b-9c62-415e-9376-c664ce5364e7.txt                                                                                  \n-rwx------ 1 steve steve 4194304 Sep 19 18:17 831ccf16-db5e-4625-8053-be1c71ceefe6.txt    \n```\n\n## Seed Data for files:\n```\nhead -c 10MB \u003c/dev/urandom \u003etestFile.txt\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitlytwiser%2Ftinychunk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbitlytwiser%2Ftinychunk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbitlytwiser%2Ftinychunk/lists"}