{"id":16400648,"url":"https://github.com/maraisr/piecemeal","last_synced_at":"2025-03-21T02:32:58.267Z","repository":{"id":57324017,"uuid":"421265643","full_name":"maraisr/piecemeal","owner":"maraisr","description":"🪀 Send your data piecemeal","archived":false,"fork":false,"pushed_at":"2024-01-19T02:08:10.000Z","size":116,"stargazers_count":24,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-12T05:28:20.756Z","etag":null,"topics":["cloudflare","incremental","incremental-delivery","multipart","serviceworker","workers"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maraisr.png","metadata":{"funding":{"github":"maraisr"},"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-10-26T03:26:11.000Z","updated_at":"2023-03-11T12:21:45.000Z","dependencies_parsed_at":"2024-01-19T03:24:30.692Z","dependency_job_id":"ea95310c-b18d-4c63-ae48-7aea168e6d6b","html_url":"https://github.com/maraisr/piecemeal","commit_stats":{"total_commits":29,"total_committers":1,"mean_commits":29.0,"dds":0.0,"last_synced_commit":"eb1966740717b2a848fdecbea786f06517dfec60"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maraisr%2Fpiecemeal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maraisr%2Fpiecemeal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maraisr%2Fpiecemeal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maraisr%2Fpiecemeal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maraisr","download_url":"https://codeload.github.com/maraisr/piecemeal/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221811380,"owners_count":16884305,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloudflare","incremental","incremental-delivery","multipart","serviceworker","workers"],"created_at":"2024-10-11T05:28:16.382Z","updated_at":"2024-10-28T09:12:08.139Z","avatar_url":"https://github.com/maraisr.png","language":"TypeScript","funding_links":["https://github.com/sponsors/maraisr"],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\t\u003ch1\u003e\u003cimg src=\"./shots/logo.svg\" alt=\"piecemeal\"/\u003e\u003c/h1\u003e\n\t\u003cp\u003e\u003ccode\u003enpm add piecemeal\u003c/code\u003e makes incremental delivery possible\u003c/p\u003e\n\t\u003chr /\u003e\n\t\u003cdiv\u003e\n\t\t\u003ca href=\"https://github.com/maraisr/piecemeal/actions/workflows/ci.yml\"\u003e\n\t\t\t\u003cimg src=\"https://github.com/maraisr/piecemeal/actions/workflows/ci.yml/badge.svg\"/\u003e\n\t\t\u003c/a\u003e\n\t\t\u003ca href=\"https://licenses.dev/npm/piecemeal\"\u003e\n\t\t  \u003cimg src=\"https://licenses.dev/b/npm/piecemeal?style=dark\" alt=\"licenses\" /\u003e\n\t\t\u003c/a\u003e\n\t\t\u003ca href=\"https://npm-stat.com/charts.html?package=piecemeal\"\u003e\n\t\t\t\u003cimg src=\"https://badgen.net/npm/dm/piecemeal?labelColor=black\u0026color=black\" alt=\"downloads\"/\u003e\n\t\t\u003c/a\u003e\n\t\t\u003ca href=\"https://packagephobia.com/result?p=piecemeal\"\u003e\n\t\t\t\u003cimg src=\"https://badgen.net/packagephobia/install/piecemeal?labelColor=black\u0026color=black\" alt=\"size\"/\u003e\n\t\t\u003c/a\u003e\n\t\t\u003ca href=\"https://bundlephobia.com/result?p=piecemeal\"\u003e\n\t\t\t\u003cimg src=\"https://badgen.net/bundlephobia/minzip/piecemeal?labelColor=black\u0026color=black\" alt=\"size\"/\u003e\n\t\t\u003c/a\u003e\n\t\u003c/div\u003e\n\u003c/div\u003e\n\n## ⚡ Features\n\n- **Lightweight** — _Does **not** include any dependencies [see](https://npm.anvaka.com/#/view/2d/piecemeal)_.\n\n- **Familiar** — _plugs into any `node:http` or `workers` based environment._\n\n- **Incredible DX** — _passing only an `AsyncIterable | Iterable`._\n\n## ⚙️ Install\n\n```sh\nnpm add piecemeal\n```\n\n## 🚀 Usage\n\n\u003e Visit [/examples](/examples) for more info!\n\n#### _Workers_\n\n```ts\nimport * as Piecemeal from 'piecemeal/worker';\n\n// Some sort of data access\n// ~\u003e here we read from KV, but can be anything\nasync function* get_data(binding: KV.Namespace) {\n  const list = await DATA.list();\n\n  for await (let item of list.keys) {\n    yield await DATA.get(item);\n  }\n}\n\n// A handler you'd typically give a Module or Service Worker\nconst handler = (request, env, context) =\u003e {\n  // Notice we're not awaiting or spreading this iterable\n  const data = get_data(context.bindings.DATA);\n\n  // Sets up our stream and constructs the Response\n  const { pipe, response } = Piecemeal.stream(data);\n\n  // Defers the execution of the iterable, so we respond super quick\n  context.waitUntil(pipe());\n\n  return response;\n};\n```\n\n#### _Node_\n\n```ts\nimport { createServer } from 'node:http';\n\nimport * as Piecemeal from 'piecemeal/node';\n\n// An example of some method to retreive some database data\nasync function* get_data() {\n  const keys = await db.fetchAllKeys();\n\n  for await (let key of keys) {\n    yield await db.read(key);\n  }\n}\n\ncreateServer((req, res) =\u003e {\n  // Notice we're not awaiting or spreading this iterable\n  const data = get_data();\n\n  // Creates a streams — and kicks off the iterable.\n  // assumes JSON (can override)\n  const stream = Piecemeal.stream(data);\n\n  // Pipes the stream directly to a ServerResponse\n  stream.pipe(res);\n}).listen(8080);\n```\n\n## 🔎 API\n\n#### Module: [`piecemeal/worker`](./src/worker.ts)\n\nThe main module used by [Cloudflare Workers](https://workers.cloudflare.com/) — or any\n[Service Worker API](https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API).\n\n\u003e Example over at [/examples/workers](/examples/workers)\n\n#### Module: [`piecemeal/node`](./src/node.ts)\n\nThe main module used for a `node` runtime and plugs directly into `node:http` modules.\n\n\u003e Example over at [/examples/polka](/examples/polka)\n\n#### Module: [`piecemeal/message`](./src/message.ts)\n\nA module used to construct messages. Messages are the partial _bits-of-data_ flushed in increments.\n\n#### Module: [`piecemeal`](./src/index.ts)\n\nA main module one can use to build out custom runtimes — exposes all the building blocks to `generate` a stream\nsupplying the Iterable and a write method.\n\n## 🙊 Caveats\n\n- Workers doesn't abort any iterables if connection is dropped. 😔\n\n## Related\n\n- [meros](https://github.com/maraisr/meros) — makes reading multipart responses simple\n\n## License\n\nMIT © [Marais Rossouw](https://marais.io)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaraisr%2Fpiecemeal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaraisr%2Fpiecemeal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaraisr%2Fpiecemeal/lists"}