{"id":19602346,"url":"https://github.com/scottlamb/multipart-stream-js","last_synced_at":"2025-04-27T17:32:16.434Z","repository":{"id":43410625,"uuid":"243445373","full_name":"scottlamb/multipart-stream-js","owner":"scottlamb","description":"Javascript library to parse an HTTP multipart stream","archived":true,"fork":false,"pushed_at":"2023-01-27T13:17:19.000Z","size":319,"stargazers_count":7,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-14T16:12:59.250Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scottlamb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-27T06:14:37.000Z","updated_at":"2024-11-07T06:30:05.000Z","dependencies_parsed_at":"2023-02-15T09:31:26.500Z","dependency_job_id":null,"html_url":"https://github.com/scottlamb/multipart-stream-js","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottlamb%2Fmultipart-stream-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottlamb%2Fmultipart-stream-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottlamb%2Fmultipart-stream-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottlamb%2Fmultipart-stream-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scottlamb","download_url":"https://codeload.github.com/scottlamb/multipart-stream-js/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251178017,"owners_count":21548147,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T09:23:37.289Z","updated_at":"2025-04-27T17:32:16.194Z","avatar_url":"https://github.com/scottlamb.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# multipart-stream-js\n\nJavascript library to parse an HTTP multipart stream.\n\n**Maintenance status:** not maintained. I've switched to WebSockets to avoid problems with browser connection limits.\n\n## What's a multipart stream for?\n\nA multipart stream is a sequence of parts in one HTTP response, each part\nhaving its own headers and body. A stream might last forever, serving parts\nthat didn't exist at the start of the request. This is a type of \"hanging GET\"\nor [Comet](https://en.wikipedia.org/wiki/Comet_(programming)) request.\n\nIt's a simple HTTP/1.1 way of accomplishing what otherwise might require\nfancier server- and client-side technologies, such as:\n\n   * [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API)\n   * [HTTP/2 Server Push](https://en.wikipedia.org/wiki/HTTP/2_Server_Push)\n\nNever-ending multipart streams seem popular in the IP camera space:\n\n   * Dahua IP cameras provide a `multipart/x-mixed-replace` stream of events\n     such as motion detection changes.\n     ([spec](http://www.telecamera.ru/bitrix/components/bitrix/forum.interface/show_file.php?fid=1022477\u0026action=download))\n   * Hikvision IP cameras provide a `multipart/mixed` stream of events,\n     as described\n     [here](https://github.com/scottlamb/moonfire-playground/blob/4e6a786286272ee36f449d761740191c6e6a54fc/camera-motion/src/hikvision.rs#L33).\n   * wikipedia [mentions](https://en.wikipedia.org/wiki/MIME#Mixed-Replace)\n     that IP cameras use this format for MJPEG streams.\n\nThere's a big limitation, however, which is that browsers have fairly low\nlimits on the number of concurrent connections. In Chrome's case, six per\nhost.\n\nI wrote this library as part of my own [Moonfire\nNVR](https://github.com/scottlamb/moonfire-nvr) to implement live streams (a\nmultipart stream of `.mp4` media segments) and event streams. Due to the\nlimitation above, I'm likely going to use WebSockets instead.\n\n## What is a multipart stream exactly?\n\nA multipart response might look like this:\n\n```\nContent-Type: multipart/x-mixed-replace: boundary=B\n\n--B\nContent-Type: text/plain\nContent-Length: 3\n\nfoo\n\n--B\nContent-Type: text/plain\nContent-Length: 3\n\nbar\n```\n\nand is typically paired with `Transfer-Encoding: chunked` or `Connection:\nclose` to allow sending a response whose size is infinite or not known until\nthe end.\n\nI can't find a good specification. [This WHATWG\ndocument](https://html.spec.whatwg.org/multipage/iana.html#multipart/x-mixed-replace)\ndescribes `multipart/x-mixed-replace` loosely. It refers to [RFC\n2046](https://tools.ietf.org/html/rfc2046) which defines multipart encodings\noriginally used for rich emails. I don't think these HTTP multipart streams\nquite follow that RFC. My library currently requires:\n\n   * Content type `multipart/...; boundary=...`. In MIME media type\n     terminology, the `type` is multipart; the `subtype` may be anything.\n     There should be exactly one parameter, `boundary`.\n   * No preamble. That is, no arbitrary bytes to discard before the first\n     part's boundary.\n   * Zero or more newlines (to be precise: `\\r\\n` sequences) between each part\n     and the next part's boundary.\n   * A `Content-Length` line for each part. This is a much cleaner approach\n     than producers attempting to choose a boundary that doesn't appear in any\n     part and consumers having to search through the part body.\n   * No extra `--` suffix on the final part's boundary. In practice, all the\n     streams I've seen only end due to error, so this point has never come up.\n\nPlease open a github issue if you encounter a multipart stream which doesn't\nmatch these requirements.\n\n## What does this library do?\n\nIt reads from from a ReadableStream as defined in the\nWHATWG Streams API ([spec](https://streams.spec.whatwg.org/),\n[MDN](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream)) and\nturns it into another ReadableStream of parts. Each part has a header and\nbody.\n\nIt works well with the WHATWG Fetch API\n([spec](https://fetch.spec.whatwg.org/),\n[MDN](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)).\n\nExample:\n\n```js\nimport multipartStream from `.../multipart-stream.js`;\n\nasync function processStream() {\n  const response = await fetch(`http://example.com/stream`);\n  const reader = multipartStream(\n      response.headers.get('Content-Type'),\n      response.body);\n  while (true) {\n    const {done, value} = reader.read();\n    if (done) {\n      break;\n    }\n    const {headers, body} = value;\n    ...\n  }\n}\n```\n\n## Where does it work?\n\nModern web browsers. It's tested on recent\n[Chrome](https://www.google.com/chrome/) and\n[Firefox](https://www.mozilla.org/en-US/firefox/) via\n[Karma](https://karma-runner.github.io/) and\n[Jasmine](https://jasmine.github.io/).\n\nIt doesn't work on [node.js](https://nodejs.org/en/), which lacks support for\nWHATWG Streams. I found a [github\nproject](https://github.com/nodejs/whatwg-stream) for support but it's just a\nskeleton.\n\nIt uses the [npm](https://www.npmjs.com/) ecosystem for package management.\n\n## Development notes\n\nContributions welcome. There's no CI setup (yet) but each commit should be\ntested via:\n\n```\n$ npm install\n$ npm test\n$ npm run lint\n```\n\nPlease follow the [Google Javascript style\nguide](https://google.github.io/styleguide/jsguide.html).\n\n## Author\n\nScott Lamb \u0026lt;slamb@slamb.org\u003e\n\n## License\n\nYour choice of MIT or Apache; see [LICENSE-MIT.txt](LICENSE-MIT.txt) or\n[LICENSE-APACHE](LICENSE-APACHE.txt), respectively.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscottlamb%2Fmultipart-stream-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscottlamb%2Fmultipart-stream-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscottlamb%2Fmultipart-stream-js/lists"}