{"id":50992874,"url":"https://github.com/renderffx/bigpipe-mirror","last_synced_at":"2026-06-20T05:04:37.755Z","repository":{"id":361610877,"uuid":"1255103078","full_name":"renderffx/bigpipe-mirror","owner":"renderffx","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-31T13:06:51.000Z","size":25,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-31T14:17:06.819Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/renderffx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-31T12:09:13.000Z","updated_at":"2026-05-31T12:35:53.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/renderffx/bigpipe-mirror","commit_stats":null,"previous_names":["renderffx/bigpipe-mirror"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/renderffx/bigpipe-mirror","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renderffx%2Fbigpipe-mirror","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renderffx%2Fbigpipe-mirror/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renderffx%2Fbigpipe-mirror/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renderffx%2Fbigpipe-mirror/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/renderffx","download_url":"https://codeload.github.com/renderffx/bigpipe-mirror/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renderffx%2Fbigpipe-mirror/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34557553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-20T02:00:06.407Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-20T05:04:37.173Z","updated_at":"2026-06-20T05:04:37.747Z","avatar_url":"https://github.com/renderffx.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BigPipe Engine\n\nBigPipe streaming for Node.js. Pagelets arrive **out of order** - the browser renders each section as soon as it's ready, without waiting for the slowest one.\n\n## Architecture\n\n```\n                           ╔═══════════════════╗\n  Request ───────────────▶ ║  BigPipeEngine    ║\n                           ║                   ║\n                           ║  1. sendHead()    ║ ──▶ Shell HTML + JS runtime\n                           ║  2. sendPagelet() ║ ──▶ \u003cscript\u003e tags (out of order)\n                           ║  3. close()       ║ ──▶ Footer HTML\n                           ╚═══════════════════╝\n                                    │\n                    ┌───────────────┼───────────────┐\n                    ▼               ▼               ▼\n              ┌──────────┐   ┌──────────┐   ┌──────────┐\n              │ Pagelet  │   │ Pagelet  │   │ Pagelet  │\n              │  (nav)   │   │ (feed)   │   │(sidebar) │\n              └──────────┘   └──────────┘   └──────────┘\n```\n\n### How it works\n\n1. **Shell** - The server immediately flushes the HTML `\u003chead\u003e`, page skeleton, and the BigPipe JavaScript runtime. The browser can start rendering the empty layout while data is still being fetched.\n\n2. **Pagelets** - Individual page sections (e.g., nav, feed, sidebar) are fetched concurrently on the server. Each one is streamed to the browser as a `\u003cscript\u003e` tag the moment it's ready, not in any particular order.\n\n3. **Runtime** - The `bigPipe.onPageletArrive()` JS function receives each pagelet, loads any CSS/JS dependencies, and swaps the content into the correct DOM element, removing the loading shimmer.\n\n### Why BigPipe?\n\n- **TTFB (Time to First Byte)** is nearly instant - the shell is sent before any data queries finish\n- **Perceived performance** is better - fast pagelets appear immediately\n- **Out-of-order streaming** means slow backend endpoints don't block fast ones\n- **Progressive enhancement** - CSS/JS dependencies are loaded per-pagelet and deduplicated\n\n## Files\n\n| File | Role |\n|---|---|\n| `BigPipeEngine.js` | Manages the HTTP response stream (head / pagelet / close) |\n| `Pagelet.js` | A self-contained page section with HTML, CSS, JS, and priority |\n| `server.js` | Demo server with a 3-column dashboard example |\n| `test/` | Test suite (node --test, 51 tests) |\n\n## Usage\n\n```js\nimport http from 'node:http';\nimport BigPipeEngine from './BigPipeEngine.js';\nimport Pagelet from './Pagelet.js';\n\nhttp.createServer((req, res) =\u003e {\n  const engine = new BigPipeEngine(res);\n\n  // 1. Send the shell immediately\n  engine.sendHead('\u003chtml\u003e...\u003cdiv id=\"main\"\u003e\u003c/div\u003e');\n\n  // 2. Stream pagelets as data arrives\n  fetchData().then(html =\u003e {\n    engine.sendPagelet(new Pagelet({ id: 'main', html }));\n  });\n\n  // 3. Close when all pagelets are done\n  engine.close('\u003c/html\u003e');\n});\n```\n\n## Demo\n\n```bash\nnode server.js\n```\n\nOpen http://localhost:3000 - watch each column load independently. The sidebar arrives first (~400ms), nav second (~800ms), and the feed last (~2000ms). The page is interactive from the first paint.\n\n## Pagelet API\n\n```js\nconst pagelet = new Pagelet({\n  id: 'unique-id',            // Required - matches the DOM container\n  html: '\u003ch1\u003eHello\u003c/h1\u003e',     // The HTML content (alias: markup)\n  css: ['/styles.css'],       // CSS URLs to load before displaying\n  js: ['/app.js'],            // JS URLs to load after displaying\n  phase: PRIORITY.NORMAL,     // 0-3 or PRIORITY enum value\n});\n```\n\n### Priority levels\n\n| Phase | Constant | Use case |\n|---|---|---|\n| 0 | `PRIORITY.LOW` | Below-fold content, analytics |\n| 1 | `PRIORITY.NORMAL` | Standard page sections |\n| 2 | `PRIORITY.HIGH` | Navigation, search |\n| 3 | `PRIORITY.CRITICAL` | Above-fold, hero content |\n\n## How the runtime works (`clientRuntime`)\n\nThe inline `\u003cscript\u003e` injected by `BigPipeEngine.clientRuntime()` creates a global `window.bigPipe` object with:\n\n- **`onPageletArrive(data)`** - Called by each `\u003cscript\u003e` tag. Loads CSS (deduplicated via `loadedCss`), injects markup into the matching `id` element, then loads JS (deduplicated via `loadedJs`).\n- **`_display(data)`** - Replaces innerHTML and removes the `loading` CSS class (which controls the shimmer animation).\n- **`_complete(id)`** - Hook for post-render logic (e.g., analytics).\n\n## Tests\n\n```bash\nnpm test\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frenderffx%2Fbigpipe-mirror","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frenderffx%2Fbigpipe-mirror","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frenderffx%2Fbigpipe-mirror/lists"}