{"id":18429437,"url":"https://github.com/bigpipe/bigpipe","last_synced_at":"2025-04-08T12:03:51.255Z","repository":{"id":7685798,"uuid":"9049159","full_name":"bigpipe/bigpipe","owner":"bigpipe","description":"BigPipe is a radical new modular web pattern for Node.js","archived":false,"fork":false,"pushed_at":"2016-11-16T20:42:51.000Z","size":1440,"stargazers_count":1176,"open_issues_count":24,"forks_count":94,"subscribers_count":62,"default_branch":"master","last_synced_at":"2025-04-01T11:04:09.303Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://www.bigpipe.io","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bigpipe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null}},"created_at":"2013-03-27T08:02:08.000Z","updated_at":"2025-03-04T10:54:02.000Z","dependencies_parsed_at":"2022-07-15T21:17:23.887Z","dependency_job_id":null,"html_url":"https://github.com/bigpipe/bigpipe","commit_stats":null,"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigpipe%2Fbigpipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigpipe%2Fbigpipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigpipe%2Fbigpipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigpipe%2Fbigpipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bigpipe","download_url":"https://codeload.github.com/bigpipe/bigpipe/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247838441,"owners_count":21004580,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T05:17:11.005Z","updated_at":"2025-04-08T12:03:51.235Z","avatar_url":"https://github.com/bigpipe.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BigPipe\n\n[![Version npm][version]](http://browsenpm.org/package/bigpipe)[![Build Status][build]](https://travis-ci.org/bigpipe/bigpipe)[![Dependencies][david]](https://david-dm.org/bigpipe/bigpipe)[![Coverage Status][cover]](https://coveralls.io/r/bigpipe/bigpipe?branch=master)\n\n[version]: http://img.shields.io/npm/v/bigpipe.svg?style=flat-square\n[build]: http://img.shields.io/travis/bigpipe/bigpipe/master.svg?style=flat-square\n[david]: https://img.shields.io/david/bigpipe/bigpipe.svg?style=flat-square\n[cover]: http://img.shields.io/coveralls/bigpipe/bigpipe/master.svg?style=flat-square\n\nBigPipe is a radical new web framework for Node.JS. The general idea is to\ndecompose web pages into small re-usable chunks of functionality called\n`Pagelets` and pipeline them through several execution stages inside web\nservers and browsers. This allows progressive rendering at the front-end and\nresults in exceptional front-end performance.\n\nMost web frameworks are based on a request and response pattern, a request comes\nin, we process the data and output a template. But before we can output the\ntemplate we have to wait until all data has been received in order for the\ntemplate to be processed. This doesn't make any sense for Node.js applications\nwhere everything is done asynchronously. When receiving your first batch\nof data, why not send it directly to the browser so it can start downloading the\nrequired CSS, JavaScript and render it.\n\nBigPipe is made up over 20 modules whose current status is available at: [HEALTH.md](HEALTH.md)\n\n## Installation\n\nBigPipe is distributed through the node package manager (npm) and is written\nagainst Node.js 0.10.x.\n\n```\nnpm install --save bigpipe\n```\n\n## Versioning\n\nTo keep track of cross module compatibility, the imported components will be synced\non minor releases. For example, `bigpipe@0.5.0` will always be compatible with\n`pagelet@0.5.0` and `pipe.js@0.5.0`.\n\n## Support\n\nGot stuck? Or can't wrap your head around a concept or just want some feedback,\nwe got a dedicated IRC channel for that on Freenode:\n\n- **IRC Server**: `irc.freenode.net`\n- **IRC Room**: `#bigpipe`\n\nStill stuck? Create an issue. Every question you have is a bug in our\ndocumentation and that should be corrected. So please, don't hesitate to create\nissues, many of them.\n\n## Table of Contents\n\n**BigPipe**\n- [Getting started](#getting-started)\n- [BigPipe.createServer()](#bigpipecreateserver)\n- [new BigPipe()](#new-bigpipe)\n- [BigPipe.version](#bigpipeversion)\n- [BigPipe.define()](#bigpipedefine)\n- [BigPipe.before()](#bigpipebefore)\n- [BigPipe.remove()](#bigpiperemove)\n- [BigPipe.disable()](#bigpipedisable)\n- [BigPipe.enable()](#bigpipeenable)\n- [BigPipe.use()](#bigpipeuse)\n\n### Getting started\n\nIn all of these example we assume that your file is setup as:\n\n```js\n'use strict';\n\nvar BigPipe = require('bigpipe');\n```\n\n### BigPipe.createServer()\n\n**public**, _returns BigPipe_.\n\nTo create a BigPipe powered server can simply call the `createServer` method.\nThis creates an HTTP or HTTPS server based on the options provided.\n\n```js\nvar bigpipe = BigPipe.createServer(8080, {\n  pagelets: __dirname +'/pagelets',\n  dist:  __dirname +'/dist'\n});\n```\n\nThe first argument in the function call is port number you want the server to\nlisten on. The second argument is an object with the configuration/options of the\nBigPipe server. The following options are supported:\n\n- **cache** A cache which is used for storing URL lookups. This cache instance\n  should have a `.get(key)` and `.set(key, value)` method. Defaults to `false`\n- **dist** The location of our folder where we can store our compiled CSS and\n  JavaScript to disk. If the path or folder does not exist it will be\n  automatically created. Defaults to `working dir/dist`.\n- **pagelets** A directory that contains your Pagelet definitions or an array of Pagelet\n  constructors. Defaults to `working dir/pagelets`. If you don't provide Pages it\n  will serve a 404 page for every request.\n- **parser** The message parser we should use for our real-time communication.\n  See [Primus] for the available parsers. Defaults to `JSON`.\n- **pathname** The root path of an URL that we can use our real-time\n  communication. This path should not be used by your Pages. Defaults to\n  `/pagelet`\n- **transformer** The transformer or real-time framework we want to use for the\n  real-time communication. We're bundling and using `ws` by default. See [Primus]\n  for the supported transformers. Please note that you do need to add the\n  transformer dependency to your `package.json` when you choose something other\n  than `ws`.\n- **redirect** When creating a HTTPS server you could automatically start an HTTP\n  server which redirects all traffic to the HTTPS equiv. The value is the port\n  number on which this server should be started. Defaults to `false`.\n\nIn addition to the options above, all HTTPS server options are also\nsupported.  When you provide a server with cert and key files or set the\nport number to `443`, it assumes you want to setup up a HTTPS server instead.\n\n```js\nvar bigpipe = BigPipe.createServer(443, {\n  key: fs.readFileSync(__dirname +'/ssl.key', 'utf-8'),\n  cert: fs.readFileSync(__dirname +'/ssl.cert', 'utf-8')\n});\n```\n\nWhen you're creating an HTTPS server you got to option to also setup a simple\nHTTP server which redirects all content to HTTPS instead. This is done by\nsupplying the `redirect` property in the options. The value of this property\nshould be the port number you want this HTTP server to listen on:\n\n```js\nvar bigpipe = BigPipe.createServer(443, {\n  ..\n\n  key: fs.readFileSync(__dirname +'/ssl.key', 'utf-8'),\n  cert: fs.readFileSync(__dirname +'/ssl.cert', 'utf-8'),\n  redirect: 80\n});\n```\n\n### new BigPipe()\n\n**public**, _returns BigPipe_.\n\nIf you want more control over the server creation process you can manually\ncreate a HTTP or HTTPS server and supply it to the BigPipe constructor.\n\n```js\n'use strict';\n\nvar server = require('http').createServer()\n  , BigPipe = require('bigpipe');\n\nvar bigpipe = new BigPipe(server, { options });\n```\n\nIf you are using this pattern to create a BigPipe server instance you need to\nuse the `bigpipe.listen` method to listen to the server. When this is called,\nBigPipe starts compiling all assets, attach the correct listeners to the\nsupplied server, attach event listeners and finally listen on the server. The\nfirst argument of this method is the port number you want to listen on, the\nsecond argument is an optional callback function that should be called when\nserver starts listening for requests.\n\n```js\nbigpipe.listen(8080, function listening() {\n  console.log('hurray, we are listening on port 8080');\n});\n```\n\n### BigPipe.version\n\n**public**, _returns string_.\n\n```js\nbigpipe.version;\n```\n\nThe current version of the BigPipe framework that is running.\n\n### BigPipe.define()\n\n**public**, _returns BigPipe_.\n\n```js\nbigpipe.define(pagelets, callback);\n```\n\nMerge pagelet(s) in the collection of existing pagelets. If given a string it\nwill search that directory for the available Pagelet files. After all dependencies\nhave been compiled the supplied, the callback is called.\n\n```js\nbigpipe.define('../pagelets', function done(err) {\n\n});\n\nbigpipe.define([Pagelet, Pagelet, Pagelet], function done(err) {\n\n}).define('../more/pagelets', function done(err) {\n\n});\n```\n\n### BigPipe.before()\n\n**public**, _returns BigPipe_.\n\n```js\nbigpipe.before(name, fn, options);\n```\n\nBigPipe has two ways of extending it's build-in functionality, we have plugins\nbut also middleware layers. The important difference between these is that\nmiddleware layers allow you to modify the incoming requests **before** they\nreach BigPipe.\n\nThere are 2 different kinds of middleware layers, **async** and **sync**. The\nmain difference is that the **sync** middleware doesn't require a callback. It's\ncompletely optional and ideal for just introducing or modifying the properties\non a request or response object.\n\nAll middleware layers need to be named, this allows you to enable, disable or\nremove the middleware layers. The supplied middleware function can either be a\npre-configured function that is ready to modify the request and responses:\n\n```js\nbigpipe.before('foo', function (req, res) {\n  req.foo = 'bar';\n});\n```\n\nOr an unconfigured function. We assume that a function is unconfigured if the\nsupplied function has less than **2** arguments. When we detect such a function\nwe automatically call it with the context that is set to `BigPipe` and\nthe supplied options object and assume that it returns a configured middleware\nlayer.\n\n```js\nbigpipe.before('foo', function (configure) {\n  return function (req, res) {\n    res.foo = configure.foo;\n  };\n}, { foo: 'bar' });\n```\n\nIf you're building async middleware layers, you simply need to make sure that\nyour function accepts 3 arguments:\n\n- **req** The incoming HTTP request.\n- **res** The outgoing HTTP response.\n- **next** The continuation callback function. This function follows the error\n  first callback pattern.\n\n```js\nbigpipe.before('foo', function (req, res, next) {\n  asyncthings(function (err, data) {\n    req.foo = data;\n    next(err);\n  });\n});\n```\n\n### BigPipe.remove()\n\n**public**, _returns BigPipe_.\n\n```js\nbigpipe.remove(name);\n```\n\nRemoves a middleware layer from the stack based on the given name.\n\n```js\nbigpipe.before('layer', function () {});\nbigpipe.remove('layer');\n```\n\n### BigPipe.disable()\n\n**public**, _returns BigPipe_.\n\n```js\nbigpipe.disable(name);\n```\n\nTemporarily disables a middleware layer. It's not removed from the stack but it's\njust skipped when we iterate over the middleware layers. A disabled middleware layer\ncan be re-enabled.\n\n```js\nbigpipe.before('layer', function () {});\nbigpipe.disable('layer');\n```\n\n### BigPipe.enable()\n\n**public**, _returns BigPipe_.\n\n```js\nbigpipe.enable(name);\n```\n\nRe-enable a previously disabled module.\n\n```js\nbigpipe.disable('layer');\nbigpipe.enable('layer');\n```\n\n### BigPipe.use()\n\n**public**, _returns BigPipe_.\n\n```js\nbigpipe.use(name, plugin);\n```\n\nPlugins can be used to extend the functionality of BigPipe itself. You can\ncontrol the client code as well as the server side code of BigPipe using the\nplugin interface.\n\n```js\nbigpipe.use('ack', {\n  //\n  // Only run on the server.\n  //\n  server: function (bigpipe, options) {\n     // do stuff\n  },\n\n  //\n  // Runs on the client, it's automatically bundled.\n  //\n  client: function (bigpipe, options) {\n     // do client stuff\n  },\n\n  //\n  // Optional library that needs to be bundled on the client (should be a string)\n  //\n  library: '',\n\n  //\n  // Optional plugin specific options, will be merged with Bigpipe.options\n  //\n  options: {}\n});\n```\n\n## Pagelets\n\nPagelets are part of the bigpipe/pagelet module and more information is available at: https://github.com/bigpipe/pagelet\n\n## Events\n\nEverything in BigPipe is build upon the EventEmitter interface. It's either a\nplain EventEmitter or a proper stream. This a summary of the events we emit:\n\nEvent                 | Usage       | Location      | Description\n----------------------|-------------|---------------|-------------------------------\n`log`                 | public      | server        | A new log message\n`transform::pagelet`  | public      | server        | Transform a Pagelet\n`listening`           | public      | server        | The server is listening\n`error`               | public      | server        | The HTTP server received an error\n`pagelet::configure`  | public      | server        | A new pagelet has been configured\n\n## Debugging\n\nThe library makes use of the `diagnostics` module and has all it's internals namespaced\nto `bigpipe:`. These debug messages can be trigged by starting your application\nwith the `DEBUG=` env variable. In order to filter out all messages except\nBigPipe's message run your server with the following command:\n\n```bash\nDEBUG=bigpipe:* node \u003cserver.js\u003e\n```\n\nThe following `DEBUG` namespaces are available:\n\n- `bigpipe:server` The part that handles the request dispatching, page / pagelet\n  transformation and more.\n- `bigpipe:pagelet` Pagelet generation.\n- `bigpipe:compiler` Asset compilation.\n- `bigpipe:primus` BigPipe Primus setup.\n- `pagelet:primus` Pagelet and Primus interactions\n- `pagelet` Pagelet interactions\n\n## Testing\n\nTests are automatically run on [Travis CI] to ensure that everything is\nfunctioning as intended. For local development we automatically install a\n[pre-commit] hook that runs the `npm test` command every time you commit changes.\nThis ensures that we don't push any broken code into this project.\n\n## Inspiration\n\nBigpipe is inspired by the concept behind Facebook's BigPipe. For more details\nread their blog post: [Pipelining web pages for high performance][blog].\n\n\n## License\n\nBigPipe is released under MIT.\n\n[Travis CI]: http://travisci.org\n[pre-commit]: http://github.com/observing/pre-commit\n[Primus]: https://github.com/primus/primus\n[temper]: https://github.com/bigpipe/temper\n[blog]: https://www.facebook.com/notes/facebook-engineering/bigpipe-pipelining-web-pages-for-high-performance/389414033919\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbigpipe%2Fbigpipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbigpipe%2Fbigpipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbigpipe%2Fbigpipe/lists"}