{"id":13607615,"url":"https://github.com/ssbc/multiserver","last_synced_at":"2025-04-12T14:30:44.565Z","repository":{"id":23147069,"uuid":"26502371","full_name":"ssbc/multiserver","owner":"ssbc","description":"A single interface that can work with multiple protocols, and multiple transforms of those protocols (eg, security layer)","archived":false,"fork":false,"pushed_at":"2023-07-13T16:06:20.000Z","size":241,"stargazers_count":106,"open_issues_count":4,"forks_count":28,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-04-10T22:27:15.548Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ssbc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2014-11-11T20:12:01.000Z","updated_at":"2024-12-17T13:19:13.000Z","dependencies_parsed_at":"2023-07-17T10:00:30.065Z","dependency_job_id":null,"html_url":"https://github.com/ssbc/multiserver","commit_stats":null,"previous_names":["ssb-js/multiserver","dominictarr/multiserver"],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbc%2Fmultiserver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbc%2Fmultiserver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbc%2Fmultiserver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbc%2Fmultiserver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ssbc","download_url":"https://codeload.github.com/ssbc/multiserver/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248581004,"owners_count":21128091,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T19:01:20.136Z","updated_at":"2025-04-12T14:30:44.545Z","avatar_url":"https://github.com/ssbc.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# multiserver\n\nA single interface that can work with multiple protocols,\nand multiple transforms of those protocols (eg, security layer)\n\n## motivation\n\nDeveloping a p2p system is hard. Especially hard is upgrading protocol layers.\nThe contemporary approach is to [update code via a backdoor](https://whispersystems.org/blog/the-ecosystem-is-moving/),\nbut as easily as security can be added, [it can be taken away](https://nakamotoinstitute.org/trusted-third-parties/).\n\nBefore you can have a protocol, you need a connection between peers.\nThat connection is over some form of network transport,\nprobably encrypted with some encryption scheme, possibly\ncompression or other layers too.\n\nUsually, two peers connect over a standard networking transport\n(probably tcp) then they have a negotiation to decide\nwhat the next layer (of encryption, for example) should be.\nThis allows protocol implementators to roll out improved\nversions of the encryption protocol. However, it does\nnot allow them to upgrade the negotiation protocol!\nIf a negotiation protocol has a vulnerability it's much\nharder to fix, and since the negotiation needs to be unencrypted,\nit tends to reveal a lot about program the server is running.\n[in my opinion, it's time to try a different way.](https://github.com/ipfs/go-ipfs/pull/34)\n\nSome HTTP APIs provide upgradability in a better, simpler way by\nputting a version number within the url. A new version of\nthe API can then be used without touching the old one at all.\n\nmultiserver adapts this approach to lower level protocols.\nInstead of negotiating which protocol to use, run multiple\nprotocols side by side, and consider the protocol part of the address.\n\nMost network systems have some sort of address look up,\nthere is peer identifier (such it's domain) and then\na system that is queried to map that domain to the lower level\nnetwork address (such as it's ip address, retrieved via a DNS (Domain Name System) request)\nTo connect to a website secured with https, first\nyou look up the domain via DNS, then connect to the server.\nThen start a tls connection to that server, in which\na cyphersuite is negotiated, and a certificate is provided\nby the server. (this certifies that the server really\nowns that domain)\n\nIf it was using multiserver, DNS would respond with a list of cyphersuites,\n(encoded as multiserver addresses) and then you'd connect directly to a server and start using the protocol, without negotiation.\np2p systems like scuttlebutt also usually have a lookup,\nbut usually mapping from a public key to an ip address.\nSince a look up is needed anyway, it's a good place\nto provide information about the protocol that server speaks!\n\nThis enables you to do two things, upgrade and bridging.\n\n### upgrade\n\nIf a peer wants to upgrade from *weak* protocol\nto a *strong* one, they simply start serving *strong* via another port,\nand advertise that in the lookup system.\nNow peers that have support for *strong* can connect via that protocol.\n\nOnce most peers have upgraded to strong, support for *weak* can be discontinued.\n\nThis is just how some services (eg, github) have an API version\nin their URL scheme. It is now easy to use two different\nversions in parallel. later, they can close down the old API.\n\n``` js\nvar MultiServer = require('multiserver')\nvar chloride = require('chloride')\nvar keys = chloride.crypto_sign_keypair()\nvar appKey = \"dTuPysQsRoyWzmsK6iegSV4U3Qu912vPpkOyx6bPuEk=\"\n\nfunction accept_all (id, cb) {\n  cb(null, true)\n}\nvar ms = MultiServer([\n  [ //net + secret-handshake\n    require('multiserver/plugins/net')({port: 3333}),\n    require('multiserver/plugins/shs')({\n      keys: keys,\n      appKey: appKey, //application key\n      auth: accept_all\n    }),\n  ],\n  [ //net + secret-handshake2\n    //(not implemented yet, but incompatible with shs)\n    require('multiserver/plugins/net')({port: 4444}),\n    //this protocol doesn't exist yet, but it could.\n    require('secret-handshake2')({\n      keys: keys,\n      appKey: appKey, //application key\n      auth: accept_all\n    }),\n  ]\n])\n\nconsole.log(ms.stringify())\n\n//=\u003e net:\u003chost\u003e:3333~shs:\u003ckey\u003e;net:\u003chost\u003e:4444~shs2:\u003ckey\u003e\n\n//run two servers on two ports.\n//newer peers can connect directly to 4444 and use shs2.\n//this means the protocol can be _completely_ upgraded.\nms.server(function (stream) {\n  console.log('connection from', stream.address)\n})\n\n//connect to legacy protocol\nms.client('net:\u003chost\u003e:3333~shs:\u003ckey\u003e', function (err, stream) {\n  //...\n})\n\n//connect to modern protocol\nms.client('net:\u003chost\u003e:4444~shs2:\u003ckey\u003e', function (err, stream) {\n  //...\n})\n\n```\n\n### bridging\n\nBy exposing multiple network transports as part of\nthe same address, you can allow connections from\npeers that wouldn't have been able to connect otherwise.\n\nRegular servers can do TCP. Desktop clients can speak TCP,\nbut can't create TCP servers that other desktop computers can connect to reliably.\nBrowsers can use WebSockets and WebRTC.\nWebRTC gives you p2p, but needs an introducer.\nAnother option is [utp](https://github.com/mafintosh/utp-native)\n- probably the most convenient, because it doesn't need an introducer\non _every connection_ (but it does require some bootstrapping),\nbut that doesn't work in the browser either.\n\n``` js\nvar MultiServer = require('multiserver')\n\nvar ms = MultiServer([\n  require('multiserver/plugins/net')({port: 1234}),\n  require('multiserver/plugins/ws')({port: 2345})\n])\n\n//start a server (for both protocols!)\n//returns function to close the server.\nvar close = ms.server(function (stream) {\n  //handle incoming connection\n})\n\n//connect to a protocol. uses whichever\n//handler understands the address (in this case, websockets)\nvar abort = ms.client('ws://localhost:1234', function (err, stream) {\n  //...\n})\n\n//at any time abort() can be called to cancel the connection attempt.\n//if it's called after the connection is established, it will\n//abort the stream.\n```\n\n## address format\n\nAddresses describe everything needed to connect to a peer.\neach address is divided into protocol sections separated by `~`.\nEach protocol section is divided itself by `:`. A protocol section\nstarts with a name for that protocol, and then whatever arguments\nthat protocol needs. The syntax of the address format is defined by [multiserver-address](https://github.com/ssbc/multiserver-address)\n\nFor example, the address for my ssb pubserver is:\n```\nnet:wx.larpa.net:8008~shs:DTNmX+4SjsgZ7xyDh5xxmNtFqa6pWi5Qtw7cE8aR9TQ=\n```\nThat says use the `net` protocol (TCP) to connect to the domain `wx.larpa.net`\non port `8008`, and then encrypt the session using `shs` ([secret-handshake](https://github.com/auditdrivencrypto/secret-handshake))\nto the public key `DTNmX+4SjsgZ7xyDh5xxmNtFqa6pWi5Qtw7cE8aR9TQ=`.\n\nUsually, the first section is a network protocol, and the rest are transforms,\nsuch as encryption or compression.\n\nMultiserver makes it easy to use multiple protocols at once. For example,\nmy pub server _also_ supports `shs` over websockets.\n\nSo, this is another way to connect:\n\n```\nwss://wx.larpa.net~shs:DTNmX+4SjsgZ7xyDh5xxmNtFqa6pWi5Qtw7cE8aR9TQ=\n```\n\nif your server supports multiple protocols, you can concatenate addresses with `;`\nand multiserver will connect to the first address it understands.\n\n```\nnet:wx.larpa.net:8008~shs:DTNmX+4SjsgZ7xyDh5xxmNtFqa6pWi5Qtw7cE8aR9TQ=;wss://wx.larpa.net~shs:DTNmX+4SjsgZ7xyDh5xxmNtFqa6pWi5Qtw7cE8aR9TQ=\n```\nThis means use net, or wss. In some contexts, you might have a peer that understands\nwebsockets but not net (for example a browser), as long as a server speaks at least\none protocol that a peer can understand, then they can communicate.\n\n## scopes\n\naddress also have a scope. This relates to where they\ncan be connected to. Default supported scopes are:\n\n* device - can connect only if on the same device\n* local - can connect from same wifi (local network)\n* public - can connect from public global internet.\n\nsome transport plugins work only on particular scopes.\n\nwhen `stringify(scope)` is called, it will return\njust the accessible addresses in that scope.\n\n## plugins\n\nA multiserver instance is set up by composing a selection\nof plugins that construct the networking transports,\nand transforms that instance supports.\n\nThere are two types of plugins, transports and transforms.\n\n### `net({port,host,scope})`\n\nTCP is a `net:{host}:{port}` port is not optional.\n\n``` js\nvar Net = require('multiserver/plugins/net')`\nNet({port: 8889, host: 'mydomain.com'}).stringify()  =\u003e 'net:mydomain.com:8889'\nNet({port: 8889, host: 'fe80::1065:74a4:4016:6266:4849'}).stringify()  =\u003e 'net:fe80::1065:74a4:4016:6266:4849:8889'\nNet({port: 8889, host: 'fe80::1065:74a4:4016:6266:4849', scope: 'device'}).stringify()  =\u003e 'net:fe80::1065:74a4:4016:6266:4849:8889'\n```\n\n### `WebSockets({host,port,scope,handler?,key?,cert?})`\n\ncreate a websocket server. Since websockets are\njust a special mode of http, this also creates a http\nserver. If `opts.handler` is provided, requests\nto the http server can be handled, this is optional.\n\nWebSockets `ws://{host}:{port}?` port defaults to 80 if not provided.\n\nWebSockets over https is `wss://{host}:{port}?` where port is\n443 if not provided.\n\nIf `opts.key` and `opts.cert` are provided as paths, a https server\nwill be spawned.\n\n``` js\nvar WebSockets = require('multiserver/plugins/ws`)\n\nvar ws = WebSockets({\n  port: 1234,\n  host: 'mydomain.com',\n  handler: function (req, res) {\n    res.end('\u003ch1\u003ehello\u003c/h1\u003e')\n  },\n  scope:...\n})\n\nws.stringify() =\u003e 'ws://mydomain.com:1234'\n```\n\n### `Onion()`\n\nConnect over tor using local proxy to dæmon (9050) or tor browser (9150). \nBoth will be tried to find a suitable tor instance. \nThe tor ports are unconfigurable. The standard\ntor ports are always used.\n\nThis plugin does not support creating a server.\nYou should use tor's configuration files to send incoming connections to a `net`\ninstance as a hidden service.\n\nAn accepted onion address looks like: `onion:{host}:{port}`\nport is not optional. This plugin does not return\nan address, so you must construct this address manually.\n\n``` js\nvar Onion = require('multiserver/plugins/onion`)\n\n\nvar onion = WebSockets({\n  //no config is needed except scope, but you\n  //surely will use this with \"public\" which is the default\n  //scope:'public'\n})\n\nws.stringify() =\u003e null\n```\n\n\n### `Bluetooth({bluetoothManager})`\n\nThe [multiserver-bluetooth](https://github.com/Happy0/multiserver-bluetooth) module implements a multiserver protocol for to communicate over Bluetooth Serial port.\n\n### `reactnative = require('multiserver-rn-channel')`\n\nThe [multiserver-rn-channel](http://npm.im/multiserver-rn-channel) module implementes\na multiserver protocol for use inbetween the reactnative nodejs process and browser process.\n\n### `SHS({keys,timeout?,appKey,auth})`\n\nSecret-handshake is `shs:{public_key}:{seed}?`. `seed` is used to create\na one-time shared private key, that may enable a special access.\nFor example, you'll see that ssb invite codes have shs with two sections\nfollowing. Normally, only a single argument (the remote public key) is necessary.\n\n``` js\nvar SHS = require('multiserver/plugins/shs')\n\nvar shs = SHS({\n  keys: keys,\n  timeout: //set handshake timeout, if unset falls through to secret-handshake default\n  appKey: //sets an appkey\n  auth: function (id, cb) {\n    if(isNotAuthorized(id))\n      cb(new Error())\n    else\n      cb(null, authenticationDetails)\n  }\n})\nshs.stringify() =\u003e 'shs:{keys.publicKey.toString('base64')}\n```\n\nnote, if the `auth` function calls back a truthy value,\nit is considered authenticated. The value called back\nmay be an object that represents details of the authentication.\nwhen a successful connection goes through `shs` plugin,\nthe stream will have an `auth` property, which is the value called back from `auth`,\nand a `remote` property (the id of remote key).\n\n### `Noauth({keys})`\n\nThis authenticates any connection without any encryption.\nThis should only be used on `device` scoped connections,\nsuch as if net is bound strictly to localhost,\nor a unix-socket. Do not use with ws or net bound to public addresses.\n\n``` js\nvar Noauth = require('multiserver/plugins/noauth')\n\nvar noauth = Noauth({\n  keys: keys\n})\nshs.stringify() =\u003e 'shs:{keys.publicKey.toString('base64')}\n\n```\n\nstreams passing through this will look like an authenticated shs connection.\n\n### `Unix = require('multiserver/plugins/unix-socket')`\n\nnetwork transport is unix socket. to connect to this\nyou must have access to the same file system as the server.\n\n``` js\nvar Unix = require('multiserver/plugins/unix-socket')\n\nvar unix = Unix({\n  path: where_to_put_socket,\n  scope: ... //defaults to device\n})\n\nunix.stringify() =\u003e \"unix:{where_to_put_socket}\"\n```\n\n\n### createMultiServer([[transport,transforms...],...])\n\nA server that runs multiple protocols on different ports can simply join them\nwith `;` and clients should connect to their preferred protocol.\nclients may try multiple protocols on the same server before giving up,\nbut generally it's unlikely that protocols should not fail independently\n(unless there is a bug in one protocol).\n\nan example of a valid multiprotocol:\n`net:{host}:{port}~shs:{key};ws:{host}:{port}~shs:{key}`\n\n``` js\nvar MultiServer = require('multiserver')\n\nvar ms = MultiServer([\n  [net, shs],\n  [ws, shs],\n  [unix, noauth]\n])\n\nms.stringify('public') =\u003e \"net:mydomain.com:8889~shs:\u003ckey\u003e;ws://mydomain.com:1234~shs:\u003ckey\u003e\"\nms.stringify('device') =\u003e \"unix:{where_to_put_socket}\"\n\nms.server(function (stream) {\n  //now that all the plugins are combined,\n  //ready to use as an actual server.\n})\n```\n\n## interfaces\n\nTo construct a useful multiserver instance,\none or more transport is each connected with zero\nor more transforms. The combine function is\nthe default export from the `multiserver` module.\n\n``` js\nvar MultiServer = require('multiserver')\n\nvar ms = MultiServer([\n  [transport1, transform1],\n  [transport2, transform2, transform3],\n])\n\nvar close = ms.server(function (stream) {\n  //called when a stream connects\n}, onError, onListening)\n```\n\n```\ncreateMultiServer([[Transform, Transports*,...]], *]) =\u003e MultiServer\n```\n\na MultiServer has the same interface as a Transport,\nbut using a combined multiserver instance as a transport\nis **not** supported.\n\n## createTransport(Options) =\u003e Transport\n\nThe transport exposes a name and the ability to\ncreate and connect to servers running that transport.\n\n``` js\nTransport =\u003e {\n  // that describes the sub protocols\n  name,\n  // connect to server with address addr.\n  client (addr, cb),\n  // start the server\n  server (onConnect, onError, onListening),\n  // return string describing how to connect to the server, aka, \"the address\"\n  // the address applies to a `scope`.\n  stringify(scope),\n  // parse the addr,\n  // normally this would probably return the\n  // Options used to create the transport.\n  parse(string) =\u003e Options\n}\n```\n\n## createTransform(options) =\u003e Transform\n\n``` js\nTransform =\u003e {\n  name: string,\n  create(Options) =\u003e (stream, cb(null, transformed_stream)),\n  parse (str) =\u003e Options,\n  stringify() =\u003e string,\n}\n```\n\nnote the create method on a Transform takes Options,\nand returns a function that takes a stream and a callback,\nand then calls back the transformed stream.\nIn all cases the stream is a [duplex stream](https://github.com/pull-stream/pull-stream)\n\n## License\n\nMIT\n\n\n\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fssbc%2Fmultiserver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fssbc%2Fmultiserver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fssbc%2Fmultiserver/lists"}