{"id":13483097,"url":"https://github.com/mozilla/libdweb","last_synced_at":"2025-04-04T18:02:19.334Z","repository":{"id":33151509,"uuid":"132958626","full_name":"mozilla/libdweb","owner":"mozilla","description":"Extension containing an experimental libdweb APIs","archived":false,"fork":false,"pushed_at":"2023-07-06T21:50:41.000Z","size":26920,"stargazers_count":444,"open_issues_count":19,"forks_count":19,"subscribers_count":39,"default_branch":"master","last_synced_at":"2025-03-16T21:23:01.109Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mozilla.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"License.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2018-05-10T21:59:57.000Z","updated_at":"2025-02-15T13:51:40.000Z","dependencies_parsed_at":"2024-04-23T18:23:45.203Z","dependency_job_id":"dcc94352-71a4-464c-86c6-fbbef41538d5","html_url":"https://github.com/mozilla/libdweb","commit_stats":{"total_commits":156,"total_committers":11,"mean_commits":"14.181818181818182","dds":"0.20512820512820518","last_synced_commit":"a79bb1d495100baf949470321a3c20303b0ebc84"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mozilla%2Flibdweb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mozilla%2Flibdweb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mozilla%2Flibdweb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mozilla%2Flibdweb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mozilla","download_url":"https://codeload.github.com/mozilla/libdweb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245433536,"owners_count":20614507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T17:01:08.218Z","updated_at":"2025-03-28T17:02:26.723Z","avatar_url":"https://github.com/mozilla.png","language":"JavaScript","readme":"# Status\n\nIn the process of migration to https://github.com/mozilla/gecko-dev/, once completed repo will get archieved. Please [submit new bugs](https://bugzilla.mozilla.org/enter_bug.cgi?product=WebExtensions\u0026component=Experiments) in bugzilla and make them blockers for [libdweb metabug](https://bugzilla.mozilla.org/show_bug.cgi?id=libdweb)\n\n# libdweb\n\n[![travis][travis.icon]][travis.url]\n[![package][version.icon] ![downloads][downloads.icon]][package.url]\n[![styled with prettier][prettier.icon]][prettier.url]\n\nThis repository hosts a community effort to implement [experimental APIs][webextension experiments] for Firefox WebExtensions with a goal of enabling dweb protocols in Firefox through browser add-ons. The long term goal of this project is to integrate these APIs into the [WebExtensions][new apis] ecosystem.\n\n## Participation\n\nYou can help this effort in following ways:\n\n1. Use these APIs to make something illustrating its value, to build the case for adoption in the core WebExtension API set.\n2. Get involved in driving this effort: Help with an API implementation, maintenance, testing, code samples, etc.\n3. Help build [API adapters][] to enable seamless integration with existing libraries.\n4. Join our IRC channel: #dweb on irc.mozilla.org\n\n## Status: In active development\n\n| API                   | Status |\n| --------------------- | ------ |\n| [Protocol Handler][]  | 🐥     |\n| [Service Discovery][] | 🐣     |\n| [File System][]       | 🐣     |\n| [UDP Socket][]        | 🐣     |\n| [TCP Socket][]        | 🐣     |\n\n- 🥚 : In design phase\n- 🐣 : Work in progress\n- 🐥 : Try it out\n- 🐓 : Usable\n\n## API overview\n\n**Note:** You can try all the examples after you've cloned the repo and got the toolchain setup by running `npm install`. You will also need [Firefox Nightly][] to run the demos.\n\n### Protocol API\n\nThe Protocol API allows you to handle custom protocols from your Firefox extension. This is different from the existing [WebExtensions protocol handler API][webextensions protocol_handlers] in that it does not register a website for handling corresponding URLs but rather allows your WebExtension to implement the handler.\n\nThe following example implements a simple `dweb://` protocol. When firefox is navigated to `dweb://hello/world`, for example, it will invoke your registered handler and pass it a `request` object containing request URL as `request.url` string property. Your handler is expected to return a repsonse with a `content` that is [async iterator][] of [`ArrayBuffer`][]s. In our example we use a `respond` [async generator][] function to respond with some HTML markup.\n\n```js\nbrowser.protocol.registerProtocol(\"dweb\", request =\u003e {\n  return {\n    contentType: \"text/html\",\n    content: respond(request.url)\n  }\n})\n\nasync function* respond(text) {\n  const encoder = new TextEncoder(\"utf-8\")\n  yield encoder.encode(\"\u003ch1\u003eHi there!\u003c/h1\u003e\\n\").buffer\n  yield encoder.encode(\n    `\u003cp\u003eYou've succesfully loaded \u003cstrong\u003e${request.url}\u003c/strong\u003e\u003cp\u003e`\n  ).buffer\n}\n```\n\nGiven that `response.content` is an [async iterator][] it is also possible to stream response content as this next example illustrates.\n\n```js\nbrowser.protocol.registerProtocol(\"dweb\", request =\u003e {\n  switch (request.url) {\n    case \"dweb://stream/\": {\n      return {\n        contentType: \"text/html\",\n        content: streamRespond(request)\n      }\n    }\n    default: {\n      return {\n        contentType: \"text/html\",\n        content: respond(request.url)\n      }\n    }\n  }\n})\n\nasync function* streamRespond(request) {\n  const encoder = new TextEncoder(\"utf-8\")\n  yield encoder.encode(\"\u003ch1\u003eSay Hi to endless stream!\u003c/h1\u003e\\n\").buffer\n  let n = 0\n  while (true) {\n    await new Promise(resolve =\u003e setTimeout(resolve, 1000))\n    yield encoder.encode(`\u003cp\u003eChunk #${++n}\u003cp\u003e`).buffer\n  }\n}\n```\n\nYou can see the demo of the example above in [Firefox Nightly][] by running following command, and then navigating to [dweb://hello/world](dweb://hello/world) or [dweb://stream/](dweb://stream/)\n\n```\nnpm run demo:protocol\n```\n\n![protocol demo](./demo/protocol/protocol.gif)\n\n### Service Discovery API\n\nAPI provides DNS-Based Service Discovery API as per [rfc6763][]. Following example illustrates how this API can be used to discover available `http` services in the network.\n\n```js\nvoid (async () =\u003e {\n  const services = browser.ServiceDiscovery.discover({\n    type: \"dweb\",\n    protocol: \"tcp\" // Must be \"tcp\" or \"udp\"\n  })\n\n  console.log(\"Start discovery\", services.query)\n  for await (const service of services) {\n    if (service.lost) {\n      console.log(\"Lost service\", service)\n    } else {\n      console.log(\"Found service\", {\n        name: service.name,\n        type: service.type,\n        protocol: service.protocol\n      })\n\n      for (const {\n        address,\n        port,\n        host,\n        attributes\n      } of await service.addresses()) {\n        console.log(\n          `Service ${service.name} available at ${host} ${address}:${port}`,\n          attributes\n        )\n      }\n    }\n  }\n  console.log(\"End discovery\", services.query)\n})()\n```\n\nAPI also allows you to announce service that others on the network can discover. Following example illustrates that:\n\n```js\nvoid (async () =\u003e {\n  const service = await browser.ServiceDiscovery.announce({\n    name: \"My dweb service\",\n    type: \"dweb\",\n    protocol: \"tcp\", // must be \"tcp\" or \"udp\"\n    port: 3000, // ommting port will just assign you available one.\n    attributes: {\n      // optional txt records\n      version: \"1.0.\"\n    }\n  })\n\n  console.log(\"Service annouced\", {\n    name: service.name, // Note: Colud be different like \"My dweb service (2)\"\n    type: service.type,\n    protocol: service.protocol,\n    port: service.port,\n    attributes: service.attributes // Will be null if was omitted\n  })\n\n  // Wait for a 1 minute and expire service announcement\n  await new Promise(timeout =\u003e setTimeout(timeout, 60 * 1000))\n  await service.expire()\n  console.log(`Service expired`)\n})()\n```\n\n#### Demo\n\nYou can try demo WebExtension that discovers and displays `http`\nservices in your local network when button in the toolbar is clicked. You can\nrun it in [Firefox Nightly][] via the following command\n\n```\nnpm run demo:discovery\n```\n\n![discovery button](./demo/discovery/preview.gif)\n\n### FileSystem API\n\nFileSystem API provides access to an OS file system, but restricted to a user chosen directory. Below example illustrates writing a content to a file in user chosen directory.\n\n```js\nvoid (async () =\u003e {\n  const volume = await browser.FileSystem.mount({\n    read: true,\n    write: true\n  })\n  console.log(\"Mounted\", volume)\n  localStorage.setItem(\"volumeURL\", volume.url)\n\n  const fileURL = new URL(\"hello.md\", volume.url).href\n  const encoder = new TextEncoder()\n  const content = encoder.encode(\"# Hello World\\n\").buffer\n  const size = await browser.FileSystem.writeFile(fileURL, content)\n  console.log(`Wrote ${size} bytes to ${fileURL}`)\n})()\n```\n\nCall to `FileSystem.mount` will notify user that corresponding WebExtension is requesting `read / write` access to the file system, which user can deny or grant by choosing a directory. If user denies to access then promise returned by `mount` will be rejected, if user chooses to grant access to a specific directory the promise will resolve to an object like:\n\n```js\n{\n  url:\"file:///Users/user/dweb/\",\n  readable:true,\n  writable:true\n}\n```\n\nThe rest of the example that writes content into a file should be pretty straight forward.\n\n\u003e **Note:** Granted access will be preserved across sessions, and WebExtension could mount same directory without prompting a user again.\n\nFollowing is a more complete example that will either mount directory that user has already granted access to or request access to new directory otherwise.\n\n```js\nvoid (async () =\u003e {\n  const url = localStorage.getItem(\"volumeURL\")\n  const volume = await browser.FileSystem.mount({ url, read: true })\n\n  const fileURL = new URL(\"hello.md\", volume.url).href\n  const file = await browser.FileSystem.open(fileURL, { read: true })\n  const chunk = await browser.File.read(file, { position: 2, size: 5 })\n  console.log(`Read file fragment from ${fileURL}`, chunk)\n  const decoder = new TextDecoder()\n  const content = decoder.decode(chunk)\n  console.log(`Decode read fragment`, content)\n  await browser.File.close(file)\n})()\n```\n\n\u003e **Note:** Attempting to mount a URL that user has not previously granted access to will fail without even prompting a user.\n\nFileSystem API has many other functions available. You can follow the links for detailed API interface definitions of [`browser.FileSystem`][] and [`browser.File`][]\n\nYou can try demo WebExtension that provides a [REPL][] in the sidebar exposing all of the FileSystem API, which you can run in [Firefox Nightly][] via following command\n\n\u003e **Note:** Commands recognized by [REPL][] correspond to the API functions names and all the parameters are names prefixed by `--` and followed by value.\n\n```\nnpm run demo:fs\n```\n\n![FileSystem](./demo/fs/fs.gif)\n\n### UDPSocket API\n\nAPI provides an implementation of UDP Datagram sockets. Follow the link for detailed API interface for [`browser.UDPSocket`][] which corresponds to `UDPSocketManager`.\n\nThere is also a [@libdweb/dgram-adapter][] project that provides [nodejs dgram][] API adapter.\n\n#### Example\n\nFollowing example opens UDP socket on port `41234` that will act as a server and will continuously print incoming messages.\n\n```js\nvoid (async () =\u003e {\n  const server = await browser.UDPSocket.create({\n    port: 41234\n  })\n  console.log(`listening ${server.address.host}:${server.address.port}`)\n\n  const decoder = new TextDecoder()\n  for await (const { from, data } of browser.UDPSocket.messages(server)) {\n    console.log(`receive message`)\n    const message = decoder.decode(data)\n    console.log(`server got: ${message} from ${from.host}:${from.port}`)\n  }\n})()\n```\n\n\u003e **Note:** Incoming messages are represented via [async iterator][] which can be consumed via `for await`, but be aware that **messages are not buffered** so if you use `await` inside the `for await` block chances are you will miss message which will be dropped.\n\nFollowing example opens socket on arbitrary port and then sends a message to the server socket from the above example.\n\n```js\nvoid (async () =\u003e {\n  const client = await browser.UDPSocket.create({ host: \"127.0.0.1\" })\n  console.log(`opened socket ${client.address.host}:${client.address.port}`)\n  const encoder = new TextEncoder()\n  const message = encoder.encode(\"Hello UDP\").buffer\n  await browser.UDPSocket.send(client, \"127.0.0.1\", 41234, message)\n})()\n```\n\n\u003e **Note**: UDPSocket API unlike one in nodejs is not going to resolve hostnames like `\"localhost\"`. You need to use WebExtensions [dns][webextensions dns] API to resolve hostnames.\n\n\n#### Demo\n\n\n##### P2P Chat Demo\n\nYou can try demo WebExtension that uses UDP multicasting to do peer-to-peer chat in a firefox sidebar. You can run in [Firefox Nightly][] via following command\n\n\u003e **Note**: This is a demo illustrates UDP and Multicasting API.\n\n```\nnpm run demo:p2p-chat\n```\n\n![p2p-chat](./demo/p2p-chat/chat_demo.png)\n\n##### REPL Demo\n\n\nYou can try demo WebExtension that provides a [REPL][] in the sidebar exposing all of the UDPSocket API, which you can run in [Firefox Nightly][] via following command\n\n\u003e **Note:** Commands recognized by [REPL][] correspond to the API functions names and all the parameters are names prefixed by `--` and followed by corresponding values.\n\n```\nnpm run demo:dgram\n```\n\n### TCPSocket API\n\nTCPSocket API provides a client and server socket APIs for [TCP][] networking.\n\n#### Example\n\nFollowing example starts echo TCP server on port `8090`. It will accept incoming\nconnections read first chunk of data, respond by echoing messages back to.\n\n```js\nvoid (async () =\u003e {\n  const encoder = new TextEncoder()\n  const decoder = new TextDecoder()\n\n  const server = await browser.TCPSocket.listen({ port: 8090 })\n  console.log(\"Started TCP Server\", server)\n\n  const onconnect = async client =\u003e {\n    console.log(\"Client connected:\", client)\n    const message = await client.read()\n    console.log(\"Received message from client:\", decoder.decode(message))\n    const response = encoder.encode(`\u003cecho\u003e${decoder.decode(message)}\u003c/echo\u003e`)\n    await client.write(response.buffer)\n  }\n\n  for await (const client of server.connections) {\n    onconnect(client)\n  }\n})()\n```\n\n\u003e **Note:** `server.connections` are represented via [async iterator][] which can be consumed via `for await`, but be aware that connections are **not buffered** which is why handle each connection in `onconnect` function so our server can accept more connections. If you use `await` inside the `for await` block chances are you will miss connection, in which case it will be automatically closed.\n\nFollowing example connects to server from the example above writes a message to it and then reads message received back.\n\n```js\nvoid (async () =\u003e {\n  const encoder = new TextEncoder()\n  const decoder = new TextDecoder()\n\n  const client = await browser.TCPSocket.connect({\n    host: \"localhost\",\n    port: 8090\n  })\n  await client.opened\n  console.log(\"Client connected:\", client)\n\n  await client.write(encoder.encode(\"Hello TCP\").buffer)\n  const response = await client.read()\n  console.log(\"Received response:\", decoder.decode(response))\n})()\n```\n\n\n[travis.icon]: https://travis-ci.com/mozilla/libdweb.svg?branch=master\n[travis.url]: https://travis-ci.com/mozilla/libdweb\n[version.icon]: https://img.shields.io/npm/v/libdweb.svg\n[downloads.icon]: https://img.shields.io/npm/dm/libdweb.svg\n[package.url]: https://npmjs.org/package/libdweb\n[downloads.image]: https://img.shields.io/npm/dm/libdweb.svg\n[downloads.url]: https://npmjs.org/package/libdweb\n[prettier.icon]: https://img.shields.io/badge/styled_with-prettier-ff69b4.svg\n[prettier.url]: https://github.com/prettier/prettier\n[webextension experiments]: https://webextensions-experiments.readthedocs.io/en/latest/index.html\n[new apis]: https://wiki.mozilla.org/WebExtensions/NewAPIs\n[protocol handler]: https://github.com/mozilla/libdweb/blob/master/Readme.md#protocol-api\n[udp socket]: https://github.com/mozilla/libdweb/blob/master/Readme.md#udpsocket-api\n[tcp socket]: https://github.com/mozilla/libdweb/blob/master/Readme.md#tcpsocket-api\n[µtp socket]: https://github.com/mozilla/libdweb/issues/6\n[service discovery]: https://github.com/mozilla/libdweb/blob/master/Readme.md#service-discovery-api\n[file system]: https://github.com/mozilla/libdweb/blob/master/Readme.md#filesystem-api\n[web-ext]: https://www.npmjs.com/package/web-ext\n[firefox nightly]: https://blog.nightly.mozilla.org/\n[api adapters]: https://github.com/libdweb\n[webextensions protocol_handlers]: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/manifest.json/protocol_handlers\n[async iterator]: https://github.com/tc39/proposal-async-iteration#async-iterators-and-async-iterables\n[`arraybuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer\n[async generator]: https://github.com/tc39/proposal-async-iteration#async-generator-functions\n[`browser.filesystem`]: https://github.com/mozilla/libdweb/blob/master/src/FileSystem/FileSystem.js#L8-L39\n[`browser.file`]: https://github.com/mozilla/libdweb/blob/master/src/FileSystem/FileSystem.js#L41-L51\n[repl]: https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop\n[`browser.udpsocket`]: https://github.com/mozilla/libdweb/blob/master/src/UDPSocket/UDPSocket.js\n[webextensions dns]: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/dns\n[@libdweb/dgram-adapter]: https://github.com/libdweb/dgram-adapter\n[nodejs dgram]: https://nodejs.org/api/dgram.html\n[tcp]: https://en.wikipedia.org/wiki/Transmission_Control_Protocol\n[rfc6763]: https://tools.ietf.org/html/rfc6763\n","funding_links":[],"categories":["JavaScript","others","Protocols and Technologies"],"sub_categories":["Web"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmozilla%2Flibdweb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmozilla%2Flibdweb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmozilla%2Flibdweb/lists"}