{"id":13483004,"url":"https://github.com/hackergrrl/ipfs-hyperlog","last_synced_at":"2025-09-13T17:33:36.142Z","repository":{"id":57276569,"uuid":"51556808","full_name":"hackergrrl/ipfs-hyperlog","owner":"hackergrrl","description":":link: IPFS Merkle DAG that replicates based on append-only logs and causal linking.","archived":false,"fork":false,"pushed_at":"2017-12-05T20:25:45.000Z","size":102,"stargazers_count":57,"open_issues_count":0,"forks_count":6,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-05-21T01:04:53.113Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hackergrrl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-02-12T00:01:58.000Z","updated_at":"2023-09-08T17:06:56.000Z","dependencies_parsed_at":"2022-09-15T08:42:55.123Z","dependency_job_id":null,"html_url":"https://github.com/hackergrrl/ipfs-hyperlog","commit_stats":null,"previous_names":["noffle/ipfs-hyperlog"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hackergrrl%2Fipfs-hyperlog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hackergrrl%2Fipfs-hyperlog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hackergrrl%2Fipfs-hyperlog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hackergrrl%2Fipfs-hyperlog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hackergrrl","download_url":"https://codeload.github.com/hackergrrl/ipfs-hyperlog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232897462,"owners_count":18593525,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T17:01:07.389Z","updated_at":"2025-01-07T15:06:58.502Z","avatar_url":"https://github.com/hackergrrl.png","language":"JavaScript","funding_links":[],"categories":["Modules","others"],"sub_categories":[],"readme":"# ipfs-hyperlog\n\n[![build status](http://img.shields.io/travis/noffle/ipfs-hyperlog.svg?style=flat)](http://travis-ci.org/noffle/ipfs-hyperlog)\n\n\u003e [IPFS](https://ipfs.io)-compatible [Merkle DAG](https://github.com/jbenet/random-ideas/issues/20) that replicates based on scuttlebutt logs and causal linking\n\n## Installation\n\nipfs-hyperlog depends on [Node.js](http://nodejs.org) and [npm](https://npmjs.com).\n\n```\nnpm install ipfs-hyperlog\n```\n\n## Background\n\n`ipfs-hyperlog` is a drop-in replacement for\n[@mafintosh](https://github.com/mafintosh)'s\n[`hyperlog`](https://github.com/mafintosh/hyperlog). Its key difference is that\nit creates a Merkle DAG that is *binary compatible* with IPFS objects. This\nmeans any node of any DAG built using ipfs-hyperlog can be replicated to and\nfrom the IPFS network as well!\n\n### Why IPFS?\n\nThe peer-to-peer IPFS network excels at serving and replicating immutable,\nhighly available, permanent data.\n\nHyperlog DAGs can now be replicated to IPFS for permanent storage!\n\n### Why hyperlog?\n\nHyperlog is great for quick replication over a transport-agnostic stream!\n\nIn addition, it has a great ecosystem of powerful modules that IPFS can now take\nadvantage of:\n\n1. [hyperlog-index](https://github.com/substack/hyperlog-index) - forking\n   indexes for hyperlog\n2. [hyperkv](https://github.com/substack/hyperkv) - p2p key/value store over a\n   hyperlogusing a multi-value register conflict strategy\n3. [swarmlog](https://github.com/substack/swarmlog) - create a p2p webrtc swarm\n   around a hyperlog\n4. and [many more!](https://www.npmjs.com/search?q=hyperlog)\n\n\n## Usage\n\n### Create and link nodes\n\n``` js\nvar hyperlog = require('ipfs-hyperlog')\n\nvar log = hyperlog(db) // where db is a levelup instance\n\n// add a node with value 'hello' and no links\nlog.add(null, 'hello', function(err, node) {\n  console.log('inserted node', node)\n\n  // insert 'world' with a link back to the above node\n  log.add([node.key], 'world', function(err, node) {\n    console.log('inserted new node', node)\n  })\n})\n```\n\n### Replicate graph\n\nTo replicate this log with another one simply use `log.replicate()` and pipe it together with a replication stream from another log.\n\n``` js\nvar l1 = hyperlog(db1)\nvar l2 = hyperlog(db2)\n\nvar s1 = l1.replicate()\nvar s2 = l2.replicate()\n\ns1.pipe(s2).pipe(s1)\n\ns1.on('end', function() {\n  console.log('replication ended')\n})\n```\n\nA detailed write-up on how this replication protocol works will be added to this repo in the near\nfuture. For now see the source code.\n\n## API\n\n#### `log = hyperlog(db, [options])`\n\nCreate a new log instance. Options include:\n\n``` js\n{\n  id: 'a-globally-unique-peer-id',\n  valueEncoding: 'a levelup-style encoding property' // example: 'json'\n}\n```\n\nYou can also pass in a `identity` and a `sign` and `verify` function\nwhich can be used to create a signed log\n\n``` js\n{\n  identity: aPublicKeyBuffer, // will be added to all nodes you insert\n  sign: function (node, cb) {\n    // will be called with all nodes you add\n    var signatureBuffer = someCrypto.sign(node.key, mySecretKey)\n    cb(null, signatureBuffer)\n  },\n  verify: function (node, cb) {\n    // will be called with all nodes you receive\n    if (!node.signature) return cb(null, false)\n    cb(null, someCrypto.verify(node.key, node.signature. node.identity))\n  }\n}\n```\n\n#### `log.add(links, value, opts={}, [cb])`\n\nAdd a new node to the graph. `links` should be an array of node keys that this node links to.\nIf it doesn't link to any nodes use `null` or an empty array. `value` is the value that you want to store\nin the node. This should be a string or a buffer. The callback is called with the inserted node:\n\n``` js\nlog.add([link], value, function(err, node) {\n  // node looks like this\n  {\n    change: ... // the change number for this node in the local log\n    key:   ... // the hash of the node. this is also the key of the node\n    value:  ... // the value (as a buffer) you inserted\n    log:    ... // the peer log this node was appended to\n    seq:    ... // the peer log seq number\n    links: ['hash-of-link-1', ...]\n  }\n})\n```\n\nOptionally supply an `opts.valueEncoding`.\n\n#### `log.append(value, opts={}, [cb])`\n\nAdd a value that links all the current heads.\n\nOptionally supply an `opts.valueEncoding`.\n\n#### `log.get(hash, opts={}, cb)`\n\nLookup a node by its hash. Returns a node similar to `.add` above.\n\nOptionally supply an `opts.valueEncoding`.\n\n#### `log.heads(opts={}, cb)`\n\nGet the heads of the graph as a list. A head is node that no other node\nlinks to.\n\n``` js\nlog.heads(function(err, heads) {\n  console.log(heads) // prints an array of nodes\n})\n```\n\nThe method also returns a stream of heads which is useful\nif, for some reason, your graph has A LOT of heads\n\n``` js\nvar headsStream = log.heads()\n\nheadsStream.on('data', function(node) {\n  console.log('head:', node)\n})\n\nheadsStream.on('end', function() {\n  console.log('(no more heads)')\n})\n```\n\nOptionally supply an `opts.valueEncoding`.\n\n#### `changesStream = log.createReadStream([options])`\n\nTail the changes feed from the log. Everytime you add a node to the graph\nthe changes feed is updated with that node.\n\n``` js\nvar changesStream = log.createReadStream({live:true})\n\nchangesStream.on('data', function(node) {\n  console.log('change:', node)\n})\n```\n\nOptions include:\n\n``` js\n{\n  since: changeNumber     // only returns changes AFTER the number\n  live: false             // never close the change stream\n  tail: false             // since = lastChange\n  limit: number           // only return up to `limit` changes\n  until: number           // (for non-live streams) only returns changes BEFORE the number\n  valueEncoding: 'binary'\n}\n```\n\n#### `replicationStream = log.replicate([options])`\n\nReplicate the log to another one using a replication stream.\nSimply pipe your replication stream together with another log's replication stream.\n\n``` js\nvar l1 = hyperlog(db1)\nvar l2 = hyperlog(db2)\n\nvar s1 = l1.createReplicationStream()\nvar s2 = l2.createReplicationStream()\n\ns1.pipe(s2).pipe(s1)\n\ns1.on('end', function() {\n  console.log('replication ended')\n})\n```\n\nOptions include:\n\n``` js\n{\n  mode: 'push' | 'pull' | 'sync', // set replication mode. defaults to sync\n  live: true, // keep the replication stream open. defaults to false\n  metadata: someBuffer, // send optional metadata as part of the handshake\n  frame: true // frame the data with length prefixes. defaults to true\n}\n```\n\nIf you send `metadata` it will be emitted as an `metadata` event on the stream.\nA detailed write up on how the graph replicates will be added later.\n\n#### log.on('preadd', function (node) {})\n\nOn the same tick as `log.add()` is called, this event fires with the `node`\nabout to be inserted into the log. At this stage of the add process, node has\nthese properties:\n\n* `node.log`\n* `node.key`\n* `node.value`\n* `node.links`\n\n#### log.on('add', function (node) {})\n\nAfter a node has been successfully added to the log, this event fires with the\nfull `node` object that the callback to `.add()` gets.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhackergrrl%2Fipfs-hyperlog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhackergrrl%2Fipfs-hyperlog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhackergrrl%2Fipfs-hyperlog/lists"}