{"id":21365453,"url":"https://github.com/frameable/pigeon","last_synced_at":"2025-07-13T04:32:01.915Z","repository":{"id":38221768,"uuid":"299369788","full_name":"frameable/pigeon","owner":"frameable","description":"Diff, patch, merge, and synchronize JSON documents with an Automerge-compatible interface","archived":false,"fork":false,"pushed_at":"2022-12-01T19:51:32.000Z","size":563,"stargazers_count":60,"open_issues_count":3,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-10-29T21:45:08.987Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frameable.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-09-28T16:36:04.000Z","updated_at":"2024-09-12T17:48:58.000Z","dependencies_parsed_at":"2023-01-22T11:05:11.272Z","dependency_job_id":null,"html_url":"https://github.com/frameable/pigeon","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frameable%2Fpigeon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frameable%2Fpigeon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frameable%2Fpigeon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frameable%2Fpigeon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frameable","download_url":"https://codeload.github.com/frameable/pigeon/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225855955,"owners_count":17534967,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-22T07:11:16.938Z","updated_at":"2024-11-22T07:11:17.553Z","avatar_url":"https://github.com/frameable.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pigeon\n\nDiff, patch, merge, and synchronize JSON documents with an [Automerge](https://github.com/automerge/automerge)-compatible interface\n\n```javascript\nconst Pigeon = require('@frameable/pigeon')\n\n// initialize our document from an object literal\nlet doc1 = Pigeon.from({\n  cards: [\n    { id: 1, title: 'Rewrite everything in Clojure', done: false },\n    { id: 2, title: 'Rewrite everything in Haskell', done: false },\n  ]\n})\n\n// make a clone of our document\nlet doc2 = Pigeon.from(doc1)\n\n// one user deletes the clojure card\ndoc1 = Pigeon.change(doc1, doc =\u003e doc.cards.splice(0, 1))\n\n// meanwhile another user sets the haskell card to done\ndoc2 = Pigeon.change(doc2, doc =\u003e doc.cards[1].done = true)\n\n// we merge the documents together in any order\nconst merged = Pigeon.merge(doc1, doc2)\n\n// all is well with updates merged together\nassert.deepEqual(merged, {\n  cards: [\n    { id: 2, title: 'Rewrite everything in Haskell', done: true },\n  ]\n})\n```\n\n### Differences from Automerge\n\nPigeon keeps a near fully-compatible interface to [Automerge](https://github.com/automerge/automerge), but the underlying implementation is optimized for a different use case, and makes different trade-offs.  While Automerge optimizes for working offline and merging changes periodically, Pigeon is optimized for online real-time collaboration.\n\n- By default, history will grow only to 1000 items in length, after which oldest entries will be jettisoned\n- Because of the above, [performance is much improved](https://github.com/frameable/pigeon/wiki/Benchmarks) for larger docs with more changes\n- Changes are computed across entire data structures, rather than tracing via proxies\n- Documents need not have a direct common ancestor for patches from one to apply to another\n- Unix timestamps and client ids are used instead of vector clocks to ensure order and determinism\n- Change sets use [JSON-Patch](https://tools.ietf.org/html/rfc6902)-esque paths, and so are more easily introspectable using existing tools\n- Objects should have unique identifiers in order to preserve semantic integrity\n- Changes may be made in-place for situations where performance is critical\n\n\n### Installation\n\n```\nnpm install @frameable/pigeon\n```\n\n\n## API\n\n#### newDoc = Pigeon.from(data, cid=_cid)\n\nCreate a document from an array or object.\n\n#### newDoc = Pigeon.clone(doc)\n\nClone a document.\n\n#### aliasDoc = Pigeon.alias(doc)\n\nMake an alias to an existing doc; analogous to a hardlink.\n\n#### changes = Pigeon.getChanges(left, right)\n\nGet the set of changes that would transform `left` into `right`.\n\n#### newDoc = Pigeon.rewindChanges(doc, ts, cid)\n\nRoll back the document state back to the given timestamp.\n\n#### newDoc = Pigeon.fastForwardChanges(doc)\n\nRoll forward the document state up to the head.\n\n#### newDoc = Pigeon.applyChanges(doc, changes)\n\nClone the given document to a new document and apply changes to the new document.\n\n#### Pigeon.applyChangesInPlace(doc, changes)\n\nApply given changes to the document in-place.\n\n#### newDoc = Pigeon.change(doc, fn)\n\nChange the document according to the given function, which receivs the document as a parameter.\n\n```javascript\ndoc = Pigeon.from({ message: 'hello' })\nnewDoc = Pigeon.change(doc, d =\u003e d.message = 'hey there')\nchanges = Pigeon.getChanges(doc, newDoc)\n```\n\n#### changes = Pigeon.getHistory(doc)\n\nGet all of the changes to recreate the document from scratch.\n\n#### newDoc = Pigeon.load(str)\n\nLoad the document from its serialized form.\n\n#### str = Pigeon.save(doc)\n\nSerialize the document to be loaded later.\n\n#### Pigeon.configuire(options)\n\nSet configuration options.\n\n```javascript\nPigeon.configure({\n  strict: true,\n  getObjectId: x =\u003e x.id || x._id || x.uuid || x.slug,\n  getTimestamp: Date.now,\n})\n```\n\n##### `strict`\n\nIn order to preserve semantic integrity, any objects which are items in arrays should contain identifier properties named `id`, `_id`, `uuid`, or `slug`, as in the example above.  When objects have identifier properties, change sets will be keyed by those identifiers, and all will be well.  When `strict` is truthy, an error will be thrown if we try to compare objects with no identifier properties.  When `strict` is falsy however, changes will be keyed by array indexes as a best effort only, and so property changes may or may not be robust to changes in array item order.  Defaults to `true`.\n\n##### `getObjectId`\n\nCallback to return an identifier value, given an object.  By default object identifiers will be sought as shown above, but if your data uses different properties for unique identifiers, you may supply an alternate function for retrieving them.\n\n##### `getTimestamp`\n\nCallback to return a unix timestamp.  This defaults to `Date.now`, which may be good enough in many cases.  If the client's clock is set by `ntpd` for example, all will be probably well enough.  However, for more precise ordering of operations, you may wish to provide your own function which would periodically sync the server time to the client and take network latency into account to provide a more accurate timestamp.\n\n\n## More on timestamps\n\nRegardless of the accuracy of clients' clocks, clients will always end up with the same state as each other, given the same documents and changes to be applied, even if the changes arrive out of order.\n\nEach time a client changes a document, internally, the change gets decorated with a client timestamp. When we merge documents, or apply change sets, the document is rewound to just before the earliest change to be applied, and changes are played forward in order.\n\nSo, for example, if one client's clock is a few seconds slower than another, if both clients change a value at about the same time, when the changes get merged, the first client's timestamp will be later, and so both clients will apply the first client's change last, and both clients will end up with exactly the same state.\n\n\n\n## Operating directly on JSON objects\n\nPigeon also exposes methods to diff and patch JSON objects:\n\n```javascript\nconst { diff, patch } = require('@frameable/pigeon')\n\nconst a1 = [\n  { id: 3920, name: 'Chicago', population: 5239412 },\n  { id: 3977, name: 'Boston', population: 1032943 },\n]\n\nconst a2 = [\n  { id: 3920, name: 'Chicago', population: 5239412 },\n  { id: 3977, name: 'Boston', population: 1032997 },\n]\n\nconst [ changes ] = diff(a1, a2);\n\nassert.deepEqual(\n  changes,\n  { op: 'replace', path: '/[3977]/population', value: 1032997, _prev: 1032943 },\n)\n\npatch(a1, changes)\nassert.deepEqual(a1, a2)\n```\n\n### changes = Pigeon.diff(left, right)\n\nCompares data structures and returns changes required to make `left`'s content equal to `right`'s.  The format of the returned changes is based on [RFC 6902](https://tools.ietf.org/html/rfc6902), with the modification that path components which are array indexes, if they refer to an object, may take the form `[\u003cid\u003e]` where `\u003cid\u003e` is the value of a property meant to uniquely identify that object, with a property named `id`, `_id`, `uuid`, or `slug`.\n\n### left = Pigeon.patch(left, changes)\n\nApplies changes to the given data structure, making modifications in-place.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fframeable%2Fpigeon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fframeable%2Fpigeon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fframeable%2Fpigeon/lists"}