{"id":15013343,"url":"https://github.com/peerlibrary/meteor-peerdb","last_synced_at":"2025-04-12T04:41:42.918Z","repository":{"id":11165591,"uuid":"13539049","full_name":"peerlibrary/meteor-peerdb","owner":"peerlibrary","description":"Reactive database layer with references, generators, triggers, migrations, etc.","archived":false,"fork":false,"pushed_at":"2020-02-13T16:54:11.000Z","size":535,"stargazers_count":129,"open_issues_count":15,"forks_count":15,"subscribers_count":33,"default_branch":"master","last_synced_at":"2025-03-26T00:11:34.942Z","etag":null,"topics":["database","denormalization","meteor","meteor-collection","meteor-package","mongodb","reactive-documents","reactivity"],"latest_commit_sha":null,"homepage":"http://atmospherejs.com/peerlibrary/peerdb","language":"CoffeeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/peerlibrary.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-10-13T12:44:28.000Z","updated_at":"2025-03-04T10:51:58.000Z","dependencies_parsed_at":"2022-09-01T20:41:02.518Z","dependency_job_id":null,"html_url":"https://github.com/peerlibrary/meteor-peerdb","commit_stats":null,"previous_names":[],"tags_count":51,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peerlibrary%2Fmeteor-peerdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peerlibrary%2Fmeteor-peerdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peerlibrary%2Fmeteor-peerdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peerlibrary%2Fmeteor-peerdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/peerlibrary","download_url":"https://codeload.github.com/peerlibrary/meteor-peerdb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248519304,"owners_count":21117756,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","denormalization","meteor","meteor-collection","meteor-package","mongodb","reactive-documents","reactivity"],"created_at":"2024-09-24T19:44:07.709Z","updated_at":"2025-04-12T04:41:42.880Z","avatar_url":"https://github.com/peerlibrary.png","language":"CoffeeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"PeerDB\n======\n\nMeteor smart package which provides a reactive database layer with references, generators, triggers, migrations, etc.\nMeteor provides a great way to code in a reactive style and this package brings reactivity to your database as well.\nYou can now define inside your application along with the rest of your program logic also how your data should be updated\non any change and how various aspects of your data should be kept in sync and consistent no matter where the change comes\nfrom.\n\nImplemented features are:\n * reactive references between documents\n * reactive reverse references between documents\n * reactive auto-generated fields from other fields\n * reactive triggers\n * [migrations](https://github.com/peerlibrary/meteor-peerdb-migrations)\n\nPlanned features are:\n * versioning of all changes to documents\n * integration with [full-text search](http://www.elasticsearch.org/)\n * [strict-typed schema validation](https://github.com/balderdashy/anchor)\n\nAdding this package to your [Meteor](http://www.meteor.com/) application adds the `Document` object into the global scope.\n\nBoth client and server side.\n\nInstallation\n------------\n\n```\nmeteor add peerlibrary:peerdb\n```\n\nAdditional packages\n-------------------\n\n* [peerlibrary:peerdb-migrations](https://github.com/peerlibrary/meteor-peerdb-migrations) – Migrations support for PeerDB documents\n\nDocuments\n---------\n\nInstead of Meteor collections with PeerDB you are defining PeerDB documents by extending `Document`. Internally it\ndefines a Meteor collection, but also all returned documents are then an instance of that PeerDB documents class.\n\nMinimal definition:\n\n```coffee\nclass Person extends Document\n  @Meta\n    name: 'Person'\n```\n\nThis would create in your database a MongoDB collection called `Persons`. `name` must match the class name. `@Meta` is\nused for PeerDB and in addition you can define arbitrary class or object methods for your document which will then be\navailable on documents returned from the database:\n\n```coffee\nclass Person extends Document\n  # Other fields:\n  #   username\n  #   displayName\n  #   email\n  #   homepage\n\n  @Meta\n    name: 'Person'\n\n  # Class methods\n  @verboseName: -\u003e\n    @Meta._name.toLowerCase()\n\n  @verboseNamePlural: -\u003e\n    \"#{ @verboseName() }s\"\n\n  # Instance method\n  getDisplayName: -\u003e\n    @displayName or @username\n```\n\nYou can also wrap existing Meteor collections:\n\n```coffee\nclass User extends Document\n  @Meta\n    name: 'User'\n    collection: Meteor.users\n```\n\nAnd if you need to access the internal or wrapped collection you can do that by:\n\n```coffee\nPerson.Meta.collection._ensureIndex\n  username: 1\n```\n\nQuerying\n--------\n\nPeerDB provides an alternative to Meteor collections query methods. You should be using them to access documents. You\ncan access them through the `documents` property of your document class. For example:\n\n```coffee\nPerson.documents.find({}).forEach (person, i, cursor) =\u003e\n  console.log person.constructor.verboseName(), person.getDisplayName()\n\nPerson.documents.findOne().getDisplayName()\n\nPerson.documents.findOne().email\n```\n\nThe functions and arguments available are the same as those available for Meteor collections, with the addition of:\n\n* `.documents.exists(query, options)` – efficient check if any document matches given `query`\n* `.documents.bulkInsert(arrayOfDocuments, [options], callback)` – insert multiple documents in bulk, returning the list\nof IDs and calling an optional callback\n\n`bulkInsert` has a special handling of references to minimize issues of loading documents referencing documents which\nare yet to be inserted. By default, first, all documents are inserted with all optional references delayed. This means,\nall optional references are first omitted, and then all documents are updated by the second query, setting values for\nall optional references. Reference fields inside arrays are always delayed. Optional `options` object accepts field:\n* `dontDelay` – a list of paths of optional reference fields which should not be delayed\n\nIn a similar way we extend the cursor returned from `.documents.find(...)` with an `exists` method which operates\nsimilar to the `count` method, only that it is more efficient:\n\n```coffee\nPerson.documents.exists({})\nPerson.documents.find({}).exists()\n```\n\n`Person.Meta` gives you back document metadata and `Person.documents` give you access to all documents.\n\nAll this is just an easy way to define documents and collections in a unified fashion, but it becomes interesting\nwhen you start defining relations between documents.\n\nReferences\n----------\n\nIn the traditional SQL world of relational databases you do joins between related documents every time you read them from\nthe database. This makes reading slower, your database management system is redoing the same computation of joins\nfor every read, and also horizontal scaling of a database to many instances is harder because every read might potentially\nhave to talk to other instances.\n\nNoSQL databases like MongoDB remove relations between documents and leave it to users to resolve relations on their own.\nThis often means fetching one document, observing which other documents it references, and fetching those as well.\nBecause each of those documents are stand-alone and static, it is relatively easy and quick for a database management\nsystem like MongoDB to find and return them. Such an approach is quick and it scales easily, but the\ndownside is the multiple round trips you have to do in your code to get all documents you are interested in. Those\nround trips become even worse when those queries are coming over the Internet from Meteor client code,\nbecause Internet latency is much higher.\n\nFor a general case you can move this fetching of related documents to the server side into Meteor publish functions by\nusing libraries like [meteor-related](https://github.com/peerlibrary/meteor-related). It provides an easy way to fetch\nrelated documents reactively, so when dependencies change, your published documents will be updated accordingly. While\nlatency to your database instances is hopefully better on your server, we did not really improve much from the SQL\nworld: you are effectively recomputing joins and now even in a much less efficient way, especially if you are reading\nmultiple documents at the same time.\n\nLuckily, in many cases we can observe that we are mostly interested only in few fields of a related document, again\nand again. Instead of recomputing joins every time we read, we could use MongoDB's sub-documents feature to embed\nthose fields along with the reference. Instead of just storing the `_id` of a related document, we could store also\nthose few often used fields. For example, if you are displaying blog posts, you want to display the author's name together\nwith the blog post. You won't really need only the blog post without the author name. An example blog post document\ncould then look like:\n\n```json\n{\n  \"_id\": \"frqejWeGWjDTPMj7P\",\n  \"body\": \"A simple blog post\",\n  \"author\": {\n    \"_id\": \"yeK7R5Lws6MSeRQad\",\n    \"username\": \"wesley\",\n    \"displayName\": \"Wesley Crusher\"\n  },\n  \"subscribers\": [\n    {\n      \"_id\": \"k7cgWtxQpPQ3gLgxa\"\n    },\n    {\n      \"_id\": \"KMYNwr7TsZvEboXCw\"\n    },\n    {\n      \"_id\": \"tMgj8mF2zF3gjCftS\"\n    }\n  ],\n  \"reviewers\": [\n    {\n      \"_id\": \"tMgj8mF2zF3gjCftS\",\n      \"username\": \"deanna\",\n      \"displayName\": \"Deanna Troi\"\n    }\n  ]\n}\n```\n\nGreat! Now we have to fetch only this one document and we have everything needed to display a blog post. It is easy\nfor us to publish it with Meteor and use it as any other document, with direct access to author's fields.\n\nNow, storing the author's name along with every blog post document brings an issue. What if user changes their\nname? Then you have to update all those fields in documents referencing the user. So you would have to make sure that\nanywhere in your code where you are changing the name, you are also updating fields in references. What about changes\nto the database coming from outside of your code? Here is when PeerDB comes into action. With PeerDB you define those\nreferences once and then PeerDB makes sure they stay in sync. It does not matter where the changes come from, it will\ndetect them and update fields in referenced sub-documents accordingly.\n\nIf we have two documents:\n\n```coffee\nclass Person extends Document\n  # Other fields:\n  #   username\n  #   displayName\n  #   email\n  #   homepage\n\n  @Meta\n    name: 'Person'\n\nclass Post extends Document\n  # Other fields:\n  #   body\n\n  @Meta\n    name: 'Post'\n    fields: -\u003e\n      # We can reference other document\n      author: @ReferenceField Person, ['username', 'displayName']\n      # Or an array of documents\n      subscribers: [@ReferenceField Person]\n      reviewers: [@ReferenceField Person, ['username', 'displayName']]\n```\n\nWe are using `@Meta`'s `fields` argument to define references.\n\nIn the above definition, the `author` field will be a subdocument containing `_id` (always added) and the `username`\nand `displayName` fields. If the `displayName` field in the referenced `Person` document is changed, the `author` field\nin all related `Post` documents will be automatically updated with the new value for the `displayName` field.\n\n```coffee\nPerson.documents.update 'tMgj8mF2zF3gjCftS',\n  $set:\n    displayName: 'Deanna Troi-Riker'\n\n# Returns \"Deanna Troi-Riker\"\nPost.documents.findOne('frqejWeGWjDTPMj7P').reviewers[0].displayName\n\n# Returns \"Deanna Troi-Riker\", sub-documents are objectified into document instances as well\nPost.documents.findOne('frqejWeGWjDTPMj7P').reviewers[0].getDisplayName()\n```\n\nThe `subscribers` field is an array of references to `Person` documents, where every element in the array will\nbe a subdocument containing only the `_id` field.\n\nCircular references are possible as well:\n\n```coffee\nclass CircularFirst extends Document\n  # Other fields:\n  #   content\n\n  @Meta\n    name: 'CircularFirst'\n    fields: -\u003e\n      # We can reference circular documents\n      second: @ReferenceField CircularSecond, ['content']\n\nclass CircularSecond extends Document\n  # Other fields:\n  #   content\n\n  @Meta\n    name: 'CircularSecond'\n    fields: -\u003e\n      # But of course one should not be required so that we can insert without warnings\n      first: @ReferenceField CircularFirst, ['content'], false\n```\n\nIf you want to reference the same document recursively, use the string `'self'` as an argument to `@ReferenceField`.\n\n```coffee\nclass Recursive extends Document\n  # Other fields:\n  #   content\n\n  @Meta\n    name: 'Recursive'\n    fields: -\u003e\n      other: @ReferenceField 'self', ['content'], false\n```\n\nAll those references between documents can be tricky as you might want to reference documents defined afterwards\nand JavaScript symbols might not even exist yet in the scope, and PeerDB works hard to still allow you to do that.\nBut to make sure all symbols are correctly resolved you should call `Document.defineAll()` after all your definitions.\nThe best is to put it in the filename which is loaded last.\n\nOne more example to show use of nested objects:\n\n```coffee\nclass ACLDocument extends Document\n  @Meta\n    name: 'ACLDocument'\n    fields: -\u003e\n      permissions:\n        admins: [@ReferenceField User]\n        editors: [@ReferenceField User]\n```\n\nYou can also do:\n\n```coffee\nclass ACLDocument extends Document\n  # Each permission object inside \"permissions\" could have also\n  # timestamp and permission type fields.\n\n  @Meta\n    name: 'ACLDocument'\n    fields: -\u003e\n      permissions: [\n        user: @ReferenceField User\n        grantor: @ReferenceField User, [], false\n      ]\n```\n\n`ReferenceField` accepts the following arguments:\n\n* `targetDocument` – target document class, or `'self'`\n* `fields` – list of fields to sync in a reference's sub-document; instead of a field name you can use a MongoDB projection as well, like `emails: {$slice: 1}`\n* `required` – should the reference be required (default) or not. If required, when the referenced document is removed, this document will be removed as well. If not required, the reference will be set to `null`.\n* `reverseName` – name of a field for a reverse reference; specify to enable a reverse reference\n* `reverseFields` – list of fields to sync for a reference reference\n\nWhat are reverse references?\n\nReverse references\n------------------\n\nSometimes you want also to have easy access to information about all the documents referencing a given document.\nFor example, for each author you might want to have a list of all blog posts they wrote, as part of their document.\n\n```coffee\nclass Post extends Post\n  @Meta\n    name: 'Post'\n    replaceParent: true\n    fields: (fields) -\u003e\n      fields.author = @ReferenceField Person, ['username', 'displayName'], true, 'posts'\n      fields\n```\n\nWe [redefine](#abstract-documents-and-replaceparent) the `Post` document and replace it with a new definition which enables\nreverse references for the `author` field. Now `Person.documents.findOne('yeK7R5Lws6MSeRQad')` returns:\n\n```json\n{\n  \"_id\": \"yeK7R5Lws6MSeRQad\",\n  \"username\": \"wesley\",\n  \"displayName\": \"Wesley Crusher\",\n  \"email\": \"wesley@enterprise.starfleet\",\n  \"homepage\": \"https://gww.enterprise.starfleet/~wesley/\",\n  \"posts\": [\n    {\n      \"_id\": \"frqejWeGWjDTPMj7P\"\n    }\n  ]\n}\n```\n\nAuto-generated fields\n---------------------\n\nSometimes you need fields in a document which are based on other fields. PeerDB allows you an easy way to define\nsuch auto-generated fields:\n\n```coffee\nclass Post extends Post\n  # Other fields:\n  #   title\n\n  @Meta\n    name: 'Post'\n    replaceParent: true\n    generators: (generators) -\u003e\n      generators.slug = @GeneratedField 'self', ['title'], (fields) -\u003e\n        unless fields.title\n          [fields._id, undefined]\n        else\n          [fields._id, \"prefix-#{ fields.title.toLowerCase() }-suffix\"]\n      generators\n```\n\nThe last argument of `GeneratedField` is a function which receives an object populated with values based on the list of\nfields you are interested in. In the example above, this is one field named `title` from the `Posts` collection. The `_id`\nfield is always available in `fields`. Generator function receives just `_id` when document containing fields is being\nremoved. Otherwise it receives all fields requested. Generator function should return two values, a selector (often just the ID of a document)\nand a new value. If the value is undefined, the auto-generated field is removed. If the selector is undefined, nothing is done.\n\nYou can define auto-generated fields across documents. Furthermore, you can combine reactivity. Maybe you want to also\nhave a count of all posts made by a person?\n\n```coffee\nclass Person extends Person\n  @Meta\n    name: 'Person'\n    replaceParent: true\n    generators: (generators) -\u003e\n      generators.postsCount = @GeneratedField 'self', ['posts'], (fields) -\u003e\n        [fields._id, fields.posts?.length or 0]\n      generators\n```\n\nTriggers\n--------\n\nYou can define triggers which are run every time any of the specified fields changes:\n\n```coffee\nclass Post extends Post\n  # Other fields:\n  #   updatedAt\n\n  @Meta\n    name: 'Post'\n    replaceParent: true\n    triggers: -\u003e\n      updateUpdatedAt: @Trigger ['title', 'body'], (newDocument, oldDocument) -\u003e\n        # Don't do anything when document is removed\n        return unless newDocument?._id\n\n        timestamp = new Date()\n        Post.documents.update\n          _id: newDocument._id\n          updatedAt:\n            $lt: timestamp\n        ,\n          $set:\n            updatedAt: timestamp\n```\n\nThe return value is ignored. `newDocument` and `oldDocument` can be `null` when a document has been\nremoved or added, respectively. Triggers are useful when you want arbitrary code to be run when\nfields change. This could be implemented directly with [observe](http://docs.meteor.com/#observe),\nbut triggers simplify that and provide an alternative API in the PeerDB spirit.\n\nWhy we are using a trigger here and not an auto-generated field? The main reason is that we want to ensure\n`updatedAt` really just increases, so a more complicated update query is needed. Additionally, reference\nfields and auto-generated fields should be without side-effects and should be allowed to be called at any\ntime. This is to ensure that we can re-sync any broken references as needed. If you would use an\nauto-generated field, it could be called again at a later time, updating `updatedAt` to a later time\nwithout any content of a document really changing.\n\nPeerDB does not really re-sync any broken references (made while your Meteor application was not running)\nautomatically. If you believe such references exist (eg., after a hard crash of your application), you\ncan trigger re-syncing by calling `Document.updateAll()`. All references will be resynced and all\nauto-generated fields rerun. But not triggers. It is a quite heavy operation.\n\nAbstract documents and `replaceParent`\n--------------------------------------\n\nYou can define abstract documents by setting the `abstract` `Meta` flag to `true`. Such documents will not create\na MongoDB collection. They are useful to define common fields and methods you want to reuse in multiple\ndocuments.\n\nWe skimmed over `replaceParent` before. You should set it to `true` when you are defining a document with the\nsame name as a document you are extending (parent). It is a kind of a sanity check that you know what you are\ndoing and that you are promising you are not holding a reference to the extended (and replaced) document somewhere\nand you expect it to work when using it. How useful `replaceParent` really is, is a good question, but it\nallows you to define a common (client and server side) document and then augment it on the server side with\nserver-specific code.\n\nInitialization\n--------------\n\nIf you would like to run some code after Meteor startup, but before observers are enabled, you can use `Document.prepare`\nto register a callback. If you would like to run some code after Meteor startup and after observers are enabled, you can\nuse `Document.startup` to register a callback.\n\nSettings\n--------\n\n### `PEERDB_INSTANCES=1` ###\n\nAs your application grows you might want to run specialized Meteor instances just to do PeerDB reactive MongoDB\nqueries. To distribute PeerDB load, configure the number of PeerDB instances using the `PEERDB_INSTANCES` environment variable.\nSuggested setting is that your web-facing instances disable PeerDB by setting `PEERDB_INSTANCES` to 0, and then you have\ndedicated PeerDB instances.\n\n### `PEERDB_INSTANCE=0` ###\n\nIf you are running multiple PeerDB instances, which instance is this? It is zero-based index so if you configured\n`PEERDB_INSTANCES=2`, you have to have two instances, one with `PEERDB_INSTANCE=0` and another with `PEERDB_INSTANCE=1`.\n\n### `MONGO_OPLOG_URL` and `MONGO_URL` ###\n\nWhen running multiple instances you want to connect them all to the same database. You have to configure both normal\nMongoDB connection and also the oplog connection. You can use your own MongoDB instance or connect to one provided by\nrunning Meteor in development mode. In the latter case the recommended way is that one web-facing instance runs\nMongoDB and all other instances connect to that MongoDB.\n\n```\nMONGO_OPLOG_URL=mongodb://127.0.0.1:3001/local\nMONGO_URL=mongodb://127.0.0.1:3001/meteor\n```\n\nExamples\n--------\n\nSee [tests](https://github.com/peerlibrary/meteor-peerdb/blob/master/tests.coffee) for many examples. See\n[document definitions in PeerLibrary](https://github.com/peerlibrary/peerlibrary/tree/development/lib/documents) for\nreal-world definitions.\n\nRelated projects\n----------------\n\n* [matb33:collection-hooks](https://github.com/matb33/meteor-collection-hooks) – provides an alternative way to\nattach additional program logic on changes to your data, but it hooks into collection API methods so if a change comes\nfrom the outside, hooks are not called; additionally, collection API methods are delayed for the time of all hooks to\nbe executed while in PeerDB hooks run in parallel in or even in a separate process (or processes), allowing your code to\nreturn quickly while PeerDB assures that data will be eventually consistent (this has a downside of course as well,\nso if you do not want that API calls return before all hooks run, `matb33:collection-hooks` might be more suitable for\nyou)\n* [peerlibrary:meteor-related](https://github.com/peerlibrary/meteor-related) – while PeerDB provides an easy way to embed referenced\ndocuments as subdocuments, it requires that those relations are the same for all users; if you want dynamic relations\nbetween documents, `meteor-related` provides an easy way to fetch related documents reactively on the server side, so\nwhen dependencies change, your published documents will be updated accordingly\n* [herteby:denormalize](https://github.com/Herteby/denormalize) – it does similar denormalization, but uses `matb33:collection-hooks` hooks instead reactivity to maintained denormalization, moreover, it looks like `herteby:denormalize` is much more limited in features than this package, which provides, e.g., also wrapping of documents into JavaScript objects with methods, generators, and reverse fields\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeerlibrary%2Fmeteor-peerdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeerlibrary%2Fmeteor-peerdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeerlibrary%2Fmeteor-peerdb/lists"}