{"id":28740078,"url":"https://github.com/meteor-community-packages/denormalize","last_synced_at":"2025-06-16T06:41:21.118Z","repository":{"id":45493462,"uuid":"101653072","full_name":"Meteor-Community-Packages/denormalize","owner":"Meteor-Community-Packages","description":"Simple denormalization for Meteor","archived":false,"fork":false,"pushed_at":"2025-04-01T11:02:21.000Z","size":121,"stargazers_count":20,"open_issues_count":8,"forks_count":15,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-01T12:21:57.904Z","etag":null,"topics":["caching","denormalization","grapher","meteor","mongodb"],"latest_commit_sha":null,"homepage":"https://github.com/Herteby/denormalize","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Meteor-Community-Packages.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["storytellercz","jankapunkt"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2017-08-28T14:43:02.000Z","updated_at":"2025-04-01T11:02:13.000Z","dependencies_parsed_at":"2025-03-19T00:31:22.138Z","dependency_job_id":null,"html_url":"https://github.com/Meteor-Community-Packages/denormalize","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/Meteor-Community-Packages/denormalize","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Meteor-Community-Packages%2Fdenormalize","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Meteor-Community-Packages%2Fdenormalize/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Meteor-Community-Packages%2Fdenormalize/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Meteor-Community-Packages%2Fdenormalize/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Meteor-Community-Packages","download_url":"https://codeload.github.com/Meteor-Community-Packages/denormalize/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Meteor-Community-Packages%2Fdenormalize/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260115282,"owners_count":22961027,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["caching","denormalization","grapher","meteor","mongodb"],"created_at":"2025-06-16T06:41:18.337Z","updated_at":"2025-06-16T06:41:21.100Z","avatar_url":"https://github.com/Meteor-Community-Packages.png","language":"JavaScript","funding_links":["https://github.com/sponsors/storytellercz","https://github.com/sponsors/jankapunkt"],"categories":[],"sub_categories":[],"readme":"# Denormalize\n\nSimple denormalization for Meteor\n\n- [Introduction](#introduction)\n- [Collection.cache](#collectioncacheoptions)\n- [Collection.cacheCount](#collectioncachecountoptions)\n- [Collection.cacheField](#collectioncachefieldoptions)\n- [Migration](#migration)\n- [Nested referenceFields](#nested-referencefields)\n- [Recursive caching](#recursive-caching)\n- [When are the caches updated?](#when-are-the-caches-updated)\n- [Testing the package](#testing-the-package)\n\n## Introduction\n\n```\nmeteor add herteby:denormalize\n```\n\nIn this readme, *parent* always refers to the documents in which the cache is stored, while *child* refers to the documents that will be cached.\n\n**Example:** You have two collections - Users and Roles. The Users store the _id of any Roles they have been assigned. If you want each User to cache information from any Roles that are assigned to it, the Users would be the *parents* and the Roles would be the *children*, and it would be either a *one* or *many* relationship, depending on if a User can have multiple Roles. If you wanted each Role to store a list of all Users which have that role, the Roles would be the *parents* and the Users would be the *children*, and it would be an *inverse* or *many-inverse* relationship.\n## Collection.cache(options)\n\n```javascript\nPosts.cache({\n  type:'one',\n  collection:Meteor.users,\n  fields:['username', 'profile.firstName', 'profile.lastName'],\n  referenceField:'author_id',\n  cacheField:'author'\n})\n```\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003cth\u003eProperty\u003c/th\u003e\n    \u003cth\u003eValid values\u003c/th\u003e\n    \u003cth\u003eDescription\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003etype\u003c/td\u003e\n    \u003ctd\u003e'one', 'many', 'inverse' or 'many-inverse'\u003c/td\u003e\n    \u003ctd\u003e\n      \u003cdiv\u003e\u003cb\u003eone:\u003c/b\u003e The parent stores a single child _id\u003c/div\u003e\n      \u003cdiv\u003e\u003cb\u003emany:\u003c/b\u003e The parent stores an array of child _ids\u003c/div\u003e\n      \u003cdiv\u003e\u003cb\u003einverse:\u003c/b\u003e Each child stores a single parent _id\u003c/div\u003e\n      \u003cdiv\u003e\u003cb\u003emany-inverse:\u003c/b\u003e Each child stores an array of parent _ids\u003c/div\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ecollection\u003c/td\u003e\n    \u003ctd\u003eMongo.Collection\u003c/td\u003e\n    \u003ctd\u003eThe \"child collection\", from which docs will be cached\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003efields\u003c/td\u003e\n    \u003ctd\u003eArray of Strings or Object\u003c/td\u003e\n    \u003ctd\u003eThe fields to include in the cache. It can either look like \u003ccode\u003e['username', 'profile.email']\u003c/code\u003e or \u003ccode\u003e{username:1, profile:{email:1}}\u003c/code\u003e. For \"many\", \"inverse\" and \"many-inverse\", _id will always be included.\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ereferenceField\u003c/td\u003e\n    \u003ctd\u003eString\u003c/td\u003e\n    \u003ctd\u003eFor \"one\" and \"many\", the field on the parent containing _id of children. For \"inverse\" and \"many-inverse\", the field on the children containing the _id of the parent.\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ecacheField\u003c/td\u003e\n    \u003ctd\u003eString\u003c/td\u003e\n    \u003ctd\u003eThe field on the parent where children are cached. Can be a nested field, like \u003ccode\u003e'caches.field'\u003c/code\u003e, but it can not be in the same top level field as the referenceField. For \u003ccode\u003etype:'one'\u003c/code\u003e, cacheField will store a single child. For all others, it will store an array of children.\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ebypassSchema\u003c/td\u003e\n    \u003ctd\u003eBoolean (optional)\u003c/td\u003e\n    \u003ctd\u003eIf set to true, it will bypass any \u003ca href=\"https://github.com/aldeed/meteor-collection2\"\u003ecollection2\u003c/a\u003e schema that may exist. Otherwise you must add the cacheField to your schema.\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n#### Notes and clarification:\n- \"one\" and \"inverse\" are *many-to-one* relationships (with \"one\", a parent can only have one child, but many parents could have the same child). \"many\" and \"many-inverse\" are *many-to-many* relationships\n- When `cacheField` is an array (all types except \"one\"), the order of the children is not guaranteed.\n- When `referenceField` is an array, if it contains duplicate _ids, they will be ignored. The `cacheField` will always contain unique children.\n\n## Collection.cacheCount(options)\n\n```javascript\nTodoLists.cacheCount({\n  collection:Todos,\n  referenceField:'list_id',\n  cacheField:'counts.important',\n  selector:{done:null, priority:{$lt:3}}\n})\n```\n\ncacheCount() can be used on \"inverse\" and \"many-inverse\" relationships\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003cth\u003eProperty\u003c/th\u003e\n    \u003cth\u003eValid values\u003c/th\u003e\n    \u003cth\u003eDescription\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ecollection\u003c/td\u003e\n    \u003ctd\u003eMongo.Collection\u003c/td\u003e\n    \u003ctd\u003eThe collection in which docs will be counted\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ereferenceField\u003c/td\u003e\n    \u003ctd\u003eString\u003c/td\u003e\n    \u003ctd\u003eThe field on counted docs which must match the parent _id\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ecacheField\u003c/td\u003e\n    \u003ctd\u003eString\u003c/td\u003e\n    \u003ctd\u003eThe field where the count is stored. Can be a nested field like \u003ccode\u003e'counts.all'\u003c/code\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eselector\u003c/td\u003e\n    \u003ctd\u003eMongo selector (optional)\u003c/td\u003e\n    \u003ctd\u003eCan be used to filter the counted documents. \u003ccode\u003e[referenceField]:parent._id\u003c/code\u003e will always be included though.\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ebypassSchema\u003c/td\u003e\n    \u003ctd\u003eBoolean (optional)\u003c/td\u003e\n    \u003ctd\u003eIf set to true, it will bypass any \u003ca href=\"https://github.com/aldeed/meteor-collection2\"\u003ecollection2\u003c/a\u003e schema that may exist. Otherwise you must add the cacheField to your schema.\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## Collection.cacheField(options)\n```javascript\nMeteor.users.cacheField({\n  fields:['profile.firstName', 'profile.lastName'],\n  cacheField:'fullname',\n  transform(doc){\n    return doc.profile.firstName + ' ' + doc.profile.lastName\n  }\n})\n\n```\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003cth\u003eProperty\u003c/th\u003e\n    \u003cth\u003eValid values\u003c/th\u003e\n    \u003cth\u003eDescription\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003efields\u003c/td\u003e\n    \u003ctd\u003eArray of Strings or Object\u003c/td\u003e\n    \u003ctd\u003eThe fields to watch for changes. It can either look like \u003ccode\u003e['username', 'profile.email']\u003c/code\u003e or \u003ccode\u003e{username:1, profile:{email:1}}\u003c/code\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ecacheField\u003c/td\u003e\n    \u003ctd\u003eString\u003c/td\u003e\n    \u003ctd\u003eWhere the result is stored. Can be nested like \u003ccode\u003e'computed.fullName'\u003c/code\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003etransform\u003c/td\u003e\n    \u003ctd\u003eFunction (optional)\u003c/td\u003e\n    \u003ctd\u003eThe function used to compute the result. If not defined, the default is to return a string of all watched fields concatenated with \u003ccode\u003e', '\u003c/code\u003e\u003cbr\u003eThe document provided to the function only contains the fields specified in \u003ccode\u003efields\u003c/code\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ebypassSchema\u003c/td\u003e\n    \u003ctd\u003eBoolean (optional)\u003c/td\u003e\n    \u003ctd\u003eIf set to true, it will bypass any \u003ca href=\"https://github.com/aldeed/meteor-collection2\"\u003ecollection2\u003c/a\u003e schema that may exist. Otherwise you must add the cacheField to your schema.\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n**Note:** The transform function could also fetch data from other collections or through HTTP if you wanted, as long as it's done synchronously.\n\n## Migration\n\nIf you decide to add a new cache or change the cache options on a collection that already contains documents, those documents need to be updated. There are two options for this:\n\n### migrate(collectionName, cacheField, [selector])\n```javascript\nimport {migrate} from 'meteor/herteby:denormalize'\nmigrate('users', 'fullName')\nmigrate('users', 'fullAddress', {fullAddress:{$exists:false}})\n```\nThis updates the specified cacheField for all documents in the collection, or all documents matching the selector. Selector can also be an _id.\n\n### autoMigrate()\n```javascript\nimport {autoMigrate} from 'meteor/herteby:denormalize'\nautoMigrate() //should be called last in your server code, after all caches have been declared\n```\nWhen `autoMigrate()` is called, it checks all the caches you have declared against a collection (called _cacheMigrations in the DB) to see wether they need to be migrated. If any do, it will run a migration on them, and then save the options to _cacheMigrations, so that it won't run again unless you change any of the options. If you later for example decide to add another field to a cache, it will rerun automatically.\n\nOne thing it does not do is remove the old cacheField, if you were to change the name or remove the cache. That part you have to do yourself.\n\nNote: it does not check the *documents*, it just checks each *cache declaration*, so it won't thrash your DB on server start going through millions of records (unless something needs to be updated).\n\n## Nested referenceFields\nFor \"one\" and \"inverse\", nested referenceFields are simply declared like `referenceField:'nested.reference.field'`\n\nFor \"many\" and \"many-inverse\", if the referenceField is an Array containing objects, a colon is used to show where the Array starts.\n\n#### Example:\n\nIf the parent doc looks like this:\n```javascript\n{\n  //...\n  references:{\n    users:[{_id:'user1'}, {_id:'user2'}]\n  }\n}\n```\nThe referenceField string should be `'references.users:_id'`\n\n## Recursive caching\n\nYou can use the output (the `cacheField`) of one cache function as one of the fields to be cached by another cache function, or even as the referenceField. They will all be updated correctly. This way you can create \"chains\" connecting three or more collections.\n\nIn the examples below, all cache fields start with `_`, which may be a good convention to follow for all your caches.\n\n#### Use cacheField() to cache the sum of all cached items from a purchase\n```javascript\nBills.cacheField({\n  fields:['_items'],\n  cacheField:'_sum',\n  transform(doc){\n    return _.sum(_.map(doc._items, 'price'))\n  }\n})\n```\n#### Caching the cacheFields of another cache\n```javascript\nBills.cache({\n  cacheField:'_items',\n  collection:Items,\n  type:'many',\n  referenceField:'item_ids',\n  fields:['name', 'price']\n})\nCustomers.cache({\n  cacheField:'_bills',\n  collection:Bills,\n  type:'inverse',\n  referenceField:'customer_id',\n  fields:['_sum', '_items']\n})\n```\n#### Using the cacheField of another cache as referenceField\n```javascript\nCustomers.cache({\n  cacheField:'_bills2',\n  collection:Bills,\n  type:'inverse',\n  referenceField:'customer_id',\n  fields:['item_ids', '_sum']\n})\nCustomers.cache({\n  cacheField:'_items',\n  collection:Items,\n  type:'many',\n  referenceField:'_bills2:item_ids',\n  fields:['name', 'price']\n})\n```\n\n#### Incestuous relationships\n\nWith this fun title I'm simply referring to caches where the *parent* and *child* collections are the same.\n\n```javascript\nMeteor.users.cache({\n  cacheField:'_friends',\n  collection:Meteor.users,\n  type:'many',\n  referenceField:'friend_ids',\n  fields:['name', 'profile.avatar']\n})\n```\nThis works fine, but there is one thing you can not do - *cache the cacheField of a document in the same collection* - in this example it would be caching the friends of a users friends. This would lead to an infinite loop and infinitely growing caches.\n\n\n## When are the caches updated?\n\nThe caches for `cache()` and `cacheCount()` are updated immediately and synchronously.\n\n```javascript\nPosts.cache({\n  cacheField:'_author',\n  //...\n})\nPosts.insert({_id:'post1', author_id:'user1'})\nPosts.findOne('post1')._author //will contain the cached user\n```\n`cache()` uses 5 hooks: parent.after.insert, parent.after.update, child.after.insert, child.after.update and child.after.remove. There are then checks done to make sure it doesn't do unnecessary updates.\n\nBasically you should always be able to rely on the caches being updated. If they're not, that should be considered a bug.\n\n*However*, to avoid a complicated issue with \"recursive caching\", the update of `cacheField()` is always deferred.\n\n```javascript\nMeteor.users.cacheField({\n  fields:['address', 'postalCode', 'city'],\n  cacheField:'_fullAddress',\n})\nMeteor.users.insert({_id:'user1', ...})\nMeteor.users.findOne('user1')._fullAddress //will not contain the cached address yet\nMeteor.setTimeout(()=\u003e{\n  Meteor.users.findOne('user1')._fullAddress //now it should be there\n}, 50)\n```\n\n**Note:** Since this package relies on collection-hooks, it won't detect any updates you do to the DB outside of Meteor. To solve that, you can call the `migrate()` function afterwards.\n\n## Testing the package\n\n```\ncd packages/denormalize\nnpm run test\n```\nThe tests will be run in the console\u003cbr\u003e\nThe package currently has over 120 tests\u003cbr\u003e\nNote: The \"slowness warnings\" in the results are just due to the asynchronous tests","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmeteor-community-packages%2Fdenormalize","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmeteor-community-packages%2Fdenormalize","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmeteor-community-packages%2Fdenormalize/lists"}