{"id":24902909,"url":"https://github.com/yuval-a/deriveodm","last_synced_at":"2025-10-16T14:30:52.210Z","repository":{"id":87063291,"uuid":"118806102","full_name":"yuval-a/deriveODM","owner":"yuval-a","description":"DeriveODM is a reactive ODM - Object Document Mapper - framework, a \"wrapper\" around MongoDB,  that removes all the hassle of data-persistence by handling it transparently in the background, in a DRY manner.","archived":false,"fork":false,"pushed_at":"2023-08-30T22:10:07.000Z","size":944,"stargazers_count":4,"open_issues_count":5,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-20T09:49:08.147Z","etag":null,"topics":["collection","data","data-mapper","data-science","database","db","document","dry","mapper","mongo","mongodb","mongoose","node","nodejs","object","odm","persistence","persistent","react","reactive"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yuval-a.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2018-01-24T18:37:27.000Z","updated_at":"2024-08-05T16:05:38.000Z","dependencies_parsed_at":"2024-04-13T04:40:16.948Z","dependency_job_id":null,"html_url":"https://github.com/yuval-a/deriveODM","commit_stats":null,"previous_names":["yuval-a/derivejs"],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuval-a%2FderiveODM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuval-a%2FderiveODM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuval-a%2FderiveODM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuval-a%2FderiveODM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yuval-a","download_url":"https://codeload.github.com/yuval-a/deriveODM/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236726282,"owners_count":19194889,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["collection","data","data-mapper","data-science","database","db","document","dry","mapper","mongo","mongodb","mongoose","node","nodejs","object","odm","persistence","persistent","react","reactive"],"created_at":"2025-02-01T22:17:46.017Z","updated_at":"2025-10-16T14:30:51.602Z","avatar_url":"https://github.com/yuval-a.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n[Summary](https://yuval.hashnode.dev/derivejs-a-reactive-odm-object-data-mapper-framework-for-mongodb-and-nodejs-ckfspl31f02ryv6s1asqy6wvh) | [Reference](https://github.com/yuval-a/derivejs/blob/master/reference.md) | [Comparison to Mongoose](https://github.com/yuval-a/derivejs/blob/master/mongoose-derive-migration.md) | [Demonstration](https://github.com/yuval-a/spaceships-derivejs-demo)\n\n## Introduction\n**DeriveODM** (formerly *DeriveJS*)  lets you manipulate and create Javascript data objects, while **automatically** and **transparently** persisting and updating them on a database (such as MongoDB), in the background, without any additional hassle or code.\n\nIt wraps your data classes and objects with [Javascript Proxies](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy), \"tapping-in\" to native operations such as creating instances (using the normal `new` operator), and updating property values (using the normal assignment operator `=`), and then handling passing database calls and operations to the database in the background, while leveraging MongoDB's bulk operations capabilities, orchestrating them in a smart, efficient way, to save unnecessary calls to the DB engine, and running bulk operations in fixed (settable) intervals. The background engine is mostly handled transparently by a module called `SyncManager`.\n\n**Note**: this is a complete technical reference, if you'd like to read a less verbose introduction, you can read [this article on Hashnode](https://yuval.hashnode.dev/derivejs-a-reactive-odm-object-data-mapper-framework-for-mongodb-and-nodejs-ckfspl31f02ryv6s1asqy6wvh)\n\n## Table of Contents\n  * [Introduction](#introduction)\n    + [The Rationale Behind DeriveJS, Using an Analogy](#the-rationale-behind-derivejs-using-an-analogy)\n    + [Comparison to Mongoose](#comparison-to-mongoose)\n    + [Reference](#reference)\n  * [How to Use](#how-to-use)\n    * [Define a Data Model](#define-a-data-model)\n    * [Create an Instance](#create-an-instance)\n    * [Manipulate Properties](#manipulate-properties)\n    * [Call Instance Functions](#call-instance-functions)\n    * [Derive a data model](#derive-a-data-model)\n  * [Getting Started](#getting-started)\n    + [Defining a Data Model](#defining-a-data-model)\n  * [Going deeper](#going-deeper)\n    + [Modifiers](#modifiers)\n    + [Built-in Methods: Callbacks and Hooks](#built-in-methods-callbacks-and-hooks)\n      - [New Model Instance Lifecycle](#new-model-instance-lifecycle)\n      - [Database Persistence Callbacks](#database-persistence-callbacks)\n        + [`_inserted()`](#_inserted)\n        + [\"Assignment with`$callback`\" Syntax (Update Callbacks)](#assignment-with-callback-syntax-update-callbacks)\n        + [`_isDuplicate()`](#_isDuplicate)\n        + [`_error(msg)`](#_errormsg)\n      - [Database Persistence Events (`$_dbEvents`)](#database-persistence-events-_dbevents)\n          - [`inserted` Event](#inserted-event)\n          - [`updated` Event](#updated-event)\n          - [Setting a Callback Function to a Meta Property](#setting-a-callback-function-to-a-meta-property)\n      - [`watch(on)`](#watchon)\n      - [Listening for local changes](#listening-for-local-changes)\n          - [`changed`](#changedproperty-newvalue-oldvalue)\n          - [`$Listen`](#listen)\n    + [Unique Indexes](#unique-indexes)\n    + [The `remodel` Method](#the-remodel-method)\n  * [Going Further - Extending and Deriving Models](#going-further---extending-and-deriving-models)\n  * [Retrieving Data From a Database](#retrieving-data-from-a-database)\n    + [`get`](#getwhich)\n    + [`getAll`](#getallwhich-sortby-limit0-skip0)\n    + [`getAllCursor`](#getallcursorwhich-sortby-limit0-skip0)\n    + [`getAllRead`](#getallreadwhich-sortby-limit0-skip0)\n    + [`map`](#mapwhich-index-returnarray-limit0-skip0)\n    + [`mapRead`](#mapreadwhich-index-returnarray-limit0-skip0)\n    + [`has`](#haswhich-returndocument)\n    + [`count`](#countwhich)\n    + [`MainIndex`](#mainindex)\n      - [`mainIndex() method`](#mainindex-method)\n    + [`which` Argument](#which-argument)\n        - [Selective Fields (Projection)](#selective-fields-projection)\n    + [`map` - Additional Information](#map---additional-information)\n    + [A Word of Caution for When Using the `get` Functions](#a-word-of-caution-for-when-using-the-get-functions)\n    + [`$DefaultCriteria`](#defaultcriteria)\n  * [Using Model Instances as Values in Other Models](#using-model-instances-as-values-in-other-models)\n  * [Indexes and How They Are Handled in the Mongo Server](#indexes-and-how-they-are-handled-in-the-mongo-server)\n  * [Putting it All Together:](#putting-it-all-together)\n    + [Defining Our Models](#defining-our-models)\n    + [Writing our App](#writing-our-app)\n  * [Advanced Subjects](#advanced-subjects)\n    + [Join](#join)\n      - [An example use case](#an-example-use-case)\n    + [`joinAll`](#joinall)\n    + [Access the \"raw\" MongoDB Collection Object](#access-the-raw-mongodb-collection-object)\n    + [Mixins](#mixins)\n    + [`collectionReady()` (new in version 1.4.0+)](#-collectionready-new-in-version-140)\n\n\n### The Rationale Behind DeriveJS, Using an Analogy\nIf you are familiar with a front-end UI framework such as ReactJS, you know that whenever a change is made to the `state` object - React will automatically know to issue a re-render of the component - this is known as a \"pull\" methodology, where as in other similar frameworks, you might need to explicitly call a `render()` method (this is a \"push\" methodology, in that context).\nIn a similar analogy to the way React works - when using Derive - you are **not required** to call an explicit `save()` method to have your data be saved and persisted on a DB - it's enough that you make some change to an exisiting data object, or create a new one - and Derive will already know to handle that data's persistence.\n\nTo sum-up: `DeriveJS` is a reactive ODM (Object Document Mapper), that lets you deal with data, in a DRY way, without having to take care of all the hassle of the logistics of database persistency.\n\n### Comparison to Mongoose\nIf you used or are using Mongoose and considering moving to Derive, or would like to see a comparison between the two, you can go over [this document](https://github.com/yuval-a/derivejs/blob/master/mongoose-derive-migration.md).\n\n### Reference\nFor a complete reference of all available methods, functions and objects available in Derive - [see this document](https://github.com/yuval-a/derivejs/blob/master/reference.md).\n\n## How to use\nIt only takes a few easy steps:\n\n##### Define a data model\n```javascript\nconst User = Model({\n    _email$: \"\",\n    _name: \"\",\n    role:\"editor\", // set a default role for a User object\n    age: null,\n    password: null,\n    setPassword(pass) {\n        // hash the plain-text password (\"hashit\" is just an example function for your preffered hashing function)\n        var passwordHash = hashit(pass);\n        this.password = passwordHash;\n    }\n},\"User\");\n```\nThe first time you define it, a `Users` collection is defined on the database, with an `_email` unique index and a `_name` index (you can also alter the properties, change indexes later).\n\n##### Create an instance\n```javascript\nlet user = new User (\"someemail@mail.com\",\"Someone Somebody\");\n```\nThere will now be a new document in the `Users` collection, having \"`someemail@mail.com`\" as the `_email` and \"`Someone Somebody`\" as the `_name`.\n\n##### Manipulate properties \n```javascript\nuser.age = 30;\n```\n\nThe document will now have the value `30` set to its `age` property.\n\n##### Call instance functions\n```javascript\nuser.setPassword(\"plaintextpassword\");\n```\n\n##### Derive a data model\nOne of the powerful features of Derive, is the ability to [\"extend\" data models](#going-further---extending-and-deriving-models), while having the \"sub-models\" share the same collection as their \"super-models\" - this, together with the [\"Default Criteria\" feature](#defaultcriteria) can enable meaningful \"data inheritance\".\n\n```javascript\nconst Admin = \n   User.derive({\n   \trole_: \"admin\"\n   });\n\nlet admin = new Admin(\"admin@mail.com\", \"Admin Name\");\n```\n\nWith `DeriveJS` you can create and manipulate a large amount of data objects, and know that they will be persisted in the database, efficiently and in a short time.\n\nAlthough the methodology behind the framework is mostly that of \"send and forget\" regarding data persistence - DeriveJS also exposes [callback functions](#Built-in-methods:-callbacks-and-hooks) that allows getting notified exactly when specific objects\nare actually saved on the database, or exactly when specific properties have been actually updated, for the occasions when you need to know it for certain operations.\n\n\n## Getting Started\n\nInstall via [npm](https://www.npmjs.com/):\u003cbr\u003e\n`npm install derivejs` \u003cbr\u003e\nor clone the [git repository](https://github.com/yuval-a/derivejs).\n\nYou should also setup or have access to a [MongoDB](https://www.mongodb.com/) server, and have it running.\n\nTo get started, require DeriveJS, and then call the `Model` module. The module is a `Promise` that resolves with a `Model` function. By the time the Promise is resolved, the module has finished its initializations, and is connected to the MongoDB server. When you call the module, you can pass an `options` object argument to it.\nThe `options` can contain these key:value arguments:\n\n* `dbUrl`: the MongoDB server connection url, as a string. Default: \"`mongodb://localhost:27017/`\".\n* `dbName`: the name of the database, as a string. Default: \"`deriveDB`\".\n* `debugMode`: A boolean. If set to true - will display some real-time internal DB-related `SyncManager` information - such as when it is locked for operation (before running bulk database operations), or unlocked, when running bulk inserts/updates, and more... Default: `true` (!)\n* `defaultMethodsLog`: when set to `true` - the default class methods for database/data events (`_inserted()`, `_isDuplicate()`, `_error()` and `changed()`) will run a relevant `console.log`, see [Database persistence callbacks](#database-persistence-callbacks) for more information about these methods.\n* `dbOptions`: You can use this to override the default [MongoDB driver connection options](https://mongodb.github.io/node-mongodb-native/3.5/reference/connecting/connection-settings/). Note that these are the options passed by default:\n```json\nnative_parser:true, \nforceServerObjectId:true,\n// New in 3.X Mongo engine\nuseUnifiedTopology: true,\nignoreUndefined: true\n```\n\nHere is an example of how to initialize the module:\n\n```javascript\nconst derive = require('derivejs');\nderive.Model\n({\n    dbUrl: \"mongodb://localhost:27017/\",\n    dbName: \"mydatabase\",\n})\n.then(\n    Model=\u003e {\n\t\t// `Model` is a function here.\n    }\n);\n```\n\nNEW: starting with version 1.6.0 you may use the `Connect` module function (`derive.Connect`) instead of `Model` (`derive.Model`), which is exactly the same as `Model`, and is used as an alias (both reference the same module) - and was added to avoid confusion between the resolved `Model` function (the function you use to define data models) and the **module** `Model` function (the function which is used to connect to the database and resolve with the `Model` function), so the second line above could be:\n```javascript\nderive.Connect\n```\nas well.\n\n### Defining a Data Model\nOnce the promise resolves successfully, you have access to the `Model` function, which can be used to define a \"Data Model\", by passing an object literal as an argument, describing the data properties and their default values (as well as some instance methods, when needed). The `Model` function returns a **`class`** \"representing\" that data model, and its functionality (as mentioned, that class is a special \"proxied\" class that comes with some built-in stuff in it, to handle database persistence, and offer some static and default instance methods which will be described below).\n\nLet's create a data model to represent a \"spaceship\":\n\n```javascript        \nvar Spaceship = Model({\n    _name: \"\",\n    _TYPE: \"\",\n    crew: []\n}, \"Spaceship\");\n```\n\nThe property names specify the expected property key names for this data model. The values are **default values** - values that\nwill be assigned to the properties upon initial creation of the data object (and data document), you can put any values you like, including `null`, empty strings, strings, numbers, objects (with keys and values) and so on.\nUnlike many other ODMs - Derive does *not* enforce any strict data types, preserving the spirit of dynamic data typing.\n\n**A note about objects as values**: If specifying an empty object ({}) as a value, you will not be able to set any new properties to it (as they will not be part of the \"schema\"), however, it is perfectly acceptable to assign a totally new object \nwith properties and values as a value to any defined property.\nIf you assign an object, with properties and values as a default value to a property in the schema specification - you may change the values of those properties, but trying to set a new property to that object will yield an error (\"Trying to set unknown property).\n\nNow `Spaceship` is a `class` you can create new object instances of.\nThe second argument for the `Model` function is a name, that will be used for both the class name, and the collection name (where an `s` will be added to, to signify \"plural\" form; So the collection name will be `Spaceships` in this case). There is an additional optional argument, that can set the \"sync interval\" duration, the amount of time between each interval where the SyncManager class instance runs the database bulk operations stacked since the last sync. The default is 0, and the call is made using `setImmediate` to avoid \"clogging\" Node's Event Loop (Note: prior to version 2.0.0, the default was 1000 ms).\n(The Model function also have two additional arguments `_syncer`, and `_proxy`, used internally and that shouldn't be used).\n\nNotice that some properties were defined with an underscore, and some are uppercase. This is **intentional**, and meaningful. These are called \"*Modifiers*\" and are explained in the next section.\n\n## Going deeper\n\n### Modifiers\nYou can use some special characters in the property names (called \"*Modifiers*\"), that will define certain characteristics of those properties:\n\n* `_` (start) - **Index**: When used as the **first** character of a property, will mark it as an **`index`**. Other than being defined as an index inside MongoDB, an index is also always setabble from the model class **constructor**. \n*Notes*: The order of the indexes defined, matters - as this will also be the order of the arguments in the constructor. Furthermore - the order affects which index is considered as the [\"Main Index\"](#mainindex). Also note, that the property name **does** include the underscore character. \nExample: `_name`;\n\n* `$` (last) - **Unique Index**: When used as the **last** character of an **index** property, that index will be set as a *unique* index (using the same value for unique indexes will yield an error).\nExample: `_name$`, will define a *unique index* called `_name` (notice - the `$` char at the end of the property name will be removed, and will **not** be defined as part of the name).\n\n* `ALL_UPPERCASE` - **Read Only**: When a property name is defined with all capital letters, it will be marked as **read-only**. If you try to set the value of a read-only property (using the `=` operator) - you will get an error message. Note, that if you also define a read-only property as an *index*, like in the above example - that property will **still** be settable via the constructor arguments (but **not** from anywhere else).\n\n* `$` (start) - **Meta Property** Putting the Dollar sign as the **first** character of a property name - will define it as a \"**meta**\" property (aka a \"secret\" property). A meta property will **not** be considered as part of the data structure of the model - it and its value will **not** be persisted on the database. If you iterate over the values of the data instance - it will **not** appear (it won't be enumerable). But you may **still** get and set its value locally. This is useful for saving some additional information that you only need locally, and does not require persistence on the database server. These can also be used to reference callback functions, as demonstrated later-on.\u003cbr\u003e\nThere are also, in-fact, five \"built-in\" meta properties,  four are automatically created for each object:\n`$_ModelInstance` which is always equal to `true`, and is used internally when setting a DeriveJS object value to an instance of another DeriveJS object (in which case it will be saved as a DBRef object), [`$DefaultCriteria`](#defaultcriteria) (which is explained later), [`$_BARE`](#$bare) which contains the \"raw\" (unproxified) document. Setting the values of the $_BARE object - will **not** affect the database, [`$_dbEvents`](#database-persistence-events-_dbevents) - an EventEmitter for listening and responding to DB persistence events. The fifth property is [`$Listen`](#listen), which is not created automatically but can be defined as an array of property names that you want their value-changes to be \"listened\" to (as explained in \"[Listening for changes](#listening-for-local-changes)\"). \n\n* `_` (end) - **Default Criteria**: Using a `_` character as the **last** character in the end of a property name, will add the property **and** its value to the [`$DefaultCriteria`](#defaultcriteria) object, (note, the last `_` will be omitted from the property name). \nIf using both last `_` and last `$` (i.e. setting both a unique index and a default-criteria value, make sure the `_` is the **last** character, and the `$` is one before it).\n\nHaving obtained our data class, we can create object instances of it:\n\n```javascript \nvar ship = new Spaceship('The Beyond'); \n```\nEach new instance - will have an identical data record (a \"document\") in a MongoDB database, in a `Spaceships` collection,  which will always be synced with the changes you make to the \"local\" object. Once you create a new instance, that instance will also have an auto-generated `_id` value (of type [ObjectID](https://docs.mongodb.com/manual/reference/method/ObjectId/)) associated with it. \u003cbr\u003e\n(**Warning**: You should not alter the `_id` property yourself, nor should you try to give it your own value, doing so may lead to data inconsistency, and may affect data integrity).\n\nAfter a while you should see a message on the console: `The Beyond inserted`, that message comes from an instance method that all model instances have by default,\nand can be overridden by subclasses, or by redefining in the model definition:\n```javascript\n_inserted() {\n    console.log (this[MainIndex]+\" inserted\");\n}\n```\n([`MainIndex`](#mainindex) is an internal variable that holds the \"primary\" index of the collection, in this case `_name`).\nThe method is called as soon as the data object is persisted on the MongoDB server.\n\nNotice how we passed a `_name` for the new instance via the first argument of the constructor. We can do that, since we defined it as an index. We can also pass a value\nfor the `_TYPE` index as the second argument. Let's create a different ship, and define it as \"TYPE A\":\n```javascript \nvar shipA = new Spaceship('The Beyonder','A');\n```\n\nIf we now run the Mongo console, and run a \"find all\" query on the `Spaceships` collection (`db.Spaceships.find({})`, we will see our two `Spaceship` data objects saved on the server:\n\n```\n{ \"_id\" : ObjectId(\"5a063879cc5cac16b82d05f0\"), \"_name\" : \"The Beyond\", \"_TYPE\" : \"\", \"crew\" : [ ] }\n{ \"_id\" : ObjectId(\"5a063879cc5cac16b82d05f1\"), \"_name\" : \"The Beyonder\", \"_TYPE\" : \"A\", \"crew\" : [ ] }\n```\n\nYour `_id` values will vary, of-course.\n\n### Built-in Methods: Callbacks and Hooks\n\n**NEW VERSION UPDATE:**  \n* Version 3 made the use of [Change Streams](https://docs.mongodb.com/manual/changeStreams/) optional, by adding an additional last boolean argument to all data retrieval functions, \nset to `false` by default. Using ChangeStreams and assigning event listeners to data object instances creates strong references to the instances in memory, thereby preventing them \nfrom being garbage collected, this can amount to excess memory usage and can lead to memory leaks - therfore this feature is turned off by default, and can be optionally turned on, \nfor specific data objects. Consider this when using this feature, and use it sparingly. Prior (2+) versions always used ChangeStreams by default for all data objects.\n* Added the instance `watch(on)` method, that lets you turn on or off ChangeStream support.\n* Turning on the use of [Change Streams](https://docs.mongodb.com/manual/changeStreams/) on data objects means that when you work with a local document, \neven when the data changes in the DB from some other external source - **this change will be detected** and **will be** automatically reflected in your version of the document as well. \u003cbr\u003e\n**NOTE**: Change Streams are only supported for Replica Sets. If you use a single DB instance, then they will **not be used**, and changes will only be triggered by the \n`SyncManager` itself - this means that in this case - changes to the DB from another external source - **will not** be immediately reflected in your local documents (you would need to call the `.get()` method to retrieve a \"fresh copy\"). Note that the Mongo team recommends to **always use** replica sets in production environments. \u003cbr\u003e\nChange Streams are also only supported in WiredTiger storage engine (which is the default for Mongo).\n* New in version 2+: all Derive objects now have a `$_dbEvents` meta property which is an `EventEmitter`, that you can use to listen for DB persistence events for specific instances. \nSee [Database Persistence Events (`$_dbEvents`)](#database-persistence-events-_dbevents) for more information.\n* Version 2+ update: the `$_updated` method is now *deprecated* in favour of a new different syntax for defining callbacks for specific DB updates. \nSee [\"Assignment with`$callback`\" Syntax (Update Callbacks)](#assignment-with-callback-syntax-update-callbacks) for more information.\n\n#### New Model Instance Lifecycle\nWhen creating a new model object instance - first the, \"internal\" constructor of the model class is called. \nThis constructor is defined within the `Model` module, and is the same for **all** model classes (having different initialization properties, of-course). First, if the constructor was called as a result of retrieving a data object (document) from the database - the object values will be populated accordingly. Then, this constructor function is basically in-charge of the following things, in this order:\n\n1. \"Proxifiying\" all (non-meta) property values of the object (so they can all be automatically persisted to the DB upon change).\n2. Starting (running) the associated `SyncManager` class (if it's not already running) - which will \"gather\" DB-related operations and bulk-run them in fixed intervals for the associated DB collection).\n3. If this is a **new** data object - create a **local** data (MongoDB) document first - this is the point where the `_created` function of the data class is called (if defined) - and is the \"last chance\" to make new changes before the data object will be persisted on the DB (of-course, you can also update any of its properties as you like later-on, and they'll be persisted as well).\n4. The `SyncManager` puts an `insert` operation in the queue of the next bulk DB operations to be run.\n\nIt's important to note, that if you extend a Model class, and define both a constructor on the extended class **and** a `_created` method - \nthe `_created` method will be called **before** the extended class constructor finishes - as it needs to call the base class (\"super\") constructor first, which will call `_created` if defined.\nSo, for example with the following:\n\n```javascript\nvar Spaceship = Model({\n   _name: \"\",\n}, \"Spaceship\");\n\nclass Ship extends Spaceship {\n   constructor (_name) {\n      super(_name);\n      console.log (_name+\" called from constructor\");\n   }\n   _created() {\n      console.log (this._name+\" called from _created\");\n   }\n}\n\nvar ship = new Ship(\"a new ship\");\n```\nYou will first see the `console.log` message from the `_created` and *then* the message from the (extended class) constructor.\nAfter a short while, when the document is inserted on the DB - you will see messages logged from the `_inserted` method (if a `console.log` was defined there and `debugMode` was defined with `true` when initializing the `Model` module).\n\n#### Database Persistence Callbacks\n\nDerive has several predefined callbacks defined on the Model class level, are available to all Model instances, and can be overriden in a child class, or in the model definition itself. These method names start with an underscore (`_`).\n\n##### `_inserted()`\n\nIf you need to know exactly when an object is actually persisted in the database every instance has a built-in instance method: `_inserted()` which is called as soon as it's inserted in the DB.\nYou can override that method - either by extending a class - or defining the method directly inside the model definition -- and put some \"post-persistence\" code there, if you need.\n\n```javascript\nclass Ship extends Spaceship {\n    _inserted() {\n        console.log (this._name+\" created, with id: \"+this._id); \n    }\n}\nvar ship = new Ship(\"The Created\");\n```\nwill yield:\n`The Created created, with id: 5a063f842ef67924f4e0f9bb` (with a different id of-course).\n\n##### \"Assignment with `$callback`\" Syntax (Update Callbacks)\nThis is the new way in version 2 and up to assign function callbacks for specific property updates to a data object. With this syntax, instead of directly assigning a value \nto a property of a data object, you instead assign it an object with two properties:\n###### `$value`\nThe actual value you want to assign.\n###### `$callback`\nA function that will be called once the property of the equalivent document in the DB is actually updated.\n###### Example:\n```javascript\nFeisty.captain = {\n    $value: Wort,\n    $callback: ()=\u003e {\n        console.log (\"Wort was updated as the captain of the Feisty\");\n    }\n}\n```\nThe value of `$value` will be assigned to the property, and the function in `$callback` will be called once that property is updated with that value on the DB.\n\n##### `_isDuplicate()`\nCalled when the MongoDB server yields a \"duplicate key value\" error, and contains by default: \u003cbr\u003e\n`console.log (this[MainIndex]+\" has a duplicate key value!\");`\n\n##### `_error(msg)`\nCalled whenever there is a data-related error for this object, and contains by default: \u003cbr\u003e\n`console.log (\"Error in \"+this[MainIndex]+\": \"+msg);`\n\n#### Database Persistence Events (`$_dbEvents`)\n**NOTES**: \n* `$_inserted` and `$_updated` meta methods are *deprecated* since version 2 and up.\n* `$onUpdate()` is also *deprecated* since version 2. See [\"assignment with `$callback` syntax\"](#assignment-with-callback-syntax-update-callbacks).\n\nEach Derive data object has a built-in `$_dbEvents` meta property, which is an `EventEmitter` object, that you can use to listen for DB persistence events and changes, \nby calling the `on` or `once` methods to listen for events in specific instances, and attach handler functions (see [Node Events documentation](https://nodejs.org/api/events.html) for more information about `EventEmitter`).\n\nThese are the events available via `$_dbEvents`:\n##### `inserted` Event\nCalled once a MongoDB document for this instance was inserted to the DB collection. The callback function receives two arguments:\n###### `id` \nThe `_id` of the inserted document.\n###### `insertedObject` \nThis is the same relevant Derive data object instance that was created.\n###### Example\n```javascript\n(new PhotonTorpedos()).$_dbEvents.once(\"inserted\", (id, torpedos)=\u003e {\n    // `torpedos` is the PhotonTorpedos instance.\n});\n```\n##### `updated` Event \nCalled once a MongoDB document's property is updated on the DB. The callback function receives three arguments:\n###### `id` \nThe `_id` of the updated document.\n###### `updatedFields`\nAn object where the keys are updated property names, and the values are the new updated values.\n###### `updatedDocument` \nThe Derive object instance.\n###### Example\n```javascript\nBoldlyGo.$_dbEvents.on(\"updated\", (id, updatedFields)=\u003e {\n    console.log (\"BoldlyGo updated properties: \");\n    console.dir (updatedFields, {depth:null});\n});\n```\n\nThe following are additional ways to implement DB persistence callbacks. They were the recommended ways for previous versions of Derive. \u003cbr\u003e\nFor versions 2 and up, the recommended way is to subscribe to DB events, or use [\"assignment with `$callback`\" syntax](#assignment-with-callback-syntax).\n\n##### Setting A Callback Function to a Meta Property\nYou can define a meta property on the model, to hold a callback function.\n\n```javascript\nvar Spaceship = Model({\n    _name: \"\",\n    _TYPE: \"\",\n    crew: [],\n\n    $createdCallback: undefined\n\n}, \"Spaceship\");\n```\n\nThen extend the class, and allow passing a callback function via the constructor. Call the callback from the overridden `_inserted` function:\n\n```javascript\nclass Ship extends Spaceship {\n    constructor(_name, _TYPE, callback) {\n        super(_name, _TYPE);\n        this.$createdCallback = callback;\n    }\n    _inserted() {\n        this.$createdCallback.call(this);\n    }\n}\n\nvar ship = new Ship(\"shipA\",\"\", function() {\n    console.log (\"shipA created!\");\n});\n```\n\nNote how when overriding a constructor in a child class - you need to specify the indexes as arguments before adding new ones, and of-course, you need to call the parent constructor via `super`.\n\n#### `watch(on)`\n`watch` is a function all data object instances have, calling it can toggle on/off the ChangeStream support for the data instance. When ChangeStream support is on - the object instance will be aware of updates and changes to the document in the database **from external sources**, reflecting those in the local data object instance, as well as triggering an 'updated' event on `$_dbEvents`. Note that turning ChangeStream on, will assign an event listener to the ChangeStream - that will keep a **strong reference** in memory to the data object instance, thereby making it unavailable for garbage collection - this means that update events will trigger even if the data object instance is set to `null`. If you use `watch` to turn on ChangeStream support and then no longer need to data object instance, you should make sure you turn ChangeStream off first, by calling `watch` with `false` - this will make the instance garbage collectible once there are no more references to it.\n\n##### Listening For Local Changes\nThe fourth built-in method can be used when you want to listen for value-changes on certain properties of your object,\n(**Note**: this will trigger on *local* changes to the properties, regardless to their state in the equavilent documents in the database collection)\n\n###### `changed(property, newValue, oldValue)`\nThe `changed` method contains this code by default: \u003cbr\u003e\n`console.log (this[MainIndex]+\":\",property,\"changed from\",oldValue,\"to\",newValue);`\n\n###### `$Listen`\nTo register a property for the listener, put its name (as a string) inside an array defined as the `$Listen` meta-property (e.g. `$Listen: [ \"property\" ,\"otherproperty\", \"objectprop.prop\"]`.\n\n### Unique Indexes\nNow, we decide that we want the `_name` property index to be unique:\n\n```javascript\nvar Spaceship = Model({\n    _name$: \"\",\n    _TYPE: \"\",\n    crew: [],\n},\"Spaceship\");\n```\n\nOnce the engine modifies the `_name` index to make it unique, if there are records with duplicate `_name` values - Mongo will throw an **error**, and the unique index will **not** be defined. You will need to take care of the duplicates yourself for it to successfully be defined. You can either issue a relevant `.remove` command from the mongo console, or you can also use the static `clear()` method on the `Spaceship` class. The `clear` method can accept a \"find query\" filter as an argument (e.g. `{_name:\"The Beyond\"}`). So, for example if you have several ships named \"`The Beyond`\" and you want to define a unique constraint on `_name` and create a new, single `\"The Beyond\"` ship, then you can use:\n\n```javascript\nawait Model({},\"Spaceship\").clear({_name:\"The Beyond\"});\nvar Spaceship = Model({\n    _name$: \"\",\n    _TYPE: \"\",\n    crew: [],\n}, \"Spaceship\");\n\nvar ship = new Spaceship (\"The Beyond\");\n```\n\nThe first line is just for getting a reference to the `Spaceships` collection, to call `clear` to remove all \"`The Beyond`\" ships, then we can define the model with the new constraint, and create a new unique ship.\n\nAt this point we have the `_name` index defined as a unique index -- there can't be more than one object with the same `_name` value. Let's see what happens when we\ntry to create two Spaceships with the same name:\n```javascript\nvar ship1 = new Spaceship (\"The Boldly Go\",\"A\");\nvar ship2 = new Spaceship (\"The Boldly Go\",\"B\");\n```\n\nWe will get a message on the console:\n`The Boldly Go has a duplicate key value!`\n\nOne of the records was successfully inserted to the database, the other was detected as having a duplicate `_name` and was rejected. The message is coming\nfrom the `_isDuplicate()` instance method, that all Model instances have. Its body is defined with:\n\n```javascript\n_isDuplicate() {\n    console.log (this[mainIndex]+\" has a duplicate key value!\");\n}\n```\n\nYou may override this method in your class extension or model definition, and define your own logic to take care of duplicates. If we look inside the database, we'll see that only the ship with `_TYPE` `A` is saved on the database.\n\nIt is possible to remove a unique constraint from an index - simply define it without the `$` sign - and the engine will know how to redefine it.\nYou may also \"disable\" an index - redefine it as a \"normal\" property - to do so, just define it without the leading `_`. \nSuppose we want to unindex the `_TYPE` property, we can define our model as:\n\n```javascript\nvar Spaceship = Model({\n    _name$: \"\",\n    TYPE: \"\",\n    crew: [],\n}, \"Spaceship\" );\n```\n\n**However** - any past documents in our database collection, will *still* have the old `_TYPE` property in them. \n\n### The `remodel` method\nIf you'd like to retroactively rename all the old `_TYPE` properties in existing documents to `TYPE`, you can use the static `remodel()` method. That function gets an object of options, as an argument - \nwhich you can use to switch on certain types of the method's operations. Currently there are two:\n\n```javascript\ndeep: true,\nrenameIndexes: true\n```\n\nWith `renameIndexes:true` - the method will rename properties in documents in the collection that starts with a `_` and are defined in the model without it. Similarly it will rename properties **without** a leading `_` that appear in the model definition with it (for situations where you decide to define an index for a non-index property).\nSo, to rename all `_TYPE` properties to `TYPE` in our collection, we could have used:\n`Spaceship.remodel( { renameIndexes: true } );`\n\nThe `deep` operation allows you to retroactively define new properties on existing documents on a collection. If you add a new property to your model definition (for example, you add a `captain` property with a default value of an empty string) - then older documents won't have it (which may or may not be what you want) - \nusing `deep:true` will add the new property with its default value to **all** existing documents in the collection.\n\nTo read more about indexes and how they are managed in DeriveJS -- see \"[Indexes and how they are handled in the Mongo server](#indexes-and-how-they-are-handled-in-the-Mongo-server)\".\n\nYou will probably want to have `remodel` called only once after you make changes to your models, therefore it might be a good idea to call it in response to a certain command-line argument when running your Node app.\nFor example:\n\n```javascript\n// process.argv is an array, where the first item contains 'node', the second item contains the script file name, and the rest of the items are command-line arguments\nif (process.argv[2] == \"--remodel\") {\n    Spaceship.remodel({ deep:true, renameIndexes: true});\n}\n```\n\n\n## Going Further - Extending and Deriving Models\n\nOne of Derive's powerful features is the ability to create \"sub-models\" associated with the same collection as a \"super-model\", using the [derive](#going-further-extending-and-deriving-models) method, which, together with the [`$DefaultCriteria`](#defaultcriteria) modifier, can be used to create meaningful data \"inheritance\":\n\n```javascript\n// Define Animal data model\nconst Animal = Model({\n   // The underscore denotes this property as an index\n   _name:\"\", \n   type:\"\", \n}, \"Animal\");\n\n/* \n * Define a \"derived\" Dog data model (both Dog and Animal will be under the Animals collection), \n * and assign \"Dog\" as a default value for `type` for all `Dog` models.\n * We use an underscore as the last character to add `type: \"Dog\"` - to the \"Default Criteria\" - \n * this means that all data retrieval methods will automatically add `type:\"Dog\"` to their find queries.\n */\nconst Dog = Animal.derive({ type_:\"Dog\" });\n\n// The new data object and document (Ubu) will also automatically have `type: \"Dog\"` associated with it.\nlet ubu = new Dog(\"Ubu\");\n\n// Get all Dogs\nDog.getAll().then(dogs=\u003e {\n   // Got all dogs here.\n});\n```\n\nGoing back to our \"Spaceships\":\n\nLet's define a \"sub-type\" of `Spaceship`, we'll call it a `Battleship`, and we'll also add a property to hold its weapons.\nIf we do it like this (note, this is the WRONG way):\n\n\n```javascript\nclass Battleship extends Spaceship {\n    constructor (_name,_TYPE) {\n        super(_name,_TYPE);\n            this.weapons = [];\n        }\n    }\n}\n```\nAnd then create a new instance:\n```javascript\nvar bship = new Battleship(\"The Destroyer\");\n```\n\nThen we will get an error message on the console:\n`Error in The Destroyer: Trying to set unknown property: weapons (property value is left unchanged).`\nSince `weapons` is not a known property of the model, as it wasn't defined as part of its structure.\nThat error message, by the way, is displayed via the `_error(msg)` instance method that all instances have, and is called whenever there is an error\n**related to that instance**.\nThe new object **will** be inserted into the database, it just won't have any `weapons` property defined on it.\n\nTo add new properties to an already existing model - you should use the static `derive(modelExtension)` method on the Model class. That method\nuse an object defining the *additional* properties and values that should be added to the parent-model (the \"super-model\" :) ), \n\nSo, the more \"right\" way to extend the `Spaceship` to a `Battleship` in this case is\n(and let's also set the `_TYPE` to \"`Battleship`\" as well):\n\n```javascript\nclass Battleship extends Spaceship\n.derive ( { weapons: [] } ) \n{\n    constructor (_name) {\n        super(_name,\"Battleship\");\n    }\n}\n\nvar bship = new Battleship(\"The Destroyer\");\n```\n\nThe `derive` method returns a new Model Class (*not* a subclass of the original Model class) - that shares the same collection as its \"parent\" data model (so both will \nbe part of the `Spaceships` collection, and both will use the same existing database synchronization engine (SyncManager).\n\nIt is also possible to \"override\" properties and values in the derived model, and so we can simply \"override\" the `_TYPE` property in the derived model, and\nset its default value to \"`Battleship`\". There won't even be a need for a subclass in this case:\n\n```javascript\nvar Battleship = Spaceship.derive ({ \n    _TYPE:\"Battleship\", \n    weapons: [] \n});\n\nvar bship = new Battleship(\"The New Destroyer\");\n```\nNote: you may not set an existing index as a unique (by adding a `$`) in a derived model - doing so will have no effect on the index.\nYou **may** define new indexes - and only objects instances of the derived class will have them. Indexes defined within a derived model - \nare always defined with the `sparse:true` property on the Mongo DB.\n\nYou can read more about indexes and how they are managed in deriveJS [here](#indexes-and-how-they-are-handled-in-the-Mongo-server)\n\n## Retrieving data from a database\nYou will often want to \"restore\" existing database objects from data collections and populate your local ones with the persisted data. \nEach Model class have various different static methods used to achieve this; \nmost of these are wrappers around certain Mongo `find` queries, which will make the process easier and more intuitive.\n\nThere are several methods that can be used to retrieve data from the database, here is a brief explanation for each:\n\n**NOTE**: Starting from version 3, all data retrieval functions have an additional boolean `collectionWatch` argument as their last argument, with `false` as the default value. \nSetting this argument to `true` will enable `ChangeStream` support for the retrieved data object(s), will make the data object(s) be aware to updates and changes to the DB \ndocument **from an external source**, and reflect those in the local data object(s). Assigning ChangeStream event listeners to data instances creates strong references in memory \nto them, thereby preventing them from being garbage collected (even if you set them to `null` or `undefined`). This can amass to an excess use of memory and can lead to memory leaks -\nthereby this feature is turned off by default, and should be used sparingly (for example, if you intend to keep references to data object instances indefinitely).\nPrior versions (2+) had this feature on for all data object instances by default.\n\n### `get(which, collectionWatch = false)` \nReturns **one** object instance.\n\n### `getAll(which, sortBy, limit=0, skip=0, collectionWatch = false)` \nReturns **all** (with an optional filter query) object instances (in an array). There are 3 more additional optional arguments:\n* `sortBy` - to specify a different index to sort by, using an object such as `{\u003cindexName\u003e: \u003c-1 or 1\u003e}` - use `-1` for descending order, and `1` for ascending (the default is `{MainIndex:-1}`).\n* `limit` - to limit the number of returned results (default is `0`, which is unlimited). \n* `skip` to specify an offset index for retrieved results (default is `0`).\n\n### `getAllCursor(which, sortBy, limit=0, skip=0, collectionWatch = false)`\nThis function is identical in functionality to [`getAll`](#getallwhich-sortby-limit0-skip0), only it returns a **[Cursor](https://mongodb.github.io/node-mongodb-native/3.6/api/Cursor.html)**. The cursor has identical functionality to the Node's MongoDB driver cursor, with one additional function:\n`getNext` - which is identical to the cursor's native [`next`](https://mongodb.github.io/node-mongodb-native/3.6/api/Cursor.html#next) function, only it \nreturns a Promise that resolves with the next document as a **Derive data object** (or null if there are no more documents to retrieve ).\n\n### `getAllRead(which, sortBy, limit=0, skip=0, collectionWatch = false)`\nThis function is idential to [`getAll`](#getallwhich-sortby-limit0-skip0), only it returns \"raw\" Mongo documents, and **not** Derive data objects. \nAs with `getAll` there are some performance degradation when retrieving a large amount of documents, as a new instance is created for each, \nif you only need to retrieve data for read only purposes, you may use `getAllRead` instead. Remember that the returned documents will **not** be reactive.\n\n### `map(which, index, returnArray, limit=0, skip=0, collectionWatch = false)` \n`map` returns **all** (with an optional filter query) object instances mapped by an index as an object, or as an array (according to the third boolean argument `returnArray`). The second argument `index` lets you set the index name that will be used for the mapping (set as null to use the default `MainIndex`). \nThe 4th and 5th arguments are `limit` and `skip` that allows you to get only some of the documents.\n\n### `mapRead(which, index, returnArray, limit=0, skip=0)`\nThis function is idential to [`map`](#mapwhich-index-returnarray-limit0-skip0), only it returns \"raw\" Mongo documents, and **not** Derive data objects. \nAs with `map` there are some performance degradation when retrieving a large amount of documents, as a new instance is created for each, \nif you only need to retrieve data for read only purposes, you may use `mapRead` instead. Remember that the returned documents will **not** be reactive.\n\n### `has(which, returnDocument, collectionWatch = false)` \nReturns a boolean indicating if a database collection contains certain value(s). The second `returnDocument` is a boolean - if the document exist and this argument is set to `true` -\nthe document will also be returned (as a Derive data object), otherwise `false` will be returned (regardless of the value of `returnDocument`).\n\n### `count(which)` \nReturns the total number of documents in the DB collection.\n\nAll of these methods can use a [`which`](#which-argument) argument, and to understand how to use it, you need to know about `MainIndex`:\n\n###  `MainIndex`\n`MainIndex` is an internal value that each model class has and is determined during its definition process. \u003cbr\u003e\nThe `MainIndex` will contain the \"most important\" index for that class/collection.  \u003cbr\u003e\nIts value will be determined according to the following:\n* If there is at least one **unique** index - the first unique index defined on the model will be determined as the MainIndex. \u003cbr\u003e\n* If no unique index is defined, then the first **non-unique** will be determined as the MainIndex. \u003cbr\u003e\n* If no index is defined in the model, the **`_id`** property will be determined as the MainIndex.\n\n#### `mainIndex() method`\nYou can use the static `mainIndex` method to get the name of the main index, as a string. This can be useful, for example, when overriding any of the [callback methods](#database-persistence-callbacks), e.g.:\n\n```javascript\nconst Spaceship = Model({\n    _name: \"\",\n    _TYPE: \"\",\n    _inserted() {\n       console.log (\"Mention the value of this instance's main index: \" + this[Spaceship.mainIndex()]);\n    }\n}, \"Spaceship\");\n```\n\n### `which` Argument\nWhen you use the `which` argument, you have two options - \nif you pass a **primitive** value (string, number or boolean) - then the function will look for objects where the `MainIndex` is that value,\nif you pass an **object** - then that object will be used as a query object for Mongo's `find`. \u003cbr\u003e\n*Note* if your MainIndex is an object itself - you need to use the normal `find` query format (e.g. if your index-object is called `ob`: `{ob: {prop1:value,prop2:value}}`).\n\n#### Selective Fields (Projection)\nIf you'd like to return data objects containing only some of the fields from the model, you may pass an\nobject in `which` having a `fields` property containing an array with a list of fields to include in the returned\ndata object. \u003cbr\u003e\nNotes: \u003cbr\u003e\nThis can be used with any of the data retrieval functions (get function variations, map functions, and join functions). \u003cbr\u003e\nIf you use this, you have to specify your search criteria using the normal Mongo's `find` syntax. \u003cbr\u003e\nOnly fields you specify in the array in the `fields` property will be included in the data object instance. \u003cbr\u003e\nIf you include fields that doesn't exist on a data document - the data object will not have that field. \u003cbr\u003e\n`_id` field is always returned. You cannot disable this. \u003cbr\u003e\n\nLet's see some examples, we assume the models from the previous examples are already defined.\n\nTo get our \"`The Beyond`\" Spaceship from the database into a local object:\n\n```javascript\nvar thebeyond;\nSpaceship.get(\"The Beyond\")\n.then (ship=\u003e {\n\tthebeyond = ship;\n\t// thebeyond contains the object from the DB\n});\n```\n\nOr, using the `async/await` way:\n\n```javascript\nasync ()=\u003e {\n    var thebeyond = await Spaceship.get(\"The Beyond\");\n    // thebeyond contains the object from the DB\n});\n```\n\nPut all of our `Spaceship` objects from the DB into an array:\n\n```javascript\nvar spaceships = await Spaceship.getAll();\n```\n\nGet all \"`_TYPE C`\" spaceships into an array:\n\n```javascript\nvar spaceships = await Spaceship.getAll({_TYPE:\"C\"});\n```\n\n### `map` - Additional Information\nThe `map` method has additional two optional arguments (other than the first `which`): `index`: to specify a different index to be used as the key\nfor mapping the objects (other than `MainIndex`), and `returnArray`: a boolean specifying if to return the result as an \"associative array\" of object instances mapped to key indexes or as an object with object instances mapped to index keys.\n\n```javascript\nvar spaceships = await Spaceship.map();\n```\n`spaceships` will then be an object, where each key is a `_name` value, and each value is the equivalent `Spaceship` object.\nYou can then, for example, reference `The Beyond` ship from it:\n\n```javascript\nvar thebeyond = spaceships[\"The Beyond\"];\n```\n\n`map` is obviously meant to be used with a unique index. If it's used with a non-unique one - and several objects exist on the database with duplicate values for an index - then some of the objects will be \"overriden\" when calling `map`.\n\nSo with:\n```javascript\nvar spaceships = await Spaceship.map({},\"_TYPE\");\n```\nIf we have one Spaceship with `_TYPE` set to `C`, and several others with `_TYPE` as en empty string - then `map` will return only two ships:\nOne of the empty string ones, and the `C` one.\n\nWe can use the `which` argument to further filter-out the objects, for example - return all `_TYPE C` ships\n(they will *still* be mapped to their `_name`s):\n\n```javascript\nvar spaceships = await Spaceship.map({_TYPE:\"C\"});\n```\n\nThe last method `has` can be used to determine if a certain object exist on the database. For example:\n\n```javascript\nvar hasTheBeyond = await Spaceship.has(\"The Beyond\");\n```\nWill return `true` if a Spaceship object with its `_name` set to \"`The Beyond`\" exist on the DB.\nThe method also has a second argument - `returnDocument`, a boolean that if set to `true` will also return the object if it exist on the DB \n(or `false` if it doesn't).\n\n#### A Word of Caution for When Using the `get` Functions\nUpon retrieving the objects from the database - their `constructor` functions **will** be called for each object.\nTherefore - if you override the constructor and have any code that affects or changes the data there - it **will** run - that is usually *not* desired when retrieving data object, so you should make sure you call the `get` functions from a (usually \"higher\") class that runs a constructor that does not change the data (like the default constructor).\n\n### `$DefaultCriteria`\n\nAll model classes have a \"static\" (meta) property - `$DefaultCriteria`, this is an object containing key-value pairs that will be added by default to database queries, when calling one of the `get` methods, using a primitive value as an argument. This is useful when creating derived or extended classes, and wanting to include just the \"sub-classes\" in query results. It will be more clear with an example:\n\nReturning to our Spaceship example, we define a `Spaceship` super-model, as before:\n\n```javascript\nvar Spaceship = Model({\n    _name$: \"\",\n    TYPE: \"\",\n    crew: [],\n}, \"Spaceship\");\n```\n\nThen we define a sub-model of `Spaceship`: `Battleship`\n\n```javascript\nvar Battleship = Spaceship.derive ({ \n    _TYPE:\"Battleship\", \n    weapons: [] \n});\n```\n\nIf we call `getAll` on `Battleship`: \n```javascript\nvar await battleships = Battleship.getAll();\n```\n\nthen we'll get all **`Spaceship`** objects in the entire `Spaceship` collection into battleships, while we probably only want to get back the \"derived\" **`Battleship`** objects. When we want to differentiate the `Battleship` objects from the others in the collection - we can add certain key:value pairs into the `$DefaultCriteria`;\nto do so, we add an underscore (`_`) character to the *end* of the property name. So, we can add the `_TYPE` in `Battleship`:\n\n```javascript\nvar Battleship = Spaceship.derive ({ \n    _TYPE_:\"Battleship\",\n    weapons: [] \n});\n```\nWhen we do that, then `_TYPE:\"Battleship\"` will be added to the find query when we call `get` methods from `Battleship`, and so, for example, \ncalling `getAll`: \n\n```javascript\nvar await battleships = Battleship.getAll();\n```\nWill now return just the `Spaceship` objects that also has their `_TYPE` set to \"`Battleship`\".\n\n*Note*: You shouldn't access `$DefaultCriteria` directly. In case you do, make sure you set its properties, and **not** override it completely.\n\n## Using Model Instances as Values in Other Models\n\nLet's define a new model: `CrewMember`. The `crew` array of our `Spaceships` will contain `CrewMember` objects:\n\n```javascript\nvar CrewMember = Model({\n    _name: \"\",\n    rank: \"\",\n    role: \"\"\n}, \"CrewMember\");\n```\nWe can also add a function to the `Spaceship` model, to handle adding new `CrewMembers` to the `crew` array,\n(and let's also add a `captain` property)\n```javascript\nvar Spaceship = Model({\n    _name$: \"\",\n    TYPE: \"\",\n    captain: null,\n    crew: [],\n    addCrew: function(crewMember) {\n        this.crew.push(crewMember);\n    }\n},\"Spaceship\");\n```\n\nNext, we create a new `CrewMember` - captain Ricard:\n\n```javascript\nvar ricard = new CrewMember(\"Ricard\");\nricard.rank = \"captain\";\nricard.role = \"captain\";\n```\n\nWe get `\"The Beyond\"` spaceship, add `ricard` to its crew, and also set it as its `captain`:\n```javascript\nvar thebeyond = await Spaceship.get(\"The Beyond\");\nthebeyond.addCrew(ricard);\nthebeyond.captain = ricard;\n```\n\nNow if we look at `The Beyond` in the database, we can see:\n\n```\n\u003e db.Spaceships.find({_name:\"The Beyond\"})\n{ \"_id\" : ObjectId(\"5a0f42663432962910d45ab7\"), \"_name\" : \"The Beyond\", \"_TYPE\" : \"\", \"crew\" : [ DBRef(\"CrewMembers\", ObjectId(\"5a1c9a63b3c2df42587b6a90\")) ], \"captain\" : DBRef(\"CrewMembers\", ObjectId(\"5a1c9a63b3c2df42587b6a90\")) }\n```\n\nThe `CrewMember` object was saved as a `DBRef` object. A `DBRef` is a \"reference\" to an item from another collection (in this case the `CrewMembers` collection). It contains the `ObjectId` of the item, and the collection name.\nTo \"dereference\" the object, you can use the `get` function by passing it the DBRef itself. So, if we want to get back captain Ricard, from `The Beyond`'s `crew` array, we can use:\n\n```javascript\nvar thebeyond = await Spaceship.get(\"The Beyond\");\nvar ricard = await CrewMember.get(thebeyond._captain);\n```\n## Indexes and How They are Handled in the Mongo Server\n\nIndexes in collections are saved in 4 different compound indexes (specified by their index name:)\n* \"`nonUnique`\": an index containing all non unique (non-sparse) indexes\n* \"`unique`\": an index containing all unique (non-sparse) indexes\n* \"`sparse_nonUnique`\": an index containing all sparse non-unique indexes -- non-unique indexes defined in derived models.\n* \"`sparse_unique`\": an index containing all sparse unique indexes -- unique indexes defined in derived models.\n\nCollections may have only some or none of these indexes defined, depending on indexes defined on the model.\nThere will also be the `_id` index defined, as usual.\n\n## Putting it All Together:\nThe following is a complete demonstration, expanding on the `Spaceship` idea and models. \u003cbr\u003e\nThe following code examples can also be found in [this github repository](https://github.com/yuval-a/spaceships-derivejs-demo).\n\n### Defining Our Models\nFirst we define all of our data models, in a separate `Models.js` file:\n\n```javascript\nmodule.exports = new Promise( (resolve,reject)=\u003e {\n    var Models = {};\n\n    const derive = require('derivejs');\n\n    derive.Model\n    ({\n        dbUrl: \"mongodb://localhost:27017/\",\n        dbName: \"spaceshipyard\",\n        debugMode: false\n    })\n    .then(\n        async Model=\u003e {\n\n            Models.Weapon = Model({\n                _TYPE:\"\",\n                _DAMAGE:-1,\n                armed: false,\n                arm: function() {\n                    this.armed = true;\n                },\n                unarm: function() {\n                    this.armed = false;\n                },\n                fire: function(target) {\n                    if (!this.armed) {\n                        console.log (\"Weapon is not armed!\");\n                        return;\n                    }\n                    if (target.shields.up) {\n                        target.shields.percent -= this._DAMAGE;\n                    }\n                    else {\n                        target.integrityHull -= this._DAMAGE;\n                    }\n                },\n                _inserted: function() {\n                    if (this.$ready) this.$ready.call(this);\n                },\n                // For a weapon-ready callback\n                $ready: null\n            }, \"Weapon\");\n            \n            Models.PhotonTorpedos = class extends Models.Weapon\n            .derive({\n                _TYPE_: \"Photon Torpedos\",\n                _DAMAGE: 20\n            }) {\n                constructor(readyCallback) {\n                    super ();\n                    this.$ready = readyCallback;\n                }\n            };\n            \n            Models.CrewMember = Model({\n                _name: \"\",\n                rank: \"\",\n                role: \"\"\n            }, \"CrewMember\");\n            \n            Models.Spaceship = Model({\n                _name: \"\",\n                TYPE: \"\",\n                shields: {\n                    up: false,\n                    percent: 100\n                },\n                integrityHull: 100,\n                crew: [],\n                addCrew: function (crewMember) {\n                    this.crew.push(crewMember);\n                },\n                raiseShields: function() {\n                    this.shields.up = true;\n                    console.log (this._name+\": shields are up\");\n                },\n                lowerShields: function() {\n                    this.shields.up = false;\n                    console.log (this._name+\": shields are down\");\n                },\n                captain: \"\",\n                // Listen to changes on this properties\n                $Listen: [ \"shields.percent\", \"integrityHull\" ]\n            }, \"Spaceship\");\n            \n            Models.Cruiser = Models.Spaceship\n            .derive({\n                TYPE_: \"Cruiser\"\n            });\n\n            Models.Battleship = Models.Spaceship\n            .derive({\n                TYPE_: \"Battleship\",\n                weapons: [],\n                attack: function(target, weaponIndex) {\n                    Models.Weapon.get(this.weapons[weaponIndex])\n                    .then(w=\u003e {\n                        if (!w.armed) w.arm();\n                        w.fire(target);\n                    });\n                }\n            });\n\n            resolve (Models);\n        }\n    )\n});\n```\n\n### Writing Our App\nThe app code will be in an `app.js` file:\n\n```javascript\nrequire (\"./Models\")\n.then(async Models=\u003e {\n\n    // Save convient references to our data models\n    const PhotonTorpedos  = Models.PhotonTorpedos;\n    const CrewMember = Models.CrewMember;\n    const Cruiser = Models.Cruiser;\n    const Battleship = Models.Battleship;\n\n    // Will contain data instances\n    var BoldlyGo, Feisty,\n        Ricard, Wort,\n        torpedos;\n\n    function clearAll() {\n        return Promise.all([\n            CrewMember.clear(),\n            Cruiser.clear(),\n            Battleship.clear()\n        ]);\n    }\n    async function init() {\n        console.log (\"Creating Boldly Go Cruiser\");\n        BoldlyGo = new Cruiser (\"The Boldly Go\");\n        console.log (\"Creating Feisty Battleship\");\n        Feisty = new Battleship (\"The Feisty\");\n\n        console.log (\"Creating Ricard Crew Member\");\n        Ricard = new CrewMember(\"Ricard\");\n        Ricard.role = \"captain\";\n        Ricard.rank = \"captain\";\n\n        console.log (\"Adding Ricard to Boldly Go\")\n        BoldlyGo.addCrew (Ricard);\n        BoldlyGo.captain = Ricard;\n\n        console.log (\"Creating Wort Crew Member\");\n        Wort = new CrewMember(\"Wort\");\n        Wort.role = \"captain\";\n        Wort.rank = \"commander\";\n\n        console.log (\"Adding Wort To Feisty\");\n        Feisty.addCrew(Wort);\n        Feisty.captain = Wort;\n\n    }\n\n    function restore() {\n        return Promise.all([\n            Cruiser.get(\"The Boldly Go\"),\n            Battleship.get(\"The Feisty\")\n        ]);\n    }\n\n    function battle() {\n        console.log (\"Starting battle\");\n        BoldlyGo.raiseShields();\n        Feisty.attack(BoldlyGo, 0);\n    }\n\n    console.log (\"Clearing all...\");\n    await clearAll();\n    init();\n    console.log (\"Adding Photon Torpedos to Feisty\");\n    // Wait until PhotonTorpedos are added to Feisty, then run a battle\n    Feisty.weapons.push( new PhotonTorpedos(function() {\n        console.log (\"Photon Torpedos ready\")\n        battle();\n    }) );\n\n    /* \n    // You can also use Spaceship.map, but this is done here for the sake of example:\n    restore()\n    .then(ships=\u003e {\n        BoldlyGo = ships[0];\n        Feisty   = ships[1];\n        BoldlyGo.lowerShields();\n        Feisty.attack(BoldlyGo, 0);\n    });\n    */\n    \n})\n.catch (err=\u003e {\n    console.log (\"Error initializing models: \",err);\n});\n```\n\nIf we run the app we can see all the data creation messages appearing asynchronously, along with the battle messages:\n\n    Starting battle\n    The Boldly Go: shields are up\n    The Boldly Go: shields.percent changed from 100 to 80 \n\nThe Boldly Go has raised its shields, and was then attacked by the Feisty's Photon Torpedos, lowering its shields from 100 percent to 80 percent. These last message is from the built-in `changed` method, as we registered `shields.percent` for listening.\n\nIf we look at our database in our Spaceship collection, we can see `The Boldly Go` record with `shields.percent` as `80`.\n\nIf we now comment the last section, and comment-out this section:\n\n```javascript\nrestore()\n.then(ships=\u003e {\n    BoldlyGo = ships[0];\n    Feisty   = ships[1];\n    BoldlyGo.lowerShields();\n    Feisty.attack(BoldlyGo, 0);\n});\n```\nThen run the app again, then we will eventually see these messages:\n\n    The Boldly Go: shields are down\n    The Boldly Go: integrityHull changed from 100 to 80\n\nThe last one is from the `change` method.\nAfter we restore our ships data from the database - \nthen The Boldly Go lowers its shields, and attacked by Feisty again - \nwith the shields down - The Boldly Go \"integrity hull\" suffers a damage of 20 percent.\n\n## Advanced Subjects\n### Join\nYou can perform a \"join\" query with the `join` static method all model classes have:\n`join(which, joinWith, localField, foreignField, joinAs, returnAsModel=false, collectionWatch = false)`\n\n* `which` is the criteria for the document to retrieve from the \"primary\" (\"local\" collection),\n* `joinWith` is the name of the \"secondary\" (\"foreign\") collection (as a string).\n* `localField` is the name of the field that is equivalent to the `foreignField` on the secondary collection.\n* `joinAs` is the name of a property where the \"joined\" document will be included into.\n`returnAsModel` - if set to true, then the function will return an instance of the model (as in when using the `get` function) - you will most likely **not** want to set it to `true`, as the model will have \"foreign\" fields - and once you try setting or changing them - it will try to persist it to the DB. \nThis function is usually used only for getting \"readonly\" data, and not data you want to modify or change.\n\n#### An Example Use Case:\nLet's say you have a `Posts` collection and a `User` collection, and you want to get the data for a certain post, and join it with the user data of the user who posted it. With the following assumptions:\n* You have a `Post` model defined, with `_email` as its primary key, and `_authorId` with a string id containing the id of the user who posted it.\n* Your `Users` collection documents have a `_userId` field with string ids\n\n```javascript\nPost.join(\"user@email.com\",\"Users\",\"_authorId\",\"_userId\",\"author\").then(\n    post=\u003e {\n        // Now the post object here, will also have an \"author\" field containing all the data for the user with _authorId/_userId\n    }\n);\n```\nNote: Use join to join with a single document from another collection.\n\n### `joinAll`\n* `joinAll` lets you join data from two separate collections, and return multiple results.\nThe function accepts the following arguments:\n* `which` is the criteria for the document to retrieve from the \"primary\" (\"local\" collection),\n* `joinOpts` is an object with different options regarding the join:\n    * `joinWith`: the name of the \"secondary\" (\"foreign\") collection (as a string).\n    * `localField`: the name of the field that is equivalent to the `foreignField` on the secondary collection.\n    * `foreignField`: the name of the field on the joined collection, equivalent to `localField`\n    * `joinAs` is the name of a property where the \"joined\" document will be included into.\n* `findOpts` lets you specify additional \"post-find\" options, an object that can contain the following:\n    * `sortBy`: to return results sorted by a certain index, use Mongo's format for a sort-object,\n        e.g.: `{_date:-1}`, this will sort by the `_date` index in a **descending** order. To sort in an ascending\n        order, use `1` (positive 1) as the value.\n    * `skip`: lets you skip a number of results,\n    * `limit`: lets you limit the number of results returned.    \n* `returnAsModel` if set to true, then the function will return an instance of the model (as in when using the `get` function) - see notes about this in the documentation for `join`, and why you should almost never need to set this to\n`true`.\n* `collectionWatch` - if set to true, will turn on ChangeStream support for returned data objects.\n\n### Access the \"raw\" mongoDB collection object\nAlthough DeriveJS is designed, written, and intended to be in charge of all data persistence operations transparently in the background without direct interference,\nthere might come a rare occasion where you will need access to the collection object, to perform native MongoDB operations \"yourself\" \n(something that should generally be avoided, and should rarely happen - if you encounder a native MongoDB operation that DeriveJS doesn't enable - I would appreciate if you contact me via Github and tell me about it).\nTo get access to the MongoDB collection associated with a data model class, you can call the static method `collection()` of the class, which will return the assosicated [NodeJS MongoDB driver collection object](https://mongodb.github.io/node-mongodb-native/api-generated/collection.html).\n\n### Mixins\n\"Mixins\" are like \"plugins\" adding additional functionality to a Model, and can be used on multiple models. \nTo implement a mixin, you use the `use` static function on a Model class, giving it an object literal with **functions** only. Those\nfunctions will be available to be used by the Model instances.\nA good example will be a \"Logger\" mixin - that adds the functionality to a model - to write \"log\" messages to a \"Logs collection.\nFor example we can create this module:\n\n`Logger.js`\n```javascript\nmodule.exports = ()=\u003e\nnew Promise(async (resolve, reject)=\u003e {\n    let Model = await require('derivejs').Model({\n        dbUrl: \"mongodb://localhost:27017/\",\n        dbName: \"Logs\"\n    });\n\n    let Log = Model({\n        _logMessage: \"\",\n        _date: null\n    },\"Log\");\n\n    resolve({\n        log(msg) { new Log(msg, Date.now()); }\n    });\n});\n```\n\nThen we can implement the `log` function, for example on a `Spaceship` model:\n\n```javascript\nvar Spaceship = Model({\n    _name: \"\",\n    _TYPE: \"\",\n    crew: []\n}, \"Spaceship\");\n\nlet Logger = await require('./Logger')();\nSpaceship.use(Logger);\n\nlet ship = new Spaceship(\"The Logger\").log(\"The Logger ship created\");\n```\n\nNotice that in this example - we create a **separate* connection to the DB, for the Logs collection - \nbut this could be done on the same connection with the rest of the models (however it might be a good practice to separate the connection\nfor things such as logging).\n\n### `collectionReady()` (new in version 1.4.0+)\nWhenever a DB operation related to a collection occurs - if that collection doesn't exist yet in the DB, MongoDB implicitly creates it, this can take time (in the area of ~1 second), \nand thus if you run one of the data getter functions (`get`, `getAll`, `map` etc.) - and the collection was not yet created when reaching that point in your code - you will get an error,\nand the getter function will fail. To prevent this - the `collectionReady` static method was added to Model classes - it returns a promise that resolves when the collection is \ncreated and ready for any operations. If the collection already exists the function will resolve immediately. Use `collectionReady` in situations where it's not certain that a \ncollection exist, and you need to run a getter function on it.\n\n### A note about scaled systems\nWhen having several nodes that needs to access the same collections on a same database - you should **avoid** running Derive on each, as this could lead to unwanted race conditions -- instead, it is recommended to create a single Node that communicates directly with the DB using Derive, then have the separate nodes communicate with it via message queue - that describe the types of data operations needs to be done - this way, allowing to keep the single node to DB architecture, while still having several nodes running concurrently, sending data operations.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuval-a%2Fderiveodm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyuval-a%2Fderiveodm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuval-a%2Fderiveodm/lists"}