{"id":15069880,"url":"https://github.com/boylesoftware/x2node-dbos","last_synced_at":"2025-04-10T17:05:17.127Z","repository":{"id":12296846,"uuid":"71487492","full_name":"boylesoftware/x2node-dbos","owner":"boylesoftware","description":"SQL database operations for X2 Framework.","archived":false,"fork":false,"pushed_at":"2022-02-11T05:07:13.000Z","size":600,"stargazers_count":6,"open_issues_count":5,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-24T14:46:19.023Z","etag":null,"topics":["node-js","node-module","nodejs","nodejs-modules","sql-query","x2node"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/boylesoftware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-10-20T17:30:37.000Z","updated_at":"2020-06-18T20:59:08.000Z","dependencies_parsed_at":"2022-08-07T06:16:53.479Z","dependency_job_id":null,"html_url":"https://github.com/boylesoftware/x2node-dbos","commit_stats":null,"previous_names":[],"tags_count":41,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boylesoftware%2Fx2node-dbos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boylesoftware%2Fx2node-dbos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boylesoftware%2Fx2node-dbos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boylesoftware%2Fx2node-dbos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/boylesoftware","download_url":"https://codeload.github.com/boylesoftware/x2node-dbos/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248260211,"owners_count":21074207,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["node-js","node-module","nodejs","nodejs-modules","sql-query","x2node"],"created_at":"2024-09-25T01:45:19.795Z","updated_at":"2025-04-10T17:05:17.104Z","avatar_url":"https://github.com/boylesoftware.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# X2 Framework for Node.js | Database Operations\n\nThe _Database Operations_ or _DBOs_ module, being a part of _X2 Framework_ for _Node.js_, allows applications to perform complex operations against SQL databases without dealing with SQL queries, parsing result set rows into hierarchical JSON objects and dealing with database and database driver specifics. It allows applications to focus on the high level data structures and business logic instead. Some of the most notable features of the module include:\n\n* Centering on the concept of _records_, which are well defined JSON documents that support complex data structures including nested objects, arrays and maps (nested objects with arbitrary sets of properties). This allows applications to work with the persistent data as easily as it could be with a document-based database while having a full-featured SQL database in the back-end.\n\n* Mapping of _Record Types_ defined using the means of the [x2node-records](https://www.npmjs.com/package/x2node-records) module to the database tables.\n\n* Automatic construction of SQL queries and statements to support the four basic database operations for records and record collections: search/read, create, update and delete.\n\n* Automatic generation of multiple SQL statements to perform complex database operations when necessary, while still reaching for the best efficiency.\n\n* Support for quering records with multiple nested collection properties.\n\n* Support for fetching referred records of other record types all in a single database operation.\n\n* Support for calculated properties and aggregates.\n\n* Support for result set ranges expressed in records, not rows.\n\n* Automatic support for record meta-data such as record versions (which may be useful to support ETags and data modification conflicts detection).\n\n* Respect for the RDBMS features such as data integrity constraints and transactional data locking.\n\nIt's worth noting that the DBOs module is not an ORM and it does not attempt to recreate the concepts pioneered by such _Java_ world frameworks as _Hibernate_. It is not built around the idea of automatic synchronization of state between the application-side objects and the database. It is built around the principles of clarity, efficiency and practicality.\n\nOut of the box, the module supports _MySQL_ (_MariaDB_, _Amazon Aurora_) and _PostgreSQL_ databases. Support for more database engines will be included in the future releases. Custom database driver implementations can be plugged in as well.\n\nSee module's [API Reference Documentation](https://boylesoftware.github.io/x2node-api-reference/module-x2node-dbos.html).\n\n## Table of Contents\n\n* [Usage](#usage)\n * [Record Types](#record-types)\n * [The DBO Factory](#the-dbo-factory)\n * [Fetching Records](#fetching-records)\n * [Creating Records](#creating-records)\n * [Updating Records](#updating-records)\n * [Deleting Records](#deleting-records)\n* [Record Type Definitions](#record-type-definitions)\n * [Mapping Record Types](#mapping-record-types)\n * [Mapping Stored Record Properties](#mapping-stored-record-properties)\n * [Scalar Properties](#scalar-properties)\n * [Collection Properties](#collection-properties)\n * [Dependent Record References](#dependent-record-references)\n * [Shared Link Tables](#shared-link-tables)\n * [Calculated Properties](#calculated-properties)\n * [Aggregate Properties](#aggregate-properties)\n * [Embedded Objects](#embedded-objects)\n * [Polymorphic Objects](#polymorphic-objects)\n * [Ordered Collections](#ordered-collections)\n * [Array Index Column](#array-index-column)\n * [Filtered Collection Views](#filtered-collection-views)\n * [Record Meta-Info Properties](#record-meta-info-properties)\n * [Generated Properties](#generated-properties)\n * [Super-Properties](#super-properties)\n * [Uniqueness of the Id Property](#uniqueness-of-the-id-property)\n* [Fetch DBO](#fetch-dbo)\n * [Selected Properties Specification](#selected-properties-specification)\n * [Filter Specification](#filter-specification)\n * [Logical Junctions](#logical-junctions)\n * [Tests](#tests)\n * [Collection Tests](#collection-tests)\n * [Order Specification](#order-specification)\n * [Range Specification](#range-specification)\n * [Records Locking](#records-locking)\n* [Insert DBO](#insert-dbo)\n* [Update DBO](#update-dbo)\n* [Delete DBO](#delete-dbo)\n* [Transactions](#transactions)\n* [Record Collections Monitors](#record-collections-monitors)\n* [Data Sources](#data-sources)\n* [Database Drivers](#database-drivers)\n* [Record Types Library Descriptors Extensions](#record-types-library-descriptors-extensions)\n\n## Usage\n\nFor the purpose of the DBOs module usage demonstration let's say we have the following self-explanatory schema in a _MySQL_ database:\n\n```sql\nCREATE TABLE accounts (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n fname VARCHAR(30) NOT NULL,\n lname VARCHAR(30) NOT NULL\n);\n\nCREATE TABLE products (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n name VARCHAR(30) NOT NULL UNIQUE,\n price DECIMAL(5,2) NOT NULL\n);\n\nCREATE TABLE orders (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n account_id INTEGER UNSIGNED NOT NULL,\n placed_on TIMESTAMP(3) DEFAULT 0,\n status VARCHAR(10) NOT NULL,\n FOREIGN KEY (account_id) REFERENCES accounts (id)\n);\n\nCREATE TABLE order_items (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n order_id INTEGER UNSIGNED NOT NULL,\n product_id INTEGER UNSIGNED NOT NULL,\n quantity TINYINT UNSIGNED NOT NULL,\n FOREIGN KEY (order_id) REFERENCES orders (id),\n FOREIGN KEY (product_id) REFERENCES products (id),\n UNIQUE (order_id, product_id)\n);\n```\n\nFrom the point of view of the application, this schema provides storage for three types of records: accounts, products and orders (with order line items). The DBOs will work with these three types of records and will allow searching them, getting their data, creating, updating and deleting them.\n\n### Record Types\n\nThe first step is to define the record types library using X2 Framework's [x2node-records](https://www.npmjs.com/package/x2node-records) module and map the records and their properties to the database tables and columns:\n\n```javascript\n// load the framework modules\nconst records = require('x2node-records');\nconst dbos = require('x2node-dbos');\n\n// construct our sample record types library with DBOs extensions\nconst recordTypes = records.with(dbos).buildLibrary({\n recordTypes: {\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'firstName': {\n valueType: 'string',\n column: 'fname'\n },\n 'lastName': {\n valueType: 'string',\n column: 'lname'\n },\n 'orderRefs': {\n valueType: 'ref(Order)[]',\n reverseRefProperty: 'accountRef'\n }\n }\n },\n 'Product': {\n table: 'products',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'name': {\n valueType: 'string'\n },\n 'price': {\n valueType: 'number'\n }\n }\n },\n 'Order': {\n table: 'orders',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'accountRef': {\n valueType: 'ref(Account)',\n column: 'account_id',\n modifiable: false\n },\n 'placedOn': {\n valueType: 'datetime',\n column: 'placed_on',\n modifiable: false\n },\n 'status': {\n valueType: 'string'\n },\n 'items': {\n valueType: 'object[]',\n table: 'order_items',\n parentIdColumn: 'order_id',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'productRef': {\n valueType: 'ref(Product)',\n column: 'product_id',\n modifiable: false\n },\n 'quantity': {\n valueType: 'number'\n }\n }\n }\n }\n }\n }\n});\n```\n\nNote how the DBOs module is added to the library as an extention to allow all the additional record type and property definition attributes used to map the database schema.\n\n### The DBO Factory\n\nThe next step is to constructo the _DBO Factory_, which is used to create the various DBOs:\n\n```javascript\n// create DBO factory against the record types library and the DB driver\nconst dboFactory = dbos.createDBOFactory(recordTypes, 'mysql');\n```\n\nThe DBO factory is provided with the record types library and the database driver, so that it knows how to construct database engine-specific SQL. Out-of-the-box the module supports [mysql](https://www.npmjs.com/package/mysql) (and other compatible implementations) and [pg](https://www.npmjs.com/package/pg). Custom driver implementations can be provided to the DBO factory as well.\n\nThe `createDBOFactory()` function can also take a third argument, `options`, which is an object passed directly to the database driver implementation.\n\nThe built-in _PostgreSQL_ driver does not take any options.\n\nThe built-in _MySQL_ driver, however, can take the following options:\n\n* `orderMode` - There is a difference how some versions of _MariaDB_ (pre version 10.1.35) and _MySQL_ apply sorting to the results of SQL queries. _MySQL_ and newer versions of _MariaDB_ may apply sorting _after_ the result set rows are generated, including any variable calculations. That means that if the row includes a row counter that is incremented for each row, the counter will be calculated _before_ the sorting is applied. This is as opposed to older versions of _MariaDB_ that _always_ apply sorting _before_ row variables are calculated. Because of these differences, the driver may decide to construct the SQL queries differently. By default, the query without a counter but with the `ORDER BY` clause is created and is used as a subquery for the outer query that includes the row counter. This corresponds to _MySQL_ and newer versions of _MariaDB_. If `orderMode` option is \"pre\", no subquery is used and both the row counter and the `ORDER BY` clause are included in a single one-level query. If `orderMode` is \"preIfMariaDB\", the driver detects if it runs against a _MariaDB_ database and automatically uses the \"pre\" mode if so.\n\n* `mariaDB` - This is a deprecated option replaced by `orderMode`. If `true` (Boolean), the \"pre\" mode is forced.\n\n* `databaseCharacterSet` - The database character set. Used, for example, for case conversion in case-insensitive tests. Defaults to \"utf8\".\n\n* `useLocalTimezone` - When `true`, the driver will assume that the database is running in the same timezone as the application. By default, the database is expected to run in UTC timezone.\n\nNormally, a factory is created once by the application when it starts up and is used to construct DBOs throughout the application's lifecycle.\n\nThe DBO factory can be used to construct four types of DBOs:\n\n* _Fetch DBO_ for searching and loading records from the database.\n\n* _Insert DBO_ for creating new records.\n\n* _Update DBO_ for patching existing records.\n\n* _Delete DBO_ for deleting records.\n\nSome most basic usage examples for the four follow.\n\n### Fetching Records\n\nTo search and load _Order_ records from the database described above, the following code could be used:\n\n```javascript\n// build a fetch DBO for selecting 5 recent pending orders for a specific account\nconst fetchDBO = dboFactory.buildFetch('Order', {\n\n // select all order record properties\n props: [ '*' ],\n\n // filter by status and account id passed in as a query parameter when executed\n filter: [\n [ 'status =\u003e is', 'PENDING' ],\n [ 'accountRef =\u003e is', dbos.param('accountId') ]\n ],\n\n // order in descending order by order placement timestamp\n order: [\n 'placedOn =\u003e desc'\n ],\n\n // select first 5 matched order records (that's right, records, not rows!)\n range: [ 0, 5 ]\n});\n\n// configure MySQL database connection\nconst connection = require('mysql').createConnection({\n host: 'localhost',\n database: 'mydatabase',\n user: 'myuser',\n password: 'mypassword'\n});\n\n// connect and do the operations\nconnection.connect(err =\u003e {\n\n // check if errors\n if (err) {\n console.error('connection error');\n throw err;\n }\n\n // execute the fetch DBO on the connection and close it before exiting\n fetchDBO.execute(connection, null, {\n accountId: 10 // for example we want orders for the account id #10\n }).then(\n result =\u003e {\n console.log('result:\\n' + JSON.stringify(result, null, ' '));\n connection.end();\n },\n err =\u003e {\n console.error('error:', err);\n connection.end();\n }\n );\n});\n```\n\nThe `result` object will include a `records` property, which will be an array with all matched order records (up to 5 in our example), and may look something like this:\n\n```json\n{\n \"recordTypeName\": \"Order\",\n \"records\": [\n {\n \"id\": 1,\n \"accountRef\": \"Account#10\",\n \"placedOn\": \"2017-02-20T18:32:55.000Z\",\n \"status\": \"PENDING\",\n \"items\": [\n {\n \"id\": 101,\n \"productRef\": \"Product#1\",\n \"quantity\": 1\n },\n {\n \"id\": 102,\n \"productRef\": \"Product#2\",\n \"quantity\": 10\n }\n ]\n },\n {\n \"id\": 2,\n ...\n },\n ...\n ]\n}\n```\n\nDon't mind the `null` passed into the DBO's `execute()` method as the second argument for now. It is for the actor performing the operation (anonymous in this case) and it is explained later.\n\nWe could request to fetch only specific record properties and also include referred records such as the _Products_ and the _Accounts_:\n\n```javascript\nconst fetchDBO = dboFactory.buildFetch('Order', {\n\n // select only specific properties\n props: [\n '.count', // include total count of matched orders\n 'placedOn', // only placedOn from the order\n 'items.quantity', // plus quantity for the items\n 'items.productRef.*', // plus product and all product properties\n 'accountRef.firstName', // plus account first name\n 'accountRef.lastName' // and the last name\n ],\n ...\n});\n```\n\nThe result then would look something like the following:\n\n```json\n{\n \"recordTypeName\": \"Order\",\n \"count\": 12,\n \"records\": [\n {\n \"id\": 1,\n \"accountRef\": \"Account#10\",\n \"placedOn\": \"2017-02-20T18:32:55.000Z\",\n \"items\": [\n {\n \"productRef\": \"Product#1\",\n \"quantity\": 1\n },\n {\n \"productRef\": \"Product#2\",\n \"quantity\": 10\n }\n ]\n },\n {\n \"id\": 2,\n ...\n },\n ...\n ],\n \"referredRecords\": {\n \"Account#10\": {\n \"firstName\": \"John\",\n \"lastName\": \"Silver\"\n },\n \"Product#1\": {\n \"id\": 1,\n \"name\": \"Rope\",\n \"price\": 9.99\n },\n \"Product#2\": {\n \"id\": 2,\n \"name\": \"Nails\",\n \"price\": 4.5\n }\n }\n}\n```\n\nNote that the record id is always included even though not explicitely selected. Also, intermediate properties in the selected property paths are included.\n\nThe `count` is a so called _super-aggregate_, which is automatically added to the records types library. The detailed description of super-aggregates is provided later in this manual. For now, it shows the total number of matched records regardless of the query range, which can be convenient on the client side to drive the search results pagination.\n\n### Creating Records\n\nTo create a new order record we could use the following code:\n\n```javascript\n// create the DBO and pass new order record template to it\nconst insertDBO = dboFactory.buildInsert('Order', {\n accountRef: 'Account#10',\n placedOn: new Date(),\n status: 'PENDING',\n items: [\n {\n productRef: 'Product#1',\n quantity: 1\n },\n {\n productRef: 'Product#2',\n quantity: 10\n }\n ]\n});\n\n// execute the DBO\ninsertDBO.execute(connection, null).then(\n recordId =\u003e {\n console.log('new Order id: ' + recordId);\n },\n err =\u003e {\n console.error('error:', err);\n }\n);\n```\n\nIn our example the record ids are auto-generated in the database, so we do not include them in the record template. The promise returned by the insert DBO resolves to the new record id.\n\n### Updating Records\n\nUpdating records involves special objects called _patches_, which are specifications of what needs to be changed in the matched records. Let's say we want to make the following changes to an order with a specific id:\n\n1. Update quantity on the first order line item.\n\n2. Add a new line item to the order.\n\n3. Change the order status.\n\nThe following code can be used to do that:\n\n```javascript\n// build the patch specification\nconst patches = require('x2node-patches');\nconst patch = patches.build(recordTypes, 'Order', [\n { op: 'replace', path: '/items/0/quantity', value: 2 },\n { op: 'add', path: '/items/-', value: {\n productRef: 'Product#3',\n quantity: 1\n } },\n { op: 'replace', path: '/status', value: 'PROCESSING' }\n]);\n\n// create the DBO passing the patch and the order record selector filter\nconst updateDBO = dboFactory.buildUpdate('Order', patch, [\n [ 'id =\u003e is', dbos.param('orderId') ]\n]);\n\n// execute the DBO\nupdateDBO.execute(connection, null, null, {\n orderId: 1\n}).then(\n result =\u003e {\n console.log('update operation status:\\n' + JSON.stringify(result, null, ' '));\n },\n err =\u003e {\n console.error('error:', err);\n }\n);\n```\n\nThe example above uses X2 Framework's [x2node-patches](https://www.npmjs.com/package/x2node-patches) module to build the patch specification using [JSON Patch](https://tools.ietf.org/html/rfc6902) notation.\n\nThe third `null` argument to the DBO's `execute()` method is for an optional record validation function that allows validating records after the patch is applied but before the record is saved into the database. This functionality is described later in this manual.\n\nThe `result` object returned by the operation will contain information about what records were updated as well as the updated records data itself. This is also described in detail later.\n\n### Deleting Records\n\nDeleting an order record could be done like this:\n\n```javascript\n// build the DBO for the specific order id\nconst deleteDBO = dboFactory.buildDelete('Order', [\n [ 'id =\u003e is', dbos.param('orderId') ]\n]);\n\n// execute the DBO and pass the order id to it\ndeleteDBO.execute(connection, null, {\n orderId: 1\n}).then(\n result =\u003e {\n console.log('delete operation status:\\n' + JSON.stringify(result, null, ' '));\n },\n err =\u003e {\n console.error('error:', err);\n }\n);\n```\n\nThe `result` object will tell if any records were matched and actually deleted.\n\n### Debug Logging\n\nSometimes it is useful to see what SQL statements are run against the database. To turn on the DBOs module's debug logging, which includes that information, the `NODE_DEBUG` environment variable must include word \"X2_DBO\" (see [Node.js API docs](https://nodejs.org/docs/latest-v4.x/api/util.html#util_util_debuglog_section) for details).\n\n## Record Type Definitions\n\nThe DBOs module introduces a number of attributes used in record types library definitions to map records and their properties to the database tables and columns. This mapping allows the DBO factory to construct the SQL queries. The DBOs module itself is a record types library extension and must be added to the library for the extended attributes to get processed:\n\n```javascript\nconst records = require('x2node-records');\nconst dbos = require('x2node-dbos');\n\nconst recordTypes = records.with(dbos).buildLibrary({\n ...\n});\n```\n\n### Mapping Record Types\n\nEvery record type must have the main table, to which it is mapped. This is the table that has the record id as its primary key and normally stores the bulk of the scalar record properties as its columns. To associate a record type with a table, a `table` attribute is used in the definition:\n\n```javascript\nconst recordTypes = records.with(dbos).buildLibrary({\n recordTypes: {\n ...\n 'Order': {\n table: 'orders',\n properties: {\n ...\n }\n },\n ...\n }\n});\n```\n\nIf `table` attribute is not specified, the record type name is used.\n\n### Mapping Stored Record Properties\n\nThe term _stored property_ is used to describe record properties that belong to the record type, are stored together with the records of the type. Later we will see that not all properties defined on a record type are stored properties. For example, some properties may be calculated on the fly when records are fetched from the database and therefore are not stored. Other properties may be stored but belong to a different record type and are imported into another record type. Views supported by the basic [x2node-records](https://www.npmjs.com/package/x2node-records) module are another example of properties that are not stored.\n\nA number of attributes specific to the DBOs module is used to map stored record properties to the tables and columns.\n\n#### Scalar Properties\n\nThe simples case of a record property is a scalar property stored in the record type's main table column. To associate such property with the specific column, a `column` property definition attribute is used:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n ...\n 'firstName': {\n valueType: 'string',\n column: 'fname'\n },\n 'zip': {\n valueType: 'string',\n optional: true\n }\n }\n },\n ...\n}\n```\n\nThis will map the `firstName` property to the `fname` column of the `accounts` table and the `zip` property to its `zip` column (if `column` attribute is not specified, the property name is used):\n\n```sql\nCREATE TABLE accounts (\n ...\n fname VARCHAR(30) NOT NULL,\n zip CHAR(5),\n ...\n);\n```\n\nThe column data type must match the property value type. Also, the column may be nullable if the property is optional.\n\nA property does not have to be stored in the main record type table. If a property is stored in a separate table, the property definition must have a `table` attribute and a `parentIdColumn` attribute, which specifies the column in the property table that points back at the parent record id. Normally it does not make much sense to store scalar, simple value properties in a separate table, but it will work nonetheless. It makes more sense when it comes to nested object properties. For example, if an account address is stored in a separate table in a one-to-one relation with the main record table:\n\n```sql\nCREATE TABLE accounts (\n id INTEGER UNSIGNED PRIMARY KEY,\n ...\n);\n\nCREATE TABLE account_addresses (\n account_id INTEGER UNSIGNED NOT NULL UNIQUE,\n street VARCHAR(50) NOT NULL,\n ...\n FOREIGN KEY (account_id) REFERENCES accounts (id)\n);\n```\n\nthen it can be mapped like this:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n ...\n 'address': {\n valueType: 'object',\n table: 'account_addresses',\n parentIdColumn: 'account_id',\n properties: {\n 'street': {\n valueType: 'string'\n }\n ...\n }\n }\n }\n },\n ...\n}\n```\n\n#### Collection Properties\n\nWhen it comes to array and map properties, use of property tables becomes necessary due to the RDBMS nature. The example above can be modified to allow multiple addresses on an account:\n\n```sql\nCREATE TABLE account_addresses (\n id INTEGER UNSIGNED PRIMARY KEY,\n account_id INTEGER UNSIGNED NOT NULL,\n street VARCHAR(50) NOT NULL,\n ...\n FOREIGN KEY (account_id) REFERENCES accounts (id)\n);\n```\n\nand the record type definition:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n ...\n 'addresses': {\n valueType: 'object[]',\n table: 'account_addresses',\n parentIdColumn: 'account_id',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'street': {\n valueType: 'string'\n }\n ...\n }\n }\n }\n },\n ...\n}\n```\n\nIf an address has a type (\"Home\", \"Work\", etc.):\n\n```sql\nCREATE TABLE account_addresses (\n id INTEGER UNSIGNED PRIMARY KEY,\n account_id INTEGER UNSIGNED NOT NULL,\n type VARCHAR(10) NOT NULL,\n street VARCHAR(50) NOT NULL,\n ...\n FOREIGN KEY (account_id) REFERENCES accounts (id),\n UNIQUE (account_id, type)\n);\n```\n\nthen we could have the addresses on the account as a map:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n ...\n 'addresses': {\n valueType: 'object{}',\n keyPropertyName: 'type',\n table: 'account_addresses',\n parentIdColumn: 'account_id',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'type': {\n valueType: 'string'\n },\n 'street': {\n valueType: 'string'\n }\n ...\n }\n }\n }\n },\n ...\n}\n```\n\nIf we don't want to have the address type as a property on the address record, then the key column can be specified using `keyColumn` property definition attribute instead of using `keyPropertyName`:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n ...\n 'addresses': {\n valueType: 'object{}',\n table: 'account_addresses',\n parentIdColumn: 'account_id',\n keyColumn: 'type',\n keyValueType: 'string',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'street': {\n valueType: 'string'\n }\n ...\n }\n }\n }\n },\n ...\n}\n```\n\nNote, that in that case the `keyValueType` attribute must be provided as well (see [x2node-rsparser](https://www.npmjs.com/package/x2node-rsparser) module for details).\n\nSimple value collection properties utilize the `column` attribute to map the column in the collection table. For example, if we have a list of phone numbers associated with the account:\n\n```sql\nCREATE TABLE account_phones (\n account_id INTEGER UNSIGNED NOT NULL,\n phone CHAR(10) NOT NULL,\n FOREIGN KEY (account_id) REFERENCES accounts (id)\n);\n```\n\nthe definition will be:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n ...\n 'phones': {\n valueType: 'string[]',\n table: 'account_phones',\n parentIdColumn: 'account_id',\n column: 'phone'\n }\n }\n },\n ...\n}\n```\n\nAnd if we have a phone type as a map key:\n\n```sql\nCREATE TABLE account_phones (\n account_id INTEGER UNSIGNED NOT NULL,\n type VARCHAR(10) NOT NULL,\n phone CHAR(10) NOT NULL,\n FOREIGN KEY (account_id) REFERENCES accounts (id),\n UNIQUE (account_id, type)\n);\n```\n\nthe definition will be:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n ...\n 'phones': {\n valueType: 'string{}',\n table: 'account_phones',\n parentIdColumn: 'account_id',\n keyColumn: 'type',\n keyValueType: 'string',\n column: 'phone'\n }\n }\n },\n ...\n}\n```\n\nNote that use of `keyColumn` attribute becomes the only option for simple value map properties (no `keyPropertyName` is appropriate).\n\nThere is an important deviation between how the `keyColumn` and `keyPropertyName` attributes work when it comes to reference property maps. Such maps (as well as arrays) introduce a link table between the main tables of the referrer and the referred record types. When the `keyPropertyName` is used to map the map key, the key is located in the referred record type table. For example, if we extract the address records in the examples above into a separate record type, the tables will be:\n\n```sql\nCREATE TABLE accounts (\n id INTEGER UNSIGNED PRIMARY KEY,\n ...\n);\n\nCREATE TABLE account_addresses (\n account_id INTEGER UNSIGNED NOT NULL,\n address_id INTEGER UNSIGNED NOT NULL,\n FOREIGN KEY (account_id) REFERENCES accounts (id),\n FOREIGN KEY (address_id) REFERENCES addresses (id),\n UNIQUE (account_id, address_id)\n);\n\nCREATE TABLE addresses (\n id INTEGER UNSIGNED PRIMARY KEY,\n type VARCHAR(10) NOT NULL,\n street VARCHAR(50) NOT NULL,\n ...\n);\n```\n\nand the definitions:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n ...\n 'addressRefs': {\n valueType: 'ref(Address){}',\n keyPropertyName: 'type',\n table: 'account_addresses',\n parentIdColumn: 'account_id',\n column: 'address_id'\n }\n }\n },\n 'Address': {\n table: 'addresses',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'type': {\n valueType: 'string'\n },\n 'street': {\n valueType: 'string'\n },\n ...\n }\n },\n ...\n}\n```\n\nSo, the `type` property for the map key is taken from the `addresses` table. On the other hand, if `keyColumn` attribute is used, the column is mapped to the link table and _not_ the referred record type table:\n\n```sql\nCREATE TABLE accounts (\n id INTEGER UNSIGNED PRIMARY KEY,\n ...\n);\n\nCREATE TABLE account_addresses (\n account_id INTEGER UNSIGNED NOT NULL,\n type VARCHAR(10) NOT NULL,\n address_id INTEGER UNSIGNED NOT NULL,\n FOREIGN KEY (account_id) REFERENCES accounts (id),\n FOREIGN KEY (address_id) REFERENCES addresses (id),\n UNIQUE (account_id, type)\n);\n\nCREATE TABLE addresses (\n id INTEGER UNSIGNED PRIMARY KEY,\n street VARCHAR(50) NOT NULL,\n ...\n);\n```\n\nand the definitions:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n ...\n 'addressRefs': {\n valueType: 'ref(Address){}',\n table: 'account_addresses',\n parentIdColumn: 'account_id',\n keyColumn: 'type',\n keyValueType: 'string',\n column: 'address_id'\n }\n }\n },\n 'Address': {\n table: 'addresses',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'street': {\n valueType: 'string'\n },\n ...\n }\n },\n ...\n}\n```\n\n### Dependent Record References\n\nSome reference properties are not stored with the referring record. Instead, there is a reference property in the referred record type that points back at the referring record. The referred record type in this case is called a _dependent_ record type, because its records can exist only in the context of the referring record (unless the reverse reference property is optional), meaning the referring record must exist for the referred record to point at it.\n\nFor example, if we have records types _Account_ and _Order_ and the account has a list of references to the orders made for that account. The _Order_, even though a separate record type, has a reference back to the account it belongs to and can only exist in a context of an account. That makes the _Order_ a dependent record type.\n\nHere are the tables:\n\n```sql\nCREATE TABLE accounts (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n ...\n);\n\nCREATE TABLE orders (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n account_id INTEGER UNSIGNED NOT NULL,\n ...\n FOREIGN KEY (account_id) REFERENCES accounts (id)\n);\n```\n\nand the definitions:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n ...\n 'orderRefs': {\n valueType: 'ref(Order)[]',\n reverseRefProperty: 'accountRef'\n }\n }\n },\n 'Order': {\n table: 'orders',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'accountRef': {\n valueType: 'ref(Account)',\n column: 'account_id',\n modifiable: false\n },\n ...\n }\n },\n ...\n}\n```\n\nThe `orderRefs` property of the _Account_ record type is not stored anywhere in any tables and columns associated with it, so it is not a stored property of the _Account_ record type. Instead, it is a property of the dependent _Order_ record type and is stored with the order records in `accountRef` property. This is why the `orderRefs` property does not have any `table` and `column` definition attributes. Instead, it has a `reverseRefProperty` attribute that names the property in the referred record type that points back at the referring record.\n\nThe dependent record reference properties are only allowed as top record type properties (not in nested objects). And the reverse reference properties in the referred record types are also only allowed to be top record type properties, plus they are required to be scalar, non-polymorph and not be dependent record references themselves. They are, however, allowed to be stored in a separate table (this may be useful when `keyColumn` is used with a dependent record reference map property, in which case the map key column will reside in the separate link table).\n\nAs it will be shown later in this manual, normally, when a record with dependent record reference properties is deleted, the operation is automatically cascaded on to the dependent records, which are deleted as well. This is called _strong dependencies_. In the example above, it makes sense to delete all _Order_ records associated with an _Account_ when the _Account_ record is deleted from the database. Sometimes, however, a dependent reference mechanism needs to be used to import a reference property to a record (for data access convenience), but the referred record type is not exactly dependent. This is called a _weak dependence_. One side effect of a weak dependence is that the deletion operation is not cascaded over it (the operation will fail on the database level if referred records exist and foreign key constraints are properly utilized). To make a dependent reference property weak, a `weakDependency` property definition attribute can be added with value `true`.\n\n### Shared Link Tables\n\nSometimes, two record types may have references at each other, but none of them is dependent on another. For example, if we have record types _Account_ and _Address_ and they are in a many-to-many relationship so that a single address can be shared by multiple accounts and an account can have multiple addresses:\n\n```sql\nCREATE TABLE accounts (\n id INTEGER UNSIGNED PRIMARY KEY,\n ...\n);\n\nCREATE TABLE accounts_addresses (\n account_id INTEGER UNSIGNED NOT NULL,\n address_id INTEGER UNSIGNED NOT NULL,\n FOREIGN KEY (account_id) REFERENCES accounts (id),\n FOREIGN KEY (address_id) REFERENCES addresses (id),\n UNIQUE (account_id, address_id)\n);\n\nCREATE TABLE addresses (\n id INTEGER UNSIGNED PRIMARY KEY,\n ...\n);\n```\n\nAt the same time, we want to have a list of addresses on the account records and we want to have a list of accounts that use an address on the address record. It can be done like this:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n ...\n 'addressRefs': {\n valueType: 'ref(Address)[]',\n table: 'accounts_addresses',\n parentIdColumn: 'account_id',\n column: 'address_id'\n }\n }\n },\n 'Address': {\n table: 'addresses',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n ...\n 'accountRefs': {\n valueType: 'ref(Account)[]',\n table: 'accounts_addresses',\n parentIdColumn: 'address_id',\n column: 'account_id'\n }\n }\n },\n ...\n}\n```\n\nProperties `addressRefs` on record type _Account_ and `accountRefs` on record type _Address_ share the same link table `accounts_addresses`. This case is unique in the sense that modification of a record of one record type may have a side-effect of making changes to stored properties of some records of another record type. Such record types and the properties that reference each other are known as _entangled_.\n\n### Calculated Properties\n\nIt is possible to have properties on a record type that are not stored in the database directly, but are calculated every time a record is fetched from the database. Such properties are called _calculated properties_. Instead of a column and table, a calculated property has a value expression associated with it using `valueExpr` property definition attribute. The `valueExpr` attribute is a string that uses a special expression syntax that includes value literals, references to other properties of the record, value transformation functions and operators. Here is the expression language EBNF definition:\n\n```ebnf\nExpr = [ \"+\" | \"-\" ] , Term , { ( \"+\" | \"-\" ) , Term }\n | String\n | Boolean\n ;\n\nTerm = Factor , { ( \"*\" | \"/\" ) , Factor }\n ;\n\nFactor = FunctionCall\n | PropertyRef\n | Number\n | \"(\" , Expr , \")\"\n ;\n\nFunctionCall = FunctionName , \"(\" , Expr , { \",\" , Expr } , \")\"\n ;\n\nString = '\"' , { CHAR } , '\"' | \"'\" , { CHAR } , \"'\" ;\nBoolean = \"true\" | \"false\" ;\nNumber = DIGIT , { DIGIT } , [ \".\" , DIGIT , { DIGIT } ] ;\nFunctionName = ? See Description ? ;\nPropertyRef = ? See Description ? ;\n```\n\nThe following value transformation functions are supported:\n\n* `length`, `len` - Takes single string argument, yields string length.\n\n* `lower`, `lc`, `lcase`, `lowercase` - Takes single string argument, yields all lower-case string.\n\n* `upper`, `uc`, `ucase`, `uppercase` - Takes single string argument, yields all upper-case string.\n\n* `substring`, `sub`, `mid`, `substr` - Yields substring of the string provided as the first argument. The second argument is the first substring character index, starting from zero. The optional third argument is the maximum substring length. If not provided, the end of the input string is used.\n\n* `lpad` - Pads the string provided as the first argument on the left to achieve the minimum length provided as the second argument using the character provided as the third argument as the padding character.\n\n* `concat`, `cat` - Concatenate provided string arguments.\n\n* `coalesce` - Yield first non-null argument.\n\nThe properties in the expressions are referred using dot notation. In a calculated nested object property, to refer to a property in the parent object a caret character is used to denote the immediate parent.\n\nFor example, given the record types library used in the opening [Usage](#usage) section, the _Order_ records could be augmented with some calculated properties:\n\n```javascript\n{\n ...\n 'Order': {\n ...\n properties: {\n ...\n 'customerName': {\n valueType: 'string',\n valueExpr: 'concat(accountRef.firstName, \" \", accountRef.lastName)'\n },\n ...\n 'items': {\n ...\n properties: {\n ...\n 'totalCost': {\n valueType: 'number',\n valueExpr: 'productRef.price * quantity'\n },\n 'orderStatus': {\n valueType: 'string',\n valueExpr: '^.status'\n }\n }\n }\n }\n },\n ...\n}\n```\n\nThis example shows that reference properties can be hopped over to access properties of the referred records. Also, if there were more nesting levels, jumping over multiple children (`prop.nestedProp.moreNestedProp`) and multiple parents (`^.^.grandparentProp.nestedProp`) is possible with the dot notation.\n\nWhen the records with calculated properties are fetched from the database, the expressions are converted into SQL value expressions and are calculated on the database side.\n\n### Aggregate Properties\n\nA special case of a calculated property is an _aggregate property_. An aggregate property requires:\n\n* A collection property, elements of which it aggregates. The collection is called _aggregated collection_.\n* An expression for the aggregated values, called _aggregated value expression_.\n\n* The _aggregation function_, which determines how the values are aggregated.\n\n* And optionally a filter to include only certain elements of the aggregated collection.\n\nAn `aggregate` property definition attribute is used to specify these elements. For example, our _Order_ records could include properties that aggregate the order line items like the following:\n\n```javascript\n{\n ...\n 'Order': {\n table: 'orders',\n properties: {\n ...\n 'itemsCount': {\n valueType: 'number',\n aggregate: {\n collection: 'items',\n valueExpr: 'id =\u003e count'\n }\n },\n 'orderTotal': {\n valueType: 'number',\n aggregate: {\n collection: 'items',\n valueExpr: 'productRef.price * quantity =\u003e sum'\n }\n },\n 'expensiveProductsCount': {\n valueType: 'number',\n aggregate: {\n collection: 'items',\n valueExpr: 'quantity =\u003e sum',\n filter: [\n [ 'productRef.price =\u003e min', 1000 ]\n ]\n }\n },\n ...\n 'items': {\n valueType: 'object[]',\n table: 'order_items',\n parentIdColumn: 'order_id',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'productRef': {\n valueType: 'ref(Product)',\n column: 'product_id',\n modifiable: false\n },\n 'quantity': {\n valueType: 'number'\n }\n }\n }\n }\n },\n ...\n}\n```\n\nThe value expressions for the aggregate properties have format:\n\n```\nExpr =\u003e AggregateFunc\n```\n\nWhere `Expr` is a regular value expression calculated in the context of the aggregated collection property, and `AggregateFunc` is one of:\n\n* `count` - Number of unique values yielded by the aggregated expression.\n\n* `sum` - Sum of the values yielded by the aggregated expression.\n\n* `min` - The smallest value yielded by the aggregated expression.\n\n* `max` - The largest value yielded by the aggregated expression.\n\n* `avg` - Average of the values yielded by the aggregated expression.\n\nThe optional filter specification format will be discussed later in this manual when we talk about fetch DBO.\n\nAn aggregate property can also be a map, in which case the aggregation is further subdivided and grouped by the map key. To specify the map key use `keyPropertyName` attribute in the aggregate property definition.\n\n### Embedded Objects\n\nProperties of a scalar nested object do not have to be stored in a separate table. The nested object's properties can be stored in the columns of the main record type table and the nested object can be used simply to organize the JSON structure of the record. For example, an address on an _Account_ record could be stored in the same table:\n\n```sql\nCREATE TABLE accounts (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n fname VARCHAR(30) NOT NULL,\n lname VARCHAR(30) NOT NULL,\n street VARCHAR(50) NOT NULL,\n city VARCHAR(50) NOT NULL,\n state CHAR(2) NOT NULL,\n zip CHAR(5) NOT NULL\n);\n```\n\nbut be a nested object in the record type:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'firstName': {\n valueType: 'string',\n column: 'fname'\n },\n 'lastName': {\n valueType: 'string',\n column: 'lname'\n },\n 'address': {\n valueType: 'object',\n properties: {\n 'street': {\n valueType: 'string'\n },\n 'city': {\n valueType: 'string'\n },\n 'state': {\n valueType: 'string'\n },\n 'zip': {\n valueType: 'string'\n }\n }\n }\n }\n },\n ...\n}\n```\n\nSince the `address` property does not have a `table` attribute, the framework knows that it is stored in the parent table.\n\nIn the example above, the `address` property is required. Things turn slightly more complicated if the property needs to be optional. First, the `NOT NULL` constraints are removed from the table:\n\n```sql\nCREATE TABLE accounts (\n ...\n street VARCHAR(50),\n city VARCHAR(50),\n state CHAR(2),\n zip CHAR(5)\n);\n```\n\nBut now, the framework needs to be able to tell if the `address` nested object is present on a record it's loading from the database or not. To do that, the `presenceTest` attribute is specified on the property definition. The presence test is a Boolean expression written as a filter specification, which is discussed in detail later in this manual. But here is our example:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n ...\n 'address': {\n valueType: 'object',\n optional: true, // address is now optional\n presenceTest: [\n [ 'state =\u003e present' ],\n [ 'zip =\u003e present' ]\n ],\n properties: {\n 'street': {\n valueType: 'string',\n optional: true\n },\n 'city': {\n valueType: 'string',\n optional: true\n },\n 'state': {\n valueType: 'string'\n },\n 'zip': {\n valueType: 'string'\n }\n }\n }\n }\n },\n ...\n}\n```\n\nThis definition makes properties `street` and `city` optional, but `state` and `zip` are still required. The presence test checks if `state` and `zip` are present on the record, and if they are not, the whole `address` property is considered to be absent.\n\n### Polymorphic Objects\n\nPolymorphic objects are treated very similarly to nested objects where each polymorphic object subtype is like a nested object property on the base polymorphic object with the properties that describe the subtype-specific properties. The table mappings work the same as for scalar nested objects. For example, if we have two types of payment information that can be associated with an account\u0026mdash;credit cards and bank accounts\u0026mdash;the tables could be:\n\n```sql\nCREATE TABLE accounts (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n fname VARCHAR(30) NOT NULL,\n lname VARCHAR(30) NOT NULL\n);\n\nCREATE TABLE account_creditcards (\n account_id INTEGER UNSIGNED NOT NULL UNIQUE,\n association VARCHAR(20) NOT NULL,\n last4digits CHAR(4) NOT NULL,\n expdate CHAR(4) NOT NULL,\n FOREIGN KEY (account_id) REFERENCES accounts (id)\n);\n\nCREATE TABLE account_bankaccounts (\n account_id INTEGER UNSIGNED NOT NULL UNIQUE,\n routing_num CHAR(9) NOT NULL,\n account_type VARCHAR(10) NOT NULL,\n last4digits CHAR(4) NOT NULL,\n FOREIGN KEY (account_id) REFERENCES accounts (id)\n);\n```\n\nthen the record type definition would be:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n ...\n 'paymentInfo': {\n valueType: 'object',\n typePropertyName: 'type',\n subtypes: {\n 'CREDIT_CARD': {\n table: 'account_creditcards',\n parentIdColumn: 'account_id',\n properties: {\n 'association': {\n valueType: 'string'\n },\n 'last4Digits': {\n valueType: 'string',\n column: 'last4digits'\n },\n 'expirationDate': {\n valueType: 'string',\n column: 'expdate'\n }\n }\n },\n 'BANK_ACCOUNT': {\n table: 'account_bankaccounts',\n parentIdColumn: 'account_id',\n properties: {\n 'routingNumber': {\n valueType: 'string',\n column: 'routing_num'\n },\n 'accountType': {\n valueType: 'string',\n column: 'account_type'\n },\n 'last4Digits': {\n valueType: 'string',\n column: 'last4digits'\n }\n }\n }\n }\n }\n }\n },\n ...\n}\n```\n\nProperties common to all subtypes could be stored either in the parent table (in the example above the `last4digits` column is moved to the `accounts` table and the `last4Digits` property is moved to the `properties` section of the `paymentInfo` property definition), or they could be stored in a separate table:\n\n```sql\nCREATE TABLE accounts (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n fname VARCHAR(30) NOT NULL,\n lname VARCHAR(30) NOT NULL\n);\n\nCREATE TABLE account_paymentinfos (\n account_id INTEGER UNSIGNED NOT NULL UNIQUE,\n last4digits CHAR(4) NOT NULL,\n FOREIGN KEY (account_id) REFERENCES accounts (id)\n);\n\nCREATE TABLE account_creditcards (\n account_id INTEGER UNSIGNED NOT NULL UNIQUE,\n association VARCHAR(20) NOT NULL,\n expdate CHAR(4) NOT NULL,\n FOREIGN KEY (account_id) REFERENCES account_paymentinfos (id)\n);\n\nCREATE TABLE account_bankaccounts (\n account_id INTEGER UNSIGNED NOT NULL UNIQUE,\n routing_num CHAR(9) NOT NULL,\n account_type VARCHAR(10) NOT NULL,\n FOREIGN KEY (account_id) REFERENCES account_paymentinfos (id)\n);\n```\n\nand the definitions:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n ...\n 'paymentInfo': {\n valueType: 'object',\n typePropertyName: 'type',\n table: 'account_paymentinfos',\n parentIdColumn: 'account_id',\n properties: {\n 'last4Digits': {\n valueType: 'string',\n column: 'last4digits'\n }\n },\n subtypes: {\n 'CREDIT_CARD': {\n table: 'account_creditcards',\n parentIdColumn: 'account_id',\n properties: {\n 'association': {\n valueType: 'string'\n },\n 'expirationDate': {\n valueType: 'string',\n column: 'expdate'\n }\n }\n },\n 'BANK_ACCOUNT': {\n table: 'account_bankaccounts',\n parentIdColumn: 'account_id',\n properties: {\n 'routingNumber': {\n valueType: 'string',\n column: 'routing_num'\n },\n 'accountType': {\n valueType: 'string',\n column: 'account_type'\n }\n }\n }\n }\n }\n }\n },\n ...\n}\n```\n\nWhen the framework builds queries for fetching account records in the examples above, it will join all the subtype tables and see which one has a record. Depending on that, it will figure out the nested object subtype. That logic relies on the database having a single record in only a single subtype table for a given base polymorphic record. In the second example, it is enforced by having a `UNIQUE` constraint on the `account_id` column of the `account_paymentinfos` table. In the first example, however, it is not completely enforced on the database level\u0026mdash;physically it is possible to have a row in both `account_creditcards` and `account_bankaccounts` tables each sharing the same account id.\n\nSometimes, a subtype does not have any subtype-specific properties and therefore there is no need for a separate table for it. In that case, the framework cannot determine the subtype by joining the tables and simply checking which one has a record. To solve it, the type property needs to be actually stored in a column on the base polymorphic object's table. For example, if we have a polymorphic _Event_ record type where subtype _OPENED_ has a subtype-specific property `openerName`, which, for the example sake, is stored in a separate table, and subtype _CLOSED_ does not have any subtype-specific properties. The tables are:\n\n```sql\nCREATE TABLE events (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n type VARCHAR(10) NOT NULL,\n happened_on TIMESTAMP\n);\n\nCREATE TABLE events_open (\n id INTEGER UNSIGNED NOT NULL UNIQUE,\n opener_name VARCHAR(50) NOT NULL,\n FOREIGN KEY (id) REFERENCES events (id)\n);\n```\n\nThe `type` column of the `events` table is used to store the event type (either \"OPENED\" or \"CLOSED\"). The definition should then be:\n\n```javascript\n{\n ...\n 'Event': {\n table: 'events',\n typePropertyName: 'type',\n typeColumn: 'type',\n properties: {\n 'happenedOn': {\n valueType: 'datetime',\n column: 'happened_on'\n }\n },\n subtypes: {\n 'OPENED': {\n table: 'events_open',\n parentIdColumn: 'id',\n properties: {\n 'openerName': {\n valueType: 'string',\n column: 'opener_name'\n }\n }\n },\n 'CLOSED': {\n properties: {}\n }\n }\n },\n ...\n}\n```\n\nNote the `typeColumn` attribute that tells the framework where to find/store the record subtype.\n\n### Ordered Collections\n\nWhen collection properties are fetched from the database, it is possible to specify the order in which they are returned. For that, any collection property definition can have an `order` attribute. For example:\n\n```javascript\n{\n ...\n 'Order': {\n table: 'orders',\n properties: {\n ...\n 'items': {\n valueType: 'object[]',\n table: 'order_items',\n parentIdColumn: 'order_id',\n order: [ 'productRef.name', 'quantity =\u003e desc' ],\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'productRef': {\n valueType: 'ref(Product)',\n column: 'product_id',\n modifiable: false\n },\n 'quantity': {\n valueType: 'number'\n }\n }\n }\n }\n },\n ...\n}\n```\n\nOr, another example:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n ...\n 'orderRefs': {\n valueType: 'ref(Order)[]',\n reverseRefProperty: 'accountRef',\n order: [ 'placedOn =\u003e desc' ]\n }\n }\n },\n ...\n}\n```\n\nThe order specification is described in detail later in this manual when we talk about the fetch DBO.\n\nNote, that to order a simple value collection by the value, `$value` pseudo-property reference can be used.\n\n### Array Index Column\n\nUnless an `order` attribute is specified on an array property, the order, in which the elements will be returned in the fetch DBO result, is undetermined. It is possible, however, to make an array property strictly ordered by the insertion order. To do that, the table used to store the array elements can have a special element index column maintained automatically by the framework. For example if we want to have tags on _Product_ records and we want the tags list to have specific order, we add a tag index column to the table:\n\n```sql\nCREATE TABLE products (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n ...\n);\n\nCREATE TABLE product_tags (\n product_id INTEGER UNSIGNED NOT NULL,\n ind TINYINT UNSIGNED NOT NULL,\n tag VARCHAR(20) NOT NULL,\n FOREIGN KEY (product_id) REFERENCES products (id),\n UNIQUE (product_id, ind)\n);\n```\n\nTo let the framework manage the index column, `indexColumn` attribute is added to the array property definition:\n\n```javascript\n{\n ...\n 'Product': {\n table: 'accounts',\n properties: {\n ...\n 'tags': {\n valueType: 'string[]',\n table: 'product_tags',\n parentIdColumn: 'product_id',\n indexColumn: 'ind',\n column: 'tag'\n }\n }\n },\n ...\n}\n```\n\nWhenever the DBOs are used to create and modify the `tags` array, the framework will be generating additional SQL to maintain the index values in sync with the array element indexes. And when a fetch DBO is used to fetch _Product_ records, the `tags` property will always be ordered by the index column.\n\nThe same works with nested object arrays as well.\n\n### Filtered Collection Views\n\nIt is also possible to specify filtered views of collection properties that include only those elements that match a certain criteria. For example:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n ...\n 'orderRefs': {\n valueType: 'ref(Order)[]',\n reverseRefProperty: 'accountRef'\n },\n 'pendingOrderRefs': {\n viewOf: 'orderRefs',\n filter: [\n [ 'status =\u003e is', 'PENDING' ]\n ]\n }\n }\n },\n ...\n}\n```\n\nNote that the `filter` attribute can be specified only on a view property, not on the actual property itself, which always includes all of its elements.\n\nThe filter specification is described in detail later in this manual when we talk about the fetch DBO.\n\n### Record Meta-Info Properties\n\nThe DBOs module introduces a number special _meta-info properties_ that can be specified on a record type and are maintained automatically by the framework. Such properties use `role` property definition attribute to specify their type. For example:\n\n```javascript\n{\n ...\n 'Account': {\n table: 'accounts',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'version': {\n valueType: 'number',\n role: 'version'\n },\n 'createdOn': {\n valueType: 'datetime',\n role: 'creationTimestamp',\n column: 'created_on'\n },\n 'createdBy': {\n valueType: 'string',\n role: 'creationActor',\n column: 'created_by'\n },\n 'modifiedOn': {\n valueType: 'datetime',\n role: 'modificationTimestamp',\n column: 'modified_on'\n },\n 'modifiedBy': {\n valueType: 'string',\n role: 'modificationActor',\n column: 'modified_by'\n },\n ...\n }\n },\n ...\n}\n```\n\nWhich has to be reflected in the `accounts` database table:\n\n```sql\nCREATE TABLE accounts (\n id INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n version INTEGER UNSIGNED NOT NULL,\n created_on TIMESTAMP(3) DEFAULT 0,\n created_by VARCHAR(30) NOT NULL,\n modified_on TIMESTAMP(3) NULL,\n modified_by VARCHAR(30),\n ...\n);\n```\n\nThe following meta-info property roles are supported:\n\n* `version` - Record version. A new record will have version 1. Every time a record is updated, the framework will increment the version property.\n\n* `creationTimestamp` - Timestamp when the record was created.\n\n* `creationActor` - Stamp of the actor that created the record (see [x2node-common](https://www.npmjs.com/package/x2node-common) module for the framework's notion of _actor_).\n\n* `modificationTimestamp` - Timestamp when the record was last time modified. The property is optional by default.\n\n* `modificationActor` - Stamp of the actor that modified the record last time. The property is optional by default.\n\nAll meta-info properties are marked as non-modifiable and from the application point of view they are read only.\n\n### Generated Properties\n\nA generated property is a property whose value is automatically assigned when a new record is created. Therefore, values for generated properties do not have to be provided in the record template passed into the insert DBO. A typical example of a generated property is a record id auto-assigned by the database (the _MySQL_'s `AUTO_INCREMENT` columns, _PostgreSQL_'s `SERIAL` columns, etc.).\n\nA generated property has a _generator_ associated with it. The generator associated with properties auto-assigned by the database, like the record ids mentioned above, is called _auto_. This is the default generator assigned to all properties that have `role` attribute equal \"id\" unless explicitely overridden. To explicitely assign a generator to a property, `generator` property definition attribute can be used. It can take one of the three possible values:\n\n* String \"auto\" for the _auto_ generator. The values are generated by the database and are made available to the application upon subsequent record read.\n\n* A `null`, to disable a generator otherwise assigned to the property by default. This makes the property _not_ generated and the value must be provided by the application in the record template when the record is submitted to the insert DBO.\n\n* A generator function. The function is called by the insert DBO when the value is needed before the record data is sent to the database for saving. The function takes a single argument\u0026mdash;the database connection (driver-specific)\u0026mdash;and returns either the generated value, or a `Promise` of it. The property descriptor is made available to the function as `this`.\n\nFor example:\n\n```javascript\nlet NEXT_TEMPORAL_ID = Date.now();\n\n...\n\nrecordTypes = {\n ...\n 'Account': {\n ...\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id',\n generator: 'auto' // no need for this, just a demo\n },\n 'events': {\n valueType: 'object[]',\n ...\n properties: {\n 'eventUUID': {\n valueType: 'string',\n role: 'id',\n generator: null // assigned by the application\n },\n 'temporalId': {\n valueType: 'number',\n generator: function() { return NEXT_TEMPORAL_ID++; }\n },\n ...\n }\n },\n ...\n }\n }\n ...\n};\n```\n\nThe custom generator function in this example does not need the database connection. A function that, for example, reads the value from a sequence in the database would need it and it would get it as its only argument and return a promise of the result.\n\nAs mentioned earlier, by default all id properties are assigned _auto_ generator. To change that behavior for the record types library as a whole, the library definition attribute `defaultIdGenerator` can be used. For example, if for the whole library the record ids are assigned by the application, the auto id generation can be disabled:\n\n```javascript\nconst recordTypes = records.with(dbos).buildLibrary({\n\n defaultIdGenerator: null,\n\n recordTypes: {\n ...\n }\n});\n```\n\n### Super-Properties\n\nSimetimes information needs to be fetched from the database that is not a specific record property. Instead, this may be information about a collection of records that match the query's criteria. Mostly, this is about aggregate values, such as the total count of matched records, or a sum of all _Order_ amounts, etc. This functionality is supported via so-called _super-properties_ or, if they are aggregates, _super-aggregates_. Super-aggregates can be added to a query specification to be included in the result.\n\nEvery record type automatically defines a super-aggregate called `count`, which reflects the number of records in the matched record set. It is possible to add custom super-aggregates as well. For example, if we want to be able to query the total amount for orders matching a certain criteria (or all orders available in the database, if no filter is provided with the query), as well as the number of pending orders, corresponding super-aggregates can be defined like this:\n\n```javascript\n{\n ...\n 'Order': {\n table: 'orders',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n ...\n 'status': {\n valueType: 'string'\n },\n ...\n 'items': {\n valueType: 'object[]',\n table: 'order_items',\n parentIdColumn: 'order_id',\n properties: {\n 'id': {\n valueType: 'number',\n role: 'id'\n },\n 'productRef': {\n valueType: 'ref(Product)',\n column: 'product_id',\n modifiable: false\n },\n 'quantity': {\n valueType: 'number'\n }\n }\n }\n },\n superProperties: {\n 'countPending': {\n valueType: 'number',\n aggregate: {\n collection: 'records',\n valueExpr: 'id =\u003e count',\n filter: [\n [ 'status =\u003e is', 'PENDING' ]\n ]\n }\n },\n 'countByStatus': {\n valueType: 'number{}',\n keyPropertyName: 'status',\n aggregate: {\n collection: 'records',\n valueExpr: 'id =\u003e count'\n }\n },\n 'totalAmount': {\n valueType: 'number',\n aggregate: {\n collection: 'records.items',\n valueExpr: 'productRef.price * quantity =\u003e sum'\n }\n }\n }\n },\n ...\n}\n```\n\nNote that the records collection is referred in the super-aggregate expressions as `records`.\n\nWhen a query with super-aggregates is executed, the super-aggregates are not a subject to the range specification, if any. They always reflect all records matching the filter regardless of the requested range.\n\n### Uniqueness of the Id Property\n\nThe framework makes an assumption that any property marked with `role` attribute equal \"id\" is unique within the whole table. This is certainly true for record ids, but sometimes a nested object id may be unique only within the parent record context, but not table-wide. To override the default framework's behavior and make it aware of the fact that the id property is not table-wide unique, a `tableUnique` attribute can be added to the id property definition with value `false`.\n\n## Fetch DBO\n\nThe fetch DBO is used to search records of a given record type and fetch the requested record data. The DBO is created using DBO factory's `buildFetch()` method, which takes the record type name and the query specification:\n\n```javascript\nconst fetchDBO = dboFactory.buildFetch('Order', {\n // the query specification goes here\n});\n```\n\nOnce built, the DBO can be used and reused multiple times, which may be helpful as, depending on the complexity of the query specification, the DBO construction may be a relatively costly operation.\n\nTo execute the DBO, its `execute()` method is called, which takes up to three arguments:\n\n* `txOrCon` - Database connection (or transaction object, described later in this manual).\n\n* `actor` - Optional actor performing the operation. If not provided, the operation is anonymous.\n\n* `filterParams` - If filter specification used to construct the DBO utilizes query parameters, this object provides the values for the parameters. Using query parameters in filter specifications helps making DBOs reusable. The keys in the provided object are parameter names and the values are the parameter values. Note, that when parameters are substituted in the SQL statement, no type transformation is performed by the framework. JavaScript strings are inserted as SQL strings, numbers as numbers, etc. Datetimes must be supplied as string in ISO format. References must be supplied as the referred record ids (strings or numbers depending on the referred record id value type).\n\nThe `execute()` method returns a `Promise`, which is fulfilled with the query result object. The query result object includes:\n\n* `recordTypeName`, which is the requested record type name.\n\n* `records` array of matched records, or empty array if none matched.\n\n* `referredRecords`, which is included if any referred records were requested to be fetched along with the main record set. The keys in the object are record references and the values are objects representing the referred records.\n\n* any requested super-properties.\n\nThe promise is rejected if an error occurs.\n\nThe query specification is an object that includes the following sections, each of which is optional and has a default behavior:\n\n* Specification of what record properties, referred records and super-properties to include in the result. Specified by the `props` array.\n\n* Filter specification that asks the DBO to include only those records that match the criteria. Specified by the `filter` array.\n\n* Order specification that tells the DBO how to order the records in the result's `records` array. Specified by the `order` array.\n\n* Range specification that tells the DBO to return only the specified subrange of all the matched records. Specified by the `range` two-element array.\n\n* Records locking specification that asks the DBO to lock matched records in a specific mode until the end of the transaction.\n\nIf no query specification is provided to the `buildFetch()` method, all records of the requested record type are included in the result with all the properties that are fetched by default (normally that includes all stored properties) and in no particular order. No referred records are fetched, no super-aggregates are fetch, and the DBO makes not effort to explicitely lock any matched records.\n\n### Selected Properties Specification\n\nBy default, the fetch DBO will fetch all stored properties of the matched records. Those include all properties that are not views, not calculated, not aggregates and not dependent record references. This default behavior can be overridden by using `fetchByDefault` attribute on the property definition, which can be either `true` or `false`, but generally is not recommended. Instead, to request only specific properties, the query specification can include a `props` property that is an array of property pattern strings. Each pattern can be:\n\n* A star \"*\" to include all properties fetched by default. In the essence, not providing a query with a properties specification is equivalent to providing it with `props` equal `[ '*' ]`.\n\n* Specific property path in dot notation. All intermediate properties are automatically included as well. If the end property is a nested object, all of its properties that are fetched by default are included. If any of the intermediate properties is a reference, the referred record is fetched and added to the `referredRecords` collection in the result object (only the properties explicitely selected are included, unless a wildcard pattern is used described next).\n\n* A wildcard pattern, which is a property path in dot notation ending with \".*\". This instructs the DBO to fetch the property and all of its children that would be fetched by default. In particular, if the property path is for a reference property, this allows fetching referred records and returning them in the `referredRecords` collection in the result object with all of their properties fetched by default.\n\n* Specific property path in dot notation prefixed with a dash \"-\" to exclude a property that would otherwise be included (as a consequence of a wildcard pattern, for example).\n\n* A super-property name prefixed with a dot \".\" to include the super-property.\n\nSee [Fetching Records](#fetching-records) usage section for an example.\n\n### Filter Specification\n\nBy default, all records of the requested record type are matched. To restrict the matched records set, a `filter` property can be included in the query specification. The filter property is an array of _filter terms_ (also known as _filter specification elements_). Each term can be either a specification of a test to perform on the record, or a logical junction of nested filters.\n\nFor example, to select all _Order_ records for orders that were placed before a certain date and that have either status \"PENDING\" or \"PROCESSING\", the following filter specification can be used:\n\n```javascript\nconst fetchDBO = dboFactory.buildFetch('Order', {\n ...\n filter: [\n [ 'placedOn =\u003e lt', '2017-01-01T00:00:00.000Z' ],\n [ ':or', [\n [ 'status =\u003e is', 'PENDING' ],\n [ 'status =\u003e is', 'PROCESSING' ]\n ]]\n ],\n ...\n});\n```\n\nAlternatively, this could be written as:\n\n```javascript\nconst fetchDBO = dboFactory.buildFetch('Order', {\n ...\n filter: [\n [ 'placedOn =\u003e lt', '2017-01-01T00:00:00.000Z' ],\n [ 'status =\u003e oneof', 'PENDING', 'PROCESSING' ]\n ],\n ...\n});\n```\n\nBut then we wouldn't see the use of the `:or` logical junction.\n\n#### Logical Junctions\n\nA logical junction is a filter term that is specified by a two-element array. The first element of the array is a string that describes how the nested filter terms are logically combined. The second element of the array is an array itself that consists of the filter terms that are combined in the junction.\n\nThe following junction types are supported:\n\n* `:or`, `:any`, `:!none` - Disjunction. The nested filter terms are combined using logical _OR_.\n\n* `:!or`, `:!any`, `:none` - Inverted disjunction. The nested filter terms are combined using logical _OR_ and then negated using logical _NOT_.\n\n* `:and`, `:all` - Conjunction. The nested filter terms are combined using logical _AND_.\n\n* `:!and`, `:!all` - Inverted conjunction. The nested filter terms are combined using logical _AND_ and then negated using logical _NOT_.\n\nThe top `filter` property in a query specification does not need to be an explicit logical junction if it is a conjunction (an `:and`).\n\n#### Tests\n\nEach test is a filter term that is defined as an array with one or more elements. The first element is called the _predicate_ and is always a string that expresses what is tested and what is the test. If the test requires supplementary values to test against\u0026mdash;the test parameters\u0026mdash;the values follow the predicate as the rest of the filter term array elements.\n\nThe predicate is defined as:\n\n```\nExpr =\u003e Test\n```\n\nWhere `Expr` is a value expression as in the [Calculated Properties](#calculated-properties). This is the value that is tested. The `Test` is one of:\n\n* `is`, `eq` - Test if the value is equal to the single test parameter.\n\n* `not`, `ne`, `!eq` - Test if the value is not equal to the single test parameter.\n\n* `min`, `ge`, `!lt` - Test if the value is greater or equal to the single test parameter.\n\n* `max`, `le`, `!gt` - Test if the value is less or equal to the single test parameter.\n\n* `gt` - Test if the value is strictly greater than the single test parameter.\n\n* `lt` - Test if the value is strictly less than the single test parameter.\n\n* `in`, `oneof`, `alt` - Test if the value is equal to any of the test parameters. The test take any number of parameters, or it takes an array of values as a parameter.\n\n* `!in`, `!oneof` - Test if the value is not equal to any of the test parameters.\n\n* `between` - Test if the value is greater or equal to the first test parameter and less or equal to the second test parameter.\n\n* `!between` - Test if the value is less than the first test parameter or greater than the second test parameter.\n\n* `contains` - Test if the value contains the substring provided as the single test parameter. The substring is treated as case-sensitive.\n\n* `containsi`, `substring` - Same as `contains`, but case-insensitive.\n\n* `!contains` - Inversion of `contains`.\n\n* `!containsi`, `!substring` - Inversion of `containsi`.\n\n* `starts` - Test if the value starts with the string provided as the single test parameter. The string is treated as case-sensitive.\n\n* `startsi`, `prefix` - Same as `starts`, but case-insensitive.\n\n* `!starts` - Inversion of `starts`.\n\n* `!startsi`, `!prefix` - Inversion of `startsi`.\n\n* `matches` - Test if the value matches the regular expression provided as the single test parameter. The match is performed as case-sensitive.\n\n* `matchesi`, `pattern`, `re` - Same as `matches`, but case-insensitive.\n\n* `!matches` - Inversion of `matches`.\n\n* `!matchesi`, `!pattern`, `!re` - Inversion of `matchesi`.\n\n* `empty` - Test if there is no value (the value is absent, `undefined` or `null`). This test does not take any parameters.\n\n* `!empty`, `present` - Test if the value is not empty.\n\nIf the test is not provided at all (there is no `=\u003e` followed by the test in the predicate) and there are no test parameters, then `!empty` is assumed. If there are parameters, then `eq` is assumed.\n\nThe parameters to the tests can be provided in several different forms. The simplest way is to provide the value as is:\n\n```javascript\nfilter = [\n [ 'name', 'John' ],\n [ 'status =\u003e oneof', 'PENDING', 'PROCESSING' ],\n [ 'quantity =\u003e between', 10, 20 ],\n [ 'isAccepted', true ],\n [ 'placedOn =\u003e min', '2017-02-01T10:00:00.000Z' ],\n [ 'accountRef', 10 ],\n [ 'zip =\u003e present' ],\n [ 'length(concat(lastName, \", \", firstName)) =\u003e max', 30 ]\n]\n```\n\nThe type of the test parameter value must match the predicate's expression. The framework does not perform the conversions assuming that the application knows what value to provide for the specific test. Note, that the values for `datetime` fields are provided as ISO strings and values for reference properties are provided as the referred record ids, which can be either strings or numbers depending on the record id property value type.\n\nSpecifying test parameters to filters as values has the effect of \"baking in\" the parameters into the DBOs, which limits their reusability. It is possible to create a parameterized DBO if filter test parameters are specified not as values, but as _filter parameters_. To create a named filter parameter placeholder, the module's `param()` function is used:\n\n```javascript\nconst dbos = require('x2node-dbos');\n\n...\n\nfilter = [\n [ 'name', dbos.param('name') ],\n [ 'status =\u003e oneof', dbos.param('statuses') ],\n [ 'accountRef', dbos.param('accountId') ]\n];\n```\n\nThe values for the filter parameters are provided when the DBO is executed:\n\n```javascript\nresultPromise = fetchDBO.execute(connection, actor, {\n name: 'John',\n statuses: [ 'PENDING', 'PROCESSING' ],\n accountId: 10\n});\n```\n\nAnd finally, sometimes it is necessary to test the predicate not against a value known to the application, but against another expression. This is made possible with the use of the module's `expr()` function:\n\n```javascript\nfilter = [\n [ 'accountRef =\u003e is', dbos.expr('ownerAccountRef') ]\n [ 'length(firstName) =\u003e gt', dbos.expr('length(lastName)') ]\n];\n```\n\nUsing property paths in dot notation allows constructing complex tests that involve values even from different related records.\n\n#### Collection Tests\n\nAll tests in the paragraph above test scalar values. Another class of tests is _collection property tests_. The predicate is a path to a collection (array or map) property. No value transformation functions or any other value calculation expressions are involved as they do not apply to collections. The only tests allowed are `empty` and `!empty`, which is the default if no test is specified, and `count` and `!count`. The test as a whole tests if the specified collection has elements or is empty, or has specified number of elements, or not. For example, to select only those _Account_ records that have some _Order_ records associated with them, the filter could be:\n\n```javascript\nfilter = [\n [ 'orderRefs' ] // equivalent to [ 'orderRefs =\u003e !empty' ]\n];\n```\n\nA collection test can have a nested filter the applies to the collection elements. For example, to select accounts that have pending orders placed before certain date, the following filter could be constructed:\n\n```javascript\nfilter = [\n [ 'orderRefs', [\n [ 'status', 'PENDING' ],\n [ 'placedOn =\u003e lt', '2017-10-12T22:00:00.000Z' ]\n ]]\n];\n```\n\nTo select account that have exactly two orders for a given date:\n\n```javascript\nfilter = [\n [ 'orderRefs =\u003e count', 2, [\n [ 'deliveryDate', '2017-10-12' ]\n ]]\n];\n```\n\nNote, that the `count` (and `!count`) tests require an integer argument.\n\nFor the property paths, the nested collection filter assumes the context of the collection elements, so properties `status` and `placedOn` in the example above belong to the _Order_ records (in the context of the fetched record type, the _Account_ in our case, they would be `orderRefs.status` and `orderRefs.placedOn` and would be not allowed).\n\nNote, that collection properties are not allowed as property path intermediate elements in regular tests. In collection tests, however, they are allowed as both the end property and as an intermediate property, which allows hopping over multiple collections.\n\n### Order Specification\n\nThe query specification object may include an `order` property to request a specific order, in which fetched records are returned in the `records` array of the fetch DBO result object. The `order` property is an array of strings, each element specifying oredering by a particular value expression. The order elements follow the pattern:\n\n```\nExpr =\u003e Dir\n```\n\nWhere `Expr` is a value expression and `Dir` is either `asc` or `desc`. If not `Dir` is provided, `asc` is assumed by default. For example, we could order the _Product_ records first by the length of the product name in descending order and then by the price:\n\n```javascript\norder = [\n 'length(name) =\u003e desc', 'price'\n];\n```\n\nNote that only scalar properties are allowed to be used for sorting.\n\n### Range Specification\n\nBy default, all matched records are returned in the fetch DBO result object's `records` array. It is possible, however, to request only a certain window of the full result set. To do that, query specification object's `range` property can be used. The `range` property is a two-element array. The first element, an integer number, is the index of the first record to return, starting from zero. The second element, also an integer number, is the maximum number of records to return. For example, to request only first 5 matched records, the following range specification can be used:\n\n```javascript\nrange = [ 0, 5 ];\n```\n\nThe next 5 records could be requested like this:\n\n```javascript\nrange = [ 5, 5 ];\n```\n\nAnd so on.\n\nThe range applies to records, not SQL query result set rows, even if the requested records contain collection properties with unknown number of elements.\n\nAlso note, that the range specification does not affect the super-aggregates, which always refect the whole collection of matched records regardless of the requested range. This allows, for example, to calculate the total number of pages during result set pagination and return correct totals in general regardless of the requested page.\n\n### Records Locking\n\nWhen a fetch DBO is used as a part of a larger transaction (see [Transactions](#transactions)) it sometimes necessary to ask the DBO to lock involved data in a certain mode until the end of the transaction. There are two types of locks that are specified by the query specification object's `lock` property: \"shared\" and \"exclusive\".\n\nFrom the point of view of the DBO, there are two types of records that may be locked: the matched records of the main record type being fetched, and any referred records of other record types. When the `lock` property has value \"shared\", all records\u0026mdash;both main record type records and used referred records, if any\u0026mdash;are lock in such a way that protects them against modification by other transactions until the end of the transaction, in which the DBO is participating. When the `lock` is \"exclusive\", the main record type records are protected against reading by other transactions and any referred records are locked in the \"shared\" mode, which protects them against modification. The \"exclusive\" mode is used to fetch the data of records before making modifications to them so that the modifications are made based on the current record data and other transactions are not allowed to see the data until the modifications are completed.\n\n## Insert DBO\n\nThe insert DBO is used to create new records of a given record type. The DBO is created using DBO factory's `buildInsert()` method, which takes the record type name and the record template, which is the record data sans any properties that are automatically generated (such as record id, meta-info properties, other [generated properties](#generated-properties)). See [Creating Records](#creating-records) in the opening usage section for an example.\n\nNote that since insert DBO's take record data in during the DBO construction, their reusability is normally limited. Luckily, constructing an insert DBO is not a heavy operation.\n\nTo execute an insert DBO, its `execute()` method is called. The method takes two arguments:\n\n* `txOrCon` - Database connection (or transaction object, described later in this manual).\n\n* `actor` - Optional actor performing the operation. If not provided, the operation is anonymous. If the record type has meta-info property with role `creationActor`, anonymous executions of the DBO will not be allowed, since the DBO must fill in the field.\n\nThe `execute()` method returns a `Promise` that fulfills with the new record id. If an error happens, the promise is rejected with it.\n\n## Update DBO\n\nThe update DBO is used to modify existing records. The DBO is created using DBO factory's `buildUpdate()` method. The method takes three arguments:\n\n* The record type name.\n\n* The patch specification created using [x2node-patches](https://www.npmjs.com/package/x2node-patches) module.\n\n* A filter for the records to patch or a fetcher function that provides the DBO with the records to patch.\n\nNote, that update DBO is not made for bulk record updates. The way it works is it selects all records that pass the filter with all their properties that are fetched by default (that is the filter was specified and not a fetcher function), loads them into memory, and then applies the patch and saves the changes back into the database for each record one by one. The main use-case for the update DBO is updating a single record selected by its id, or a small number of records. For bulk record updates custom SQL statements are used.\n\nTo execute the DBO, its `execute()` method is called. The method takes the following arguments:\n\n* `txOrCon` - Database connection (or transaction object, described later in this manual).\n\n* `actor` - Optional actor performing the operation. If not provided, the operation is anonymous. If the record type has meta-info property with role `modificationActor`, anonymous executions of the DBO will not be allowed, since the DBO must fill in the field.\n\n* `recordValidators` - Optional functions used to validate/normalize the records right before and after the patch is applied, but before the changes are saved into the database. The argument is an object with two optional functions: `beforePatch` and `afterPatch`. Each function, if provided, takes the record as its only argument. The record has all the properties fetched by default on it. Anything (including nothing) returned by the function that is not a rejected `Promise` allows the operation to proceed. If the function returns a `Promise` and it is rejected, the whole operation is immediately aborted and the promise returned by the DBO's `execute()` method is rejected with the same value. If multiple records are participating in the operation, changes saved for the records processed before the rejected one can be rolled back with the database transaction. Alternatively, instead of an object a function can be provided as the `recordValidators` argument, in which case the function is treated as the `afterPatch` function.\n\n* `filterParams` - If the DBO was created with a filter that utilizes named query parameters (`dbos.param(paramName)` function), this is the values for the parameters (object with keys for the parameter names and values for the corresponding parameter values). If fetcher function was used instead of a filter, this argument is still available to the fetcher function via the DBO execution context.\n\nThe returned by the `execute()` method promise, if not rejected as a result of an error, is fulfilled with an update operation result object. The result object includes:\n\n* `records` - An array with all matched records with all properties fetched by default and the patch applied.\n\n* `updatedRecordIds` - Array with ids of those records in the `records` array that were actually modified by the patch. May be an empty array if none.\n\n* `testFailed` - The supplied patch may have contained `test` operations (see _JSON Patch_ specification). If so, those matched records, for which the test failed were not updated by the DBO. The `test` patch operation failure is not considered a DBO error. If it happens for a record, the record is left alone and the DBO continues on to the rest of the matched records. However, if a `test` failed for any of the matched records, the `testFailed` flag in the update operation result object will be set to `true`. Also note, that the records, for which `test` failed, may appear partially modified in the `records` array, even though those partial modifications were never flushed to the database by the DBO.\n\n* `failedRecordIds` - If `testFailed` is `true`, this is an array with ids of those records in the `records` array, for which the `test` patch operation failed.\n\nNote, that if the update DBO participates in a larger transaction, it places an \"exclusive\" lock on the matched records.\n\n## Delete DBO\n\nThe delete DBO is used to delete existing records. The DBO deletes the records that pass the provided filter and also cascades the deletion onto all referred strongly dependent records. To create a DBO, DBO factory's `buildDelete()` method is used. It takes two parameters: the record type name and the optional (but rarely omitted) filter specification.\n\nThe DBO's `execute()` method takes three arguments:\n\n* `txOrCon` - Database connection (or transaction object, described later in this manual).\n\n* `actor` - Optional actor performing the operation.\n\n* `filterParams` - If the DBO was created with a filter that utilizes named query parameters (`dbos.param(paramName)` function), this is the values for the parameters (object with keys for the parameter names and values for the corresponding parameter values).\n\nThe `Promise` returned by the `execute()` method either rejects with an error, or is fulfilled with a result object, which, for every record type any records of which were actually deleted, has a property with the record type name as the property name and the number of deleted records as the value. Zeros are not included, so if no records matched the filter and were deleted, the promise is fulfilled with an empty object.\n\n## Transactions\n\nEvery DBO's `execute()` method takes a `txOrCon` argument. The argument can be a database driver-specific connection object. In that case, the `execute()` method automatically creates a transaction and executes all the statements it needs to execute to complete the operation in that transaction. If everything goes well, the DBO commits the transaction and the `Promise` returned by the `execute()` method is fulfilled with the operation result. If an error happens, the transaction is rolled back and the `Promise` is rejected with the error.\n\nHere is an example with a _MySQL_ connection pool:\n\n```javascript\nconst mysql = require('mysql');\n\nconst pool = mysql.createPool({\n connectionLimit: 5,\n host: 'localhost',\n database: 'mydb',\n user: 'mydbuser',\n password: 'mydbuserpassword'\n});\n\nconst dbo = dboFactory.build...\n\npool.getConnection((err, connection) =\u003e {\n\n if (err)\n throw err;\n\n dbo.execute(connection, ...).then(\n result =\u003e {\n connection.release();\n console.log('success:', result);\n },\n err =\u003e {\n connection.release(err);\n console.error('error:', err);\n }\n );\n});\n```\n\nAlternatively, especially if transaction involves executing multiple DBOs, the application can use a transaction factory provided by the DBO factory's `createTxFactory()` method. The methos takes database connections source (see [Data Sources](#data-sources)) as its only argument and returns a reusable instance of `TxFactory` class. The `TxFactory` instance exposes a `executeTransaction()` method, which takes a callback as an argument. The method creates and starts a transaction, calls the callback passing the transaction handler to it, and then either commits the transaction or rolls it back in case of an error. For example:\n\n```javascript\nconst mysql = require('mysql');\n\nconst pool = mysql.createPool({\n ...\n});\n\n// build our DBOs that will comprise the transaction\nconst dbo1 = dboFactory.build...\nconst dbo2 = dboFactory.build...\n\n// create the standardized database connections source\nconst ds = dboFactory.adaptDataSource(pool);\n\n// create the transaction factory\nconst txFactory = dboFactory.createTxFactory(ds);\n\n// execute our DBOs in a transaction\ntxFactory.executeTransaction(tx =\u003e {\n\n return dbo1.execute(tx, ...).then(\n () =\u003e dbo2.execute(tx, ...));\n});\n```\n\nThe callback passed to `executeTransaction()` method may, and usually does, return a `Promise` with the transaction result. The `executeTransaction()` method itself also returns a `Promise`, which is fulfilled with the callback result, or is rejected if the callback returns a rejected promise or throws an error. The rejection of the promise returned by the `executeTransaction()` method indicates that the transaction was rolled back. Successful fulfillment of the promise indicates that the transaction was committed.\n\nThe callback passed to the `executeTransaction()` method receives a transaction handler, which is an instance of `Transaction` class and exposes the following properties and methods:\n\n* `id` - A string that uniquely identifies the transaction within the process.\n\n* `startedOn` - Timestamp, instance of `Date`, indicating when the transaction was started. (Available only after the promise returned by `start()` method successfully fulfills. See below.)\n\n* `connection` - The database connection object allocated from the data source provided to the transaction factory.\n\n* `dbDriver` - The dat","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fboylesoftware%2Fx2node-dbos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fboylesoftware%2Fx2node-dbos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fboylesoftware%2Fx2node-dbos/lists"}