{"id":23122688,"url":"https://github.com/folio-org/raml-module-builder","last_synced_at":"2025-08-17T02:31:04.325Z","repository":{"id":38417809,"uuid":"65975333","full_name":"folio-org/raml-module-builder","owner":"folio-org","description":"Framework allowing easy module creation based on RAML files","archived":false,"fork":false,"pushed_at":"2025-03-12T14:32:05.000Z","size":11161,"stargazers_count":23,"open_issues_count":2,"forks_count":22,"subscribers_count":29,"default_branch":"master","last_synced_at":"2025-04-04T23:05:09.865Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/folio-org.png","metadata":{"files":{"readme":"README.md","changelog":"NEWS-cql2pgjson-java.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-08-18T07:28:13.000Z","updated_at":"2025-02-28T08:45:43.000Z","dependencies_parsed_at":"2024-01-17T23:56:48.360Z","dependency_job_id":"3bfbcdd4-f1bc-4395-b5f0-1159aa55f8c8","html_url":"https://github.com/folio-org/raml-module-builder","commit_stats":null,"previous_names":[],"tags_count":189,"template":false,"template_full_name":null,"purl":"pkg:github/folio-org/raml-module-builder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Framl-module-builder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Framl-module-builder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Framl-module-builder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Framl-module-builder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/folio-org","download_url":"https://codeload.github.com/folio-org/raml-module-builder/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Framl-module-builder/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270798883,"owners_count":24648045,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-17T02:00:09.016Z","response_time":129,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-17T07:29:47.248Z","updated_at":"2025-08-17T02:31:04.296Z","avatar_url":"https://github.com/folio-org.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Raml-Module-Builder\n\nCopyright (C) 2016-2025 The Open Library Foundation\n\n[ClassPath.java](domain-models-api-aspects/src/main/java/org/folio/rest/tools/utils/ClassPath.java) and\n[ClassPathTest.java](domain-models-api-aspects/src/test/java/org/folio/rest/tools/utils/ClassPathTest.java)\nare also Copyright (C) 2012 The Guava Authors.\n\nThis software is distributed under the terms of the Apache License, Version 2.0.\nSee the file [\"LICENSE\"](LICENSE) for more information.\n\n\u003c!-- ../okapi/doc/md2toc -l 2 -h 3 README.md --\u003e\n* [Introduction](#introduction)\n* [Upgrading](#upgrading)\n* [Overview](#overview)\n* [The basics](#the-basics)\n * [Build-time workflow](#build-time-workflow)\n * [Generate-time workflow](#generate-time-workflow)\n * [Implement the interfaces](#implement-the-interfaces)\n * [Set up your pom.xml](#set-up-your-pomxml)\n * [Build and run](#build-and-run)\n* [Get started with a sample working module](#get-started-with-a-sample-working-module)\n* [Command-line options](#command-line-options)\n* [Environment Variables](#environment-variables)\n* [Read and write database instances setup](#read-and-write-database-instances-setup)\n* [Local development server](#local-development-server)\n* [Creating a new module](#creating-a-new-module)\n * [Step 1: Create new project directory layout](#step-1-create-new-project-directory-layout)\n * [Step 2: Include the jars in your project pom.xml](#step-2-include-the-jars-in-your-project-pomxml)\n * [Step 3: Add the plugins to your pom.xml](#step-3-add-the-plugins-to-your-pomxml)\n * [Step 4: Build your project](#step-4-build-your-project)\n * [Step 5: Implement the generated interfaces](#step-5-implement-the-generated-interfaces)\n * [Step 6: Design the RAML files](#step-6-design-the-raml-files)\n* [RestVerticle](#restverticle)\n* [Adding an init() implementation](#adding-an-init-implementation)\n* [Adding code to run periodically](#adding-code-to-run-periodically)\n* [Adding a hook to run immediately after verticle deployment](#adding-a-hook-to-run-immediately-after-verticle-deployment)\n* [Adding a shutdown hook](#adding-a-shutdown-hook)\n* [Implementing uploads](#implementing-uploads)\n* [Implement chunked bulk download](#implement-chunked-bulk-download)\n* [PostgreSQL integration](#postgresql-integration)\n * [Minimum PostgreSQL server version](#minimum-postgresql-server-version)\n * [Saving binary data](#saving-binary-data)\n * [Securing DB Configuration file](#securing-db-configuration-file)\n * [Foreign keys constraint](#foreign-keys-constraint)\n* [CQL (Contextual Query Language)](#cql-contextual-query-language)\n * [CQL2PgJSON: CQL to PostgreSQL JSON converter](#cql2pgjson-cql-to-postgresql-json-converter)\n * [CQL2PgJSON: Usage](#cql2pgjson-usage)\n * [CQL: Field names](#field-names)\n * [CQL: Relations](#cql-relations)\n * [CQL: Modifiers](#cql-modifiers)\n * [CQL: Matching, comparing and sorting numbers](#cql-matching-comparing-and-sorting-numbers)\n * [CQL: Matching id and foreign key fields](#cql-matching-id-and-foreign-key-fields)\n * [CQL: Matching full text](#cql-matching-full-text)\n * [CQL: Matching all records](#cql-matching-all-records)\n * [CQL: Matching undefined or empty values](#cql-matching-undefined-or-empty-values)\n * [CQL: Matching array elements](#cql-matching-array-elements)\n * [CQL: @-relation modifiers for array searches](#cql--relation-modifiers-for-array-searches)\n * [CQL2PgJSON: Multi Field Index](#cql2pgjson-multi-field-index)\n * [CQL2PgJSON: Foreign key cross table index queries](#cql2pgjson-foreign-key-cross-table-index-queries)\n * [CQL2PgJSON: Foreign key tableAlias and targetTableAlias](#cql2pgjson-foreign-key-tablealias-and-targettablealias)\n * [CQL2PgJSON: Exceptions](#cql2pgjson-exceptions)\n * [CQL2PgJSON: Unit tests](#cql2pgjson-unit-tests)\n* [Tenant API](#tenant-api)\n* [RAMLs API](#ramls-api)\n* [JSON Schemas API](#json-schemas-api)\n* [Query Syntax](#query-syntax)\n* [Estimated totalRecords](#estimated-totalrecords)\n* [Metadata](#metadata)\n* [Optimistic Locking](#optimistic-locking)\n* [Facet Support](#facet-support)\n* [JSON Schema fields](#json-schema-fields)\n* [Overriding RAML (traits) / query parameters](#overriding-raml-traits--query-parameters)\n* [Boxed types](#boxed-types)\n* [Messages](#messages)\n* [Documentation of the APIs](#documentation-of-the-apis)\n* [Logging](#logging)\n* [Monitoring](#monitoring)\n* [Instrumentation](#instrumentation)\n* [Overriding Out of The Box RMB APIs](#overriding-out-of-the-box-rmb-apis)\n* [Client Generator](#client-generator)\n* [A Little More on Validation](#a-little-more-on-validation)\n* [Advanced Features](#advanced-features)\n* [Additional Tools](#additional-tools)\n* [Some REST examples](#some-rest-examples)\n* [Additional information](#additional-information)\n\n## Introduction\n\nThis documentation includes information about the Raml-Module-Builder (RMB) framework\nand examples of how to use it.\n\nThe goal of the project is to abstract away as much boilerplate functionality as\npossible and allow a developer to focus on implementing business functions. In\nother words: **simplify the process of developing a micro service module**.\nThe framework is RAML driven, meaning a developer / analyst declares APIs that the\n'to be developed' module is to expose (via RAML files) and declares the objects\nto be used and exposed by the APIs (via JSON schemas). Once the schemas and RAML\nfiles are in place, the framework generates code and offers a number of tools\nto help implement the module.\nNote that this framework is both a build and a run-time library.\n\n\nThe framework consists of a number of tools:\n\n- `domain-models-api-interfaces` -- project exposes tools that receive as input\n these RAML files and these JSON schemas, and generates java POJOs and java\n interfaces.\n\n- `domain-models-api-aspects` -- project exposes tools that enforce strict\n adherence to the RAML declaration to any API call by exposing validation\n functionality.\n\n - for example: a RAML file may indicate that a specific parameter is\n mandatory or that a query parameter value must be a specific regex pattern.\n The aspects project handles this type of validation for developers so that it\n does not need to be re-developed over and over. More on validation\n [below](https://github.com/folio-org/raml-module-builder#a-little-more-on-validation).\n\n- `domain-models-runtime` -- project exposes a run-time library which should be\n used to run a module. It is Vert.x based. When a developer implements the\n interfaces generated by the interfaces project, the run-time library should be\n included in the developed project and run. The run-time library will\n automatically map URLs to the correct implemented function so that developers\n only need to implement APIs, and so all the wiring, validation,\n parameter / header / body parsing, logging (every request is logged in an\n apache like format) is handled by the framework. Its goal is to abstract\n away all boilerplate functionality and allow a module implementation to focus\n on implementing business functions.\n\n - The runtime framework also exposes hooks that allow developers to\n implement one-time jobs, scheduled tasks, etc.\n\n - Provides tooling (Postgres client, etc.) for developers\n to use while developing their module.\n\n - Runtime library runs a Vert.x verticle.\n\n- `rules` -- Basic Drools functionality allowing module developers to create\n validation rules via `*.drl` files for objects (JSON schemas).\n\n## Upgrading\n\nSee separate [upgrading notes](doc/upgrading.md) how to upgrade an RMB based module to\na new RMB version.\n\n## Overview\n\nFollow the [Introduction](#introduction) section above to generally understand\nthe RMB framework.\nReview the separate [Okapi Guide and Reference](https://github.com/folio-org/okapi/blob/master/doc/guide.md).\nScan the [Basics](#the-basics) section below for a high-level overview of RMB. Then follow the\n[Get started with a sample working module](#get-started-with-a-sample-working-module)\nsection which demonstrates an already constructed example.\nWhen that is understood, then move on to the section\n[Creating a new module](#creating-a-new-module) to get your project started.\n\nNote that actually building this RAML Module Builder framework is not required.\n(Some of the images below are out-of-date.) The already published RMB artifacts will\nbe [incorporated](#step-2-include-the-jars-in-your-project-pomxml) into your project from the repository.\n\n## The basics\n\n### Build-time workflow\n\nBuild the `raml-module-builder` project to generate the needed jars, then add them to your project's `pom.xml`.\n\n![](images/build.png)\n\n### Generate-time workflow\n\nCall the domain-models-maven-plugin to generate POJOs and interfaces within your project.\n\n![](images/generate.png)\n\nSee the [domain-models-runtime-it/pom.xml](domain-models-runtime-it/pom.xml) for an example.\n\n#### Generated Files\n\nThe following RAML snippet will be used for this example:\n\n```raml\n/bibs/{bibId}:\n type:\n post:\n example:\n !include examples/bid.sample\n schema:\n bib.schema\n```\n\nAdditionally, the following query parameters and header requirements will be used:\n```raml\nheaders:\n Authorization:\n description: |\n Used to send a valid JWT token.\n example:\n Bearer Hc8KNK7LAJPasAwX9pIbN7yeTwSCAq\n required: true\nqueryParameters:\n lang:\n description: |\n Requested language. Optional. [lang=en]\n type: string\n required: false\n default: en\n pattern: \"[a-zA-Z]{2}\"\n```\n\nFrom this snippet, an interface (`BibInterface.java`) is generated based on the paths (each path+verb pair generates a method). This is documented with the examples from your RAML.\n\nAdditionally, an object (`Bib.java`) is generated based on the JSON Schema provided. An example of this may be found in the [A Little More on Validation](#a-little-more-on-validation) section.\n\nThe following is an example of the interface method signature that would be generated:\n\n```java\nvoid postBibs(\n @HeaderParam(\"Authorization\")\n @NotNull\n String authorization,\n @QueryParam(\"lang\")\n @DefaultValue(\"en\")\n @Pattern(regexp = \"[a-zA-Z]{2}\")\n String lang,\n Bib entity\n) throws Exception;\n```\n\n### Implement the interfaces\n\nAn example of the generated constraints is shown [above](#generated-files).\n\n- When implementing the interfaces, you must add the @Validate\n annotation to enforce the annotated constraints declared by the interface.\n- Note that a Bib entity was passed as a parameter. The runtime framework\n transforms the JSON passed in the body into the correct POJO.\n\n\n### Set up your pom.xml\n\n- Add Maven repositories for FOLIO. There must be a section for regular\n dependencies (`\u003crepositories\u003e`) and a section for plugins\n (`\u003cpluginRepositories\u003e`).\n\n- Add the `domain-models-maven-plugin`. This will generate the POJOs and interfaces based on\n the RAML files. Use `domain-models-maven-plugin` as a plugin in the `\u003cplugins\u003e` pom.xml\n section only. Please do not add it to the `\u003cdependencies\u003e` section, it is based on\n old libraries with security vulnerabilities. The vulnerabilities do not affect us when\n running `domain-models-maven-plugin` as a plugin but they may affect a module if those\n old dependencies are included as explicit dependencies.\n\n- Add the `aspectj-maven-plugin`. This is required if you\n would like the runtime framework to validate all URLs.\n\n- Add the `maven-shade-plugin`, indicating the main class to\n run as `RestLauncher` and main verticle as `RestVerticle`. This will create a\n runnable jar with the runtime's `RestVerticle` serving as the main class.\n\n- Add the `maven-resources-plugin`. This will copy\n your RAML files to the /apidocs directory where they will be made visible\n online (html view) by the runtime framework.\n\nThese are further explained below.\n\n### Build and run\n\nDo `mvn clean install` ... and run :)\n\nThe runtime framework will route URLs in your RAML to the correct method\nimplementation. It will validate (if `@Validate` was used), log, and expose\nvarious tools.\n\nNotice that no web server was configured or even referenced in the implementing\nmodule - this is all handled by the runtime framework.\n\nSome sample projects:\n\n- https://github.com/folio-org/mod-configuration\n- https://github.com/folio-org/mod-notes\n\nand other [modules](https://dev.folio.org/source-code/#server-side) (not all do use the RMB).\n\n\n## Get started with a sample working module\n\nThe [mod-organizations-storage](https://github.com/folio-org/mod-organizations-storage)\nis a full example which uses the RMB. Clone it, and then investigate:\n\n```\n$ git clone --recursive https://github.com/folio-org/mod-organizations-storage.git\n$ cd mod-organizations-storage\n$ mvn clean install\n```\n\n- Its RAMLs and JSON schemas can be found in the `ramls` directory.\nThese are also displayed as local [API documentation](#documentation-of-the-apis).\n\n- Open the pom.xml file - notice the jars in the `dependencies` section as well as the `plugins` section. The `ramls` directory is declared in the pom.xml and passed to the interface and POJO generating tool via a maven exec plugin. The tool generates source files into the `target/generated-sources/raml-jaxrs` directory. The generated interfaces are implemented within the project in the `org.folio.rest.impl` package.\n\n- Investigate the `src/main/java/org/folio/rest/impl/OrganizationsAPI.java` class. Notice that there is a function representing each endpoint that is declared in the RAML file. The appropriate parameters (as described in the RAML) are passed as parameters to these functions so that no parameter parsing is needed by the developer. Notice that the class contains all the code for the entire module. All handling of URLs, validations, objects, etc. is all either in the RMB jars, or generated for this module by the RMB at build time.\n\n- **IMPORTANT NOTE:** Every interface implementation - by any module -\n must reside in package `org.folio.rest.impl`. This is the package that is\n scanned at runtime by the runtime framework, to find the needed runtime\n implementations of the generated interfaces.\n\nNow run the module in standalone mode:\n\n```\n$ java -jar target/mod-organizations-storage-fat.jar\n```\n\nNow send some requests using '[curl](https://curl.haxx.se)' or '[httpie](https://httpie.org)'\n\nAt this stage there is not much that can be queried, so stop that quick demonstration now.\nAfter explaining general command-line options, etc.\nwe will get your local development server running and populated with test data.\n\n## Command-line options\n\n- `-Dhttp.port=8080` Optional -- defaults to 8081\n\n- `-Ddebug_log_package=*` Optional -- Set log level to debug for all packages.\nOr use `org.folio.rest.*` for all classes within a specific package,\nor `org.folio.rest.RestVerticle` for a specific class.\n\n- `db_connection=[path]` Optional -- path to a JSON config file with\n connection parameters to a PostgreSQL DB\n\n - for example Postgres: `{\"host\":\"localhost\", \"port\":5432, \"maxPoolSize\":50,\n \"username\":\"postgres\",\"password\":\"mysecretpassword\", \"database\":\"postgres\",\n \"charset\":\"windows-1252\", \"queryTimeout\" : 10000}`\n\n - path defaults to /postgres-conf.json\n\n - tries to read a file at the path if the path is absolute\n\n - if file not found or path is relative tries to read a class path resource with that path\n\n- Other module-specific arguments can be passed via the command line in the format key=value. These will be accessible to implementing modules via `RestVerticle.MODULE_SPECIFIC_ARGS` map.\n\n- Optional JVM arguments can be passed before the `-jar` argument, e.g.\n`-XX:+HeapDumpOnOutOfMemoryError`\n`-XX:+PrintGCDetails`\n`-XX:+PrintGCTimeStamps`\n`-Xloggc:C:\\Git\\circulation\\gc.log`\n\n## Environment Variables\n\nRMB implementing modules expect a set of environment variables to be passed in at module startup. The environment variables expected by RMB modules are:\n\n - DB_HOST\n - DB_PORT\n - DB_USERNAME\n - DB_PASSWORD\n - DB_DATABASE\n - DB_HOST_READER\n - DB_PORT_READER\n - DB_SERVER_PEM\n - DB_QUERYTIMEOUT\n - DB_CHARSET\n - DB_MAXPOOLSIZE\n - DB_MAXSHAREDPOOLSIZE\n - DB_CONNECTIONRELEASEDELAY\n - DB_RECONNECTATTEMPTS\n - DB_RECONNECTINTERVAL\n - DB_EXPLAIN_QUERY_THRESHOLD\n - DB_ALLOW_SUPPRESS_OPTIMISTIC_LOCKING\n - TESTCONTAINERS_POSTGRES_IMAGE\n\nThe first five are mandatory, the others are optional.\n\nEnvironment variables with periods/dots in their names are deprecated in RMB because a period is [not POSIX compliant](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html) and therefore some shells, notably, the BusyBox /bin/sh included in Alpine Linux, strip them (reference: [warning in OpenJDK docs](https://hub.docker.com/_/openjdk/)).\n\nSee the [Vert.x Async PostgreSQL Client Configuration documentation](https://vertx.io/docs/vertx-mysql-postgresql-client/java/#_configuration) for the details.\n\nThe environment variable `DB_MAXPOOLSIZE` sets the maximum number of concurrent connections for a tenant that one module instance opens. They are only opened if needed. If all connections for a tenant are in use further requests for that tenant will wait until one connection becomes free. Other tenants and other instances of a module are unaffected. The default is 4.\n\nThe environment variable `DB_MAXSHAREDPOOLSIZE` sets the maximum number of concurrent connections that one module instance opens. They are only opened if needed. If all connections are in use further requests will wait until one connection becomes free. This way one tenant may block other tenants. If the variable is set `DB_MAXPOOLSIZE` is ignored and all connections are shared across tenants.\n\nUse `DB_SERVER_PEM` (or `server_pem` in the JSON config) to set SSL/TLS certificate(s) in PEM format to validate the PostgreSQL server certificate, this can be the server certificate, the root CA certificate, or the chain of the intermediate CA and the CA certificate. Defaults to none allowing unencrypted connection only. If set requires a TLSv1.3 connection and a valid server certificate, and rejects unencrypted connections.\n\nThe environment variable `DB_CONNECTIONRELEASEDELAY` sets the delay in milliseconds after which an idle connection is closed. A connection becomes idle if the query ends, it is not idle if it is waiting for a response. Use 0 to keep idle connections open forever. RMB's default is one minute (60000 ms).\n\n`DB_RECONNECTATTEMPTS` and `DB_RECONNECTINTERVAL` set the maximum number of retries after a connect to the database fails, and how many milliseconds to wait before the next reconnect. Reconnecting is disabled by default.\n\nThe environment variable `DB_EXPLAIN_QUERY_THRESHOLD` is not observed by\nPostgres itself, but is a value - in milliseconds - that triggers query\nexecution analysis. If a single query exceeds this threshold, it will be\nanalyzed by using `EXPLAIN ANALYZE`. Note that this in turn further adds\ntime to the query, so this should only be executed for slow queries that\nneeds further attention. The analysis can effectively be turned off by setting\nit to a high value (e.g. 300000 ~ 5 minutes). Like the DB-environment\nvariables, this pertains per RMB-module (process). The default\nvalue of `DB_EXPLAIN_QUERY_THRESHOLD` is 1000 (1 second).\n\nThe EXPLAIN ANALYZE - is only performed for PostgresClient.get,\nPostgresClient.select and PostgresClient.join. Not for methods such\nas PostgresClient.getById or PostgresClient.streamGet.\n\nThe environment variable `DB_QUERYTIMEOUT` sets the number of milliseconds after which RMB sends a\n\u003ca href=\"https://www.postgresql.org/docs/14/protocol-flow.html#id-1.10.5.7.10\"\u003ecancel request\u003c/a\u003e\nto a running PostgreSQL query. 0 disables this timeout and is the default.\nTo take effect an RMB based module must get it via PostgresClient.getConnectionConfig().getInteger(\"queryTimeout\")\nand pass it to the RMB method that starts the connection, transaction or query.\n\nThe environment variables `DB_HOST_READER` and `DB_PORT_READER` are for the synchronously replicated [read and write database instances setup](#read-and-write-database-instances-setup).\n\nThe environment variables `DB_HOST_ASYNC_READER` and `DB_PORT_ASYNC_READER` are for the asynchronously replicated read and write database instances setup.\n\n`DB_ALLOW_SUPPRESS_OPTIMISTIC_LOCKING` is a timestamp in the format `2022-12-31T23:59:59Z`. Setting it disables optimistic locking when sending a record that contains `\"_version\":-1` before that time, after that time `\"_version\":-1` is rejected. This applies only to tables with `failOnConflictUnlessSuppressed`, see below. The timestamp ensures that disabling this option cannot be forgotten. Suppressing optimistic locking is known to lead to data loss in some cases, don't use in production, you have been warned!\n\n`TESTCONTAINERS_POSTGRES_IMAGE` changes the PostgreSQL container image name used at build time for testing; it is not used at runtime.\n\nSee the [Environment Variables](https://github.com/folio-org/okapi/blob/master/doc/guide.md#environment-variables) section of the Okapi Guide for more information on how to deploy environment variables to RMB modules via Okapi.\n\n## Read and write database instances setup\nA PostgreSQL instance (the write instance) can be replicated for horizontal scaling (scale out). Each replica is a read-only standby instance.\n\nRMB supports separating read and write requests. By default the write instance configured with `DB_HOST` and `DB_PORT` environment variables is used for reading as well, but optionally a read instance (or a load balancer for multiple read instances) can be configured by setting its host and port using the `DB_HOST_READER` and `DB_PORT_READER` environment variables. If either of these reader variables are not set then it will default to use the writer instance.\n\nRMB supports both [synchronous standby servers](https://www.postgresql.org/docs/current/warm-standby.html#SYNCHRONOUS-REPLICATION) and asynchronously replicated standby servers (the default)[https://www.postgresql.org/docs/current/warm-standby.html#STREAMING-REPLICATION]. When using synchronous replication, write instance must list the `DB_HOST_READER` instance(s) in its `synchronous_standby_names` configuration. This ensures ACID for both write and read instance.\n\nPostgreSQL's default asynchronous replication is supported by RMB by configuring `DB_HOST_ASYNC_READER` and `DB_PORT_ASYNC_READER`. Asynchronous replication is eventually consistent and suitable for read-only applications like reporting, analytics, and data warehousing. To use the async read host in queries, get an instance of `PostgresClient` using `PostgresClientWithAsyncReadConn.getInstance(...)`. If no async read host is configured, it falls back to the sync read host if configured, otherwise it uses the write host.\n\nAWS RDS does not support synchronous replication. For AWS it is recommended to only use `DB_HOST` and `DB_HOST_ASYNC_READER` in a given deployment.\n\nAPIs using the async client should provide a warning in the API documentation that the API uses stale data (for performance reasons).\n\n## Local development server\n\nTo get going quickly with running a local instance of Okapi, adding a tenant and some test data,\nand deploying some modules, see\n[Running a local FOLIO system](https://dev.folio.org/guides/run-local-folio/).\n\n## Creating a new module\n\n### Step 1: Create new project directory layout\n\nCreate the new project using the [normal layout](https://dev.folio.org/guides/commence-a-module/) of files, and basic POM file.\n\nAdd the `/ramls` directory, the area for the RAML, schemas, and examples files.\nFor a maven subproject the directory may be at the parent project only.\n\nTo get a quick start, copy the \"ramls\" directory and POM file from\n[mod-notify](https://github.com/folio-org/mod-notify).\n(At [Step 6](#step-6-design-the-raml-files) below, these will be replaced to suit your project's needs.)\n\nAdjust the POM file to match your project, e.g. artifactID, version, etc.\n\n### Step 2: Include the jars in your project pom.xml\n\n```xml\n \u003crepositories\u003e\n \u003crepository\u003e\n \u003cid\u003efolio-nexus\u003c/id\u003e\n \u003cname\u003eFOLIO Maven repository\u003c/name\u003e\n \u003curl\u003ehttps://repository.folio.org/repository/maven-folio\u003c/url\u003e\n \u003c/repository\u003e\n \u003c/repositories\u003e\n \u003cpluginRepositories\u003e\n \u003cpluginRepository\u003e\n \u003cid\u003efolio-nexus\u003c/id\u003e\n \u003cname\u003eFOLIO Maven repository\u003c/name\u003e\n \u003curl\u003ehttps://repository.folio.org/repository/maven-folio\u003c/url\u003e\n \u003c/pluginRepository\u003e\n \u003c/pluginRepositories\u003e\n \u003cdependencies\u003e\n \u003cdependency\u003e\n \u003cgroupId\u003eorg.folio\u003c/groupId\u003e\n \u003cartifactId\u003edomain-models-runtime\u003c/artifactId\u003e\n \u003cversion\u003e32.2.0\u003c/version\u003e\n \u003c/dependency\u003e\n \u003cdependency\u003e\n \u003cgroupId\u003eorg.folio\u003c/groupId\u003e\n \u003cartifactId\u003epostgres-testing\u003c/artifactId\u003e\n \u003cversion\u003e32.2.0\u003c/version\u003e\n \u003cscope\u003etest\u003c/scope\u003e\n \u003c/dependency\u003e\n ...\n ...\n \u003c/dependencies\u003e\n```\n\n(postgres-testing is available in version 32.2.0 and later)\n\n### Step 3: Add the plugins to your pom.xml\n\nFour plugins need to be declared in the POM file:\n\n- The `exec-maven-plugin` which will generate the POJOs and interfaces based on\n the RAML files.\n\n- The `aspectj-maven-plugin` which will pre-compile your code with validation aspects\n provided by the framework - remember the `@Validate` annotation. The\n validation supplied by the framework verifies that headers are passed\n correctly, parameters are of the correct type and contain the correct content\n as indicated by the RAML file.\n\n- The `maven-shade-plugin` which will generate a fat-jar runnable jar.\n The important thing to\n notice is the main class that will be run when running your module. Notice the\n `Main-class` and `Main-Verticle` in the shade plugin configuration.\n\n- The `maven-resources-plugin` which will copy the RAML files into a directory\n under `/apidocs` so that the runtime framework can pick it up and display html\n documentation based on the RAML files.\n\nAdd `ramlfiles_path` property indicating the location of the RAML directory.\n\n```xml\n \u003cproperties\u003e\n \u003cramlfiles_path\u003e${basedir}/ramls\u003c/ramlfiles_path\u003e\n \u003c/properties\u003e\n```\n\nCompare the POM with other FOLIO RMB-based modules.\n\n### Step 4: Build your project\n\nDo `mvn clean install`\n\nThis should:\n\n- Create java interfaces for each added RAML file.\n\n- Each interface will contain functions to be implemented (each function represents\n an API endpoint declared in the RAML).\n\n- The parameters within each function interface will be annotated with\n validation annotations that were declared in the RAML. So, if a trait was\n marked as mandatory, it will be marked as @NOT_NULL. This is not something that\n needs to be handled by the implementer. This is handled by the framework,\n which handles validation.\n\n- POJOs -- The JSON schemas will be generated into java objects.\n\n- All generated code can be found in the `org.folio.rest.jaxrs` package in the\n `target/generated-sources/raml-jaxrs/` directory.\n\n### Step 5: Implement the generated interfaces\n\nImplement the interfaces associated with the RAML files you created. An\ninterface is generated for every root endpoint in the RAML files.\nFor example an\n`org.folio.rest.jaxrs.resource.Ebooks` interface will be generated.\nNote that the `org.folio.rest.jaxrs.resource` will be the package for every\ngenerated interface.\n\nThe implementations must go into the `org.folio.rest.impl` package because RMB's\n[RestVerticle](https://github.com/folio-org/raml-module-builder/blob/master/domain-models-runtime/src/main/java/org/folio/rest/RestVerticle.java)\nscans this package for a class that implements the required interface. The class can\nhave any name.\nRMB then uses reflection to invoke the constructor and the method.\n\nSee [mod-notify's org.folio.rest.impl package](https://github.com/folio-org/mod-notify/tree/master/src/main/java/org/folio/rest/impl)\nfor example implementations.\n\n### Step 6: Design the RAML files\n\nIt is beneficial at this stage to take some time to design and prepare the RAML files for the project.\nInvestigate the other FOLIO modules for guidance.\nThe [mod-notify](https://github.com/folio-org/mod-notify/tree/master/ramls) is an exemplar.\n\nRemove the temporary copy of the \"ramls\" directory from Step 1, and replace with your own.\n\nThe end-points that share the same first path segment must go into the same .raml file\nbecause the first path segment determines the name of the resource .java interface, for example\n`/foo/bar` and `/foo/baz` should go into foo.raml, and foo.raml will generate\n`org/folio/rest/jaxrs/resource/Foo.java`. However, you may\n[overrule the convention for the resource interface name](https://github.com/mulesoft-labs/raml-for-jax-rs/issues/111)\nby implementing a GeneratorExtension.\n\nAdd the shared suite of [RAML utility](https://github.com/folio-org/raml) files\nas the \"raml-util\" directory inside your \"ramls\" directory:\n```\ngit submodule add https://github.com/folio-org/raml ramls/raml-util\n```\nThe \"raml1.0\" branch is the current and default branch.\n\nCreate JSON schemas indicating the objects exposed by the module.\n\nUse the `description` field alongside the `type` field to explain the content and\nusage and to add documentation.\n\nUse `\"javaType\": \"org.folio.rest.jaxrs.model.MyEntity\"` to set the class name of the\ngenerated Java type to prevent a duplicate name where the second class overwrites the first class\nor to avoid a misleading name. Otherwise the field name in the .json schema file is used as class name.\n\nSee [jsonschema2pojo Reference](https://github.com/joelittlejohn/jsonschema2pojo/wiki/Reference)\nfor JSON schema details.\n\nThe domain-models-maven-plugin automatically dereferences the schema files and places them into the\n`target/classes/ramls/` directory. It scans the `${basedir}/ramls/` directory including\nsubdirectories, if not found then `${basedir}/../ramls/` supporting maven submodules with\ncommon ramls directory.\n\nThe documentation of HTTP response codes\nis in [HttpStatus.java](util/src/main/java/org/folio/HttpStatus.java)\n\nUse the collection/collection-item pattern provided by the\n[collection resource type](https://github.com/folio-org/raml/tree/raml1.0/rtypes) explained\nin the [RAML 200 tutorial](https://raml.org/developers/raml-200-tutorial#resource-types).\n\nThe RMB does do some validation of RAML files at compile-time.\nThere are some useful tools to assist with command-line validation,\nand some can be integrated with text editors, e.g.\n[raml-cop](https://github.com/thebinarypenguin/raml-cop).\n\nSee the guide to [Use raml-cop to assess RAML, schema, and examples](https://dev.folio.org/guides/raml-cop/)\nand the [Primer for RAML and JSON Schema](https://dev.folio.org/start/primer-raml/) quick-start document.\n\nRAML-aware text editors are very helpful, such as\n[api-workbench](https://github.com/mulesoft/api-workbench) for Atom.\n\nRemember that the POM configuration enables viewing your RAML and interacting\nwith your application via the local [API documentation](#documentation-of-the-apis).\n\n## RestVerticle\n\nRestVerticle is implemented using the reactor pattern and Vert.x HttpServer and\nmust be started on the Vert.x event loop. Don't start it on a Vert.x worker context, this\n[may fail because of unsupported concurrency](https://issues.folio.org/browse/MODINVSTOR-635).\n\nUse VertxOptions\n[setMaxEventLoopExecuteTime](https://vertx.io/docs/apidocs/io/vertx/core/VertxOptions.html#setMaxEventLoopExecuteTime-long-)\nand\n[setWarningExceptionTime](https://vertx.io/docs/apidocs/io/vertx/core/VertxOptions.html#setWarningExceptionTime-long-)\nto adjust warnings like\n`Thread vertx-eventloop-thread-3 has been blocked for 20458 ms`\nand fix these violations of the [Golden Rule](https://vertx.io/docs/vertx-core/java/#golden_rule)\nthat make the module unresponsive. For details see [Running blocking code](https://vertx.io/docs/vertx-core/java/#blocking_code).\n\n## Adding an init() implementation\n\nIt is possible to add custom code that will run once before the application is deployed\n(e.g. to init a DB, create a cache, create static variables, etc.) by implementing\nthe `InitAPIs` interface. You must implement the\n`init(Vertx vertx, Context context, Handler\u003cAsyncResult\u003cBoolean\u003e\u003e resultHandler)`. Only one implementation per module is supported.\nCurrently the implementation should sit in the\n`org.folio.rest.impl` package in the implementing project. The implementation\nwill run during verticle deployment. The verticle will not complete deployment\nuntil the init() completes. The init() function can do anything basically, but\nit must call back the Handler. For example:\n\n```java\npublic class InitAPIs implements InitAPI {\n\n public void init(Vertx vertx, Context context, Handler\u003cAsyncResult\u003cBoolean\u003e\u003e resultHandler){\n try {\n sayHello();\n resultHandler.handle(io.vertx.core.Future.succeededFuture(true));\n } catch (Exception e) {\n e.printStackTrace();\n resultHandler.handle(io.vertx.core.Future.failedFuture(e.getMessage()));\n }\n }\n}\n```\n\nThe init may also modify HTTP server options by using `RestVerticle.getHttpServerOptions`.\n\n## Adding code to run periodically\n\nThis API can be used if your module *instance* needs to perform ongoing tasks.\nConsider using Okapi's\n[timer facility](https://github.com/folio-org/okapi/blob/master/doc/guide.md#timer-interface)\nif the task to be performed is \"per-tenant\"\nand only needs to be executed on one instance and not all instances of the module.\nIf the task performs operations on persistent storage, that is typically a sign\nthat it should be using the Okapi timer facility (not what is presented in this section).\n\nIt is possible to add custom code that will run periodically. For example,\nto ongoingly check status of something in the system and act upon that.\n\nNeed to implement the PeriodicAPI interface:\n\n```java\npublic interface PeriodicAPI {\n /** this implementation should return the delay in which to run the function */\n public long runEvery();\n /** this is the implementation that will be run every runEvery() milliseconds*/\n public void run(Vertx vertx, Context context);\n\n}\n```\n\nFor example:\n\n```java\npublic class PeriodicAPIImpl implements PeriodicAPI {\n\n\n @Override\n public long runEvery() {\n return 45000;\n }\n\n @Override\n public void run(Vertx vertx, Context context) {\n try {\n InitAPIs.amIMaster(vertx, context, v-\u003e {\n if(v.failed()){\n //TODO - what should be done here?\n }\n });\n } catch (Exception e) {\n e.printStackTrace();\n }\n }\n```\n\nThere can be multiple implementations of the periodic hook, all will be called by the RMB framework.\n\n## Adding a hook to run immediately after verticle deployment\n\nIt is possible to add custom code that will be run immediately after the verticle running the module is deployed.\n\n```java\npublic interface PostDeployVerticle {\n\n /** this implementation will be run immediately after the verticle is initially deployed. Failure does not stop\n * deployment success. The implementing function MUST call the resultHandler to pass back\n * control to the verticle, like so: resultHandler.handle(io.vertx.core.Future.succeededFuture(true));\n * if not, this function will hang the verticle during deployment */\n public void init(Vertx vertx, Context context, Handler\u003cAsyncResult\u003cBoolean\u003e\u003e resultHandler);\n\n}\n```\n\nAn implementation example:\n\n```java\npublic class InitConfigService implements PostDeployVerticle {\n\n @Override\n public void init(Vertx vertx, Context context, Handler\u003cAsyncResult\u003cBoolean\u003e\u003e handler) {\n\n System.out.println(\"Getting secret key to decode DB password.\");\n /** hard code the secret key for now - in production env - change this to read from a secure place */\n String secretKey = \"b2+S+X4F/NFys/0jMaEG1A\";\n int port = context.config().getInteger(\"http.port\");\n AdminClient ac = new AdminClient(\"http://localhost:\" + port, null);\n ac.postSetAESKey(secretKey, reply -\u003e {\n if(reply.statusCode() == 204){\n handler.handle(io.vertx.core.Future.succeededFuture(true));\n }\n else{\n handler.handle(io.vertx.core.Future.failedFuture(reply.statusCode() + \", \" + reply.statusMessage()));\n }\n });\n }\n\n}\n```\n\n## Adding a shutdown hook\n\nIt is possible to add custom code that will run just before the verticle is\nundeployed and the JVM stopped. This will occur on graceful shutdowns, but can\nnot be guaranteed to run if the JVM is forcefully shutdown.\n\nThe interface to implement:\n\n```java\npublic interface ShutdownAPI {\n\n public void shutdown(Vertx vertx, Context context, Handler\u003cAsyncResult\u003cVoid\u003e\u003e handler);\n\n}\n```\n\nAn implementation example:\n\n```java\npublic class ShutdownImpl implements ShutdownAPI {\n\n @Override\n public void shutdown(Vertx vertx, Context context, Handler\u003cAsyncResult\u003cVoid\u003e\u003e handler) {\n try {\n AuditLogger.getInstance().publish(new LogRecord(Level.INFO, \"closing audit logger\"));\n AuditLogger.getInstance().close();\n handler.handle(io.vertx.core.Future.succeededFuture());\n }\n catch (Exception e) {\n e.printStackTrace();\n handler.handle(io.vertx.core.Future.failedFuture(e.getMessage()));\n }\n }\n}\n```\n\n\n\nNote that when implementing the generated interfaces it is possible to add a constructor to the implementing class. This constructor will be called for every API call. This is another way you can implement custom code that will run per request.\n\n\n## Implementing uploads\n\nThe RMB allows for content to be streamed to a specific implemented interface.\nFor example, to upload a large file without having to save it all in memory:\n\n - Mark the function to handle the upload with the `org.folio.rest.annotations.Stream` annotation `@Stream`.\n - Declare the RAML as receiving `application/octet-stream`.\n\nExample RAML:\n\n```raml\n /uploadOctet:\n description: Uploads a file\n post:\n description: |\n Uploads a file\n body:\n application/octet-stream:\n```\n\n\nThe RMB will then call the function every time a chunk of data is received.\nThis means that a new Object is instantiated by the RMB for each chunk of\ndata, and the function of that object is called with the partial data included in a `java.io.InputStream` object.\n\nFor each invocation, RMB adds header `streamed_id` which will be unique\nfor the current stream. For the last invocation, header `complete` is supplied\nto indicate \"end-of-stream\".\n\nAs of RMB 23.12.0 and later, if an HTTP client prematurely closes the upload\nbefore complete, the handler will be called with `streamed_abort`.\n\n## Implement chunked bulk download\n\nRMB supports bulk downloads of chunks using [CQL](#cql-contextual-query-language) ordered by primary key id (since version 25).\n\n* 1st CQL query: `cql.allRecords=1 sortBy id`\n* 2nd CQL query: `id \u003e [last id from 1st CQL query] sortBy id`\n* 3rd CQL query: `id \u003e [last id from 2nd CQL query] sortBy id`\n* ...\n\nThe chunk size is set using the API's limit parameter, for example `limit=10000`\nfor chunks of 10000 records each.\n\n## PostgreSQL integration\n\nThe PostgreSQL connection parameters locations are searched in this order:\n\n- org.folio.rest.tools.utils.Envs.setEnv\n- [DB_* environment variables](#environment-variables)\n- Configuration file, defaults to `resources/postgres-conf.json` but can be set via [command-line options](#command-line-options)\n\nWith `PostgresClient.setPostgresTester`, testing may be performed against\nan instance implementing `postgresTester` interface, for example\n[TestContainers Postgres Module](https://www.testcontainers.org/modules/databases/postgres/).\nIf database configuration is already provided, this call is ignored and testing\nis performed against the database instance given by configuration.\n\nThe runtime framework exposes a PostgreSQL async client which offers CRUD\noperations in an ORM type fashion.\nhttps://github.com/folio-org/raml-module-builder/blob/master/domain-models-runtime/src/main/java/org/folio/rest/persist/PostgresClient.java\n\n**Important Note:** The PostgreSQL client currently implemented assumes\nJSONB tables in PostgreSQL. This is not mandatory and developers can work with\nregular PostgreSQL tables but will need to implement their own data access\nlayer.\n\n**Important Note:** For performance reasons the Postgres client will return accurate counts for result sets with less than 50,000 results. Queries with over 50,000 results will return an estimated count.\n\nThe PostgresClient expects tables in the following format:\n\n```sql\ncreate table \u003cschema\u003e.\u003ctable_name\u003e (\n id UUID PRIMARY KEY,\n jsonb JSONB NOT NULL\n);\n```\n\nThis means that all the fields in the JSON schema (representing the JSON object) **are** the \"jsonb\" (column) in the Postgres table.\n\n### Minimum PostgreSQL server version\n\nThe minimum PostgreSQL server version for RMB 35.3 is 16.0.0. RMB fails a POST /_/tenant call if the actual version is lower. You may set a higher or lower requirement using [server_version_num format](https://www.postgresql.org/docs/current/runtime-config-preset.html#GUC-SERVER-VERSION-NUM):\n\n```\npublic class TenantRefAPI extends TenantAPI {\n @Validate\n @Override\n public void postTenant(TenantAttributes tenantAttributes, Map\u003cString, String\u003e headers,\n Handler\u003cAsyncResult\u003cResponse\u003e\u003e handler, Context context) {\n // https://www.postgresql.org/docs/current/runtime-config-preset.html#GUC-SERVER-VERSION-NUM\n context.putLocal(\"postgres_min_version_num\", \"150000\");\n context.putLocal(\"postgres_min_version\", \"15.0\"); // human readable, for error message only\n super.postTenant(tenantAttributes, headers, handler, context);\n }\n}\n```\n\n### Saving binary data\n\nAs indicated, the PostgresClient is jsonb oriented. If there is a need to store data in binary form, this can be done in the following manner (only id based upsert is currently supported):\n```\nbyte[] data = ......;\nJsonArray jsonArray = new JsonArray().add(data);\nclient.upsert(TABLE_NAME, id, jsonArray, false, replyHandler -\u003e {\n.....\n});\n```\n\n### Securing DB Configuration file\n\nAs previously mentioned, the Postgres Client supplied by the RMB looks for a file called `postgres-conf.json`. However, leaving a file which contains the DB password to a superuser in plain text on the server is not a good idea. It is possible to encrypt the password in the file. The encryption should be an AES encryption (symmetric block cipher). This encryption is done with a secret key.\n\nMeaning: password in plain text + secret key = encrypted password\n\nThe RMB comes with an AES class that supports generating secret keys, encrypting and decrypting them, https://github.com/folio-org/raml-module-builder/blob/master/domain-models-runtime/src/main/java/org/folio/rest/security/AES.java\n\nNote that the use of this class is optional.\n\nTo work with an encrypted password the RMB exposes an API that can be used to set the secret key (stored only in memory). When creating the DB connection the RMB will check to see if the secret key has been set. If the secret key has been set, the RMB will decrypt the password with the secret key, and use the decrypted password to connect to the DB. Otherwise it will assume an un-encrypted password, and will connect using that password as-is.\nA module can also set the secret key via the static method `AES.setSecretKey(mykey)`\n\nThe needed steps are:\n\n - Generate a key\n - Encrypt a password\n - Include that password in the config file\n - Either call `AES.setSecretKey(mykey)` or the `admin/set_AES_key` API (to load the secret key into memory)\n\nA good way for a module to set the secret key is by using the post deployment hook interface in the RMB.\n\n```java\npublic class InitConfigService implements PostDeployVerticle {\n @Override\n public void init(Vertx vertx, Context context, Handler\u003cAsyncResult\u003cBoolean\u003e\u003e handler) {\n System.out.println(\"Getting secret key to decode DB password.\");\n //** hard code the secret key - in production env - read from a secure place *//\n String secretKey = \"b2%2BS%2BX4F/NFys/0jMaEG1A\";\n int port = context.config().getInteger(\"http.port\");\n AdminClient ac = new AdminClient(\"http://localhost:\" + port, null);\n ac.postSetAESKey(secretKey, reply -\u003e {\n if(reply.statusCode() == 204){\n handler.handle(io.vertx.core.Future.succeededFuture(true));\n }\n else{\n handler.handle(io.vertx.core.Future.failedFuture(reply.statusCode() + \", \" + reply.statusMessage()));\n }\n });\n handler.handle(io.vertx.core.Future.succeededFuture(true));\n }\n}\n```\n\n### Foreign keys constraint\n\nA `foreignKeys` entry in schema.json of the Tenant API automatically creates the following columns, triggers and indexes for the foreign key.\n\nThis additional column is needed because PostgreSQL does not directly support a foreign key constraint (referential integrity) of a field inside the JSONB.\n\nExample, similar to the SQL produced by a `foreignKeys` entry:\n\n```sql\nCREATE TABLE item (\n id UUID PRIMARY KEY,\n jsonb JSONB NOT NULL,\n permanentLoanTypeId UUID REFERENCES loan_type,\n temporaryLoanTypeId UUID REFERENCES loan_type\n);\nCREATE OR REPLACE FUNCTION update_item_references()\nRETURNS TRIGGER AS $$\nBEGIN\n NEW.permanentLoanTypeId = NEW.jsonb-\u003e\u003e'permanentLoanTypeId';\n NEW.temporaryLoanTypeId = NEW.jsonb-\u003e\u003e'temporaryLoanTypeId';\n RETURN NEW;\nEND;\n$$ language 'plpgsql';\nCREATE TRIGGER update_item_references\n BEFORE INSERT OR UPDATE ON item\n FOR EACH ROW EXECUTE PROCEDURE update_item_references();\nCREATE INDEX IF NOT EXISTS ON item (permanentLoanTypeId);\nCREATE INDEX IF NOT EXISTS ON item (temporaryLoanTypeId);\n```\n\nCQL2PgJSON automatically uses this extracted column and its index whenever the foreign key is used.\n\nThe overhead of this trigger and foreign key constraint reduces the number of UPDATE transactions per second on this table by about 10% (when tested against an external stand alone Postgres database). See\nhttps://github.com/folio-org/raml-module-builder/blob/master/domain-models-runtime/src/test/java/org/folio/rest/persist/ForeignKeyPerformanceIT.java\nfor the performance test. Doing the foreign key check manually by sending additional SELECT queries takes much more time than 10%.\n\nSee also [foreign key CQL support](#cql2pgjson-foreign-key-cross-table-index-queries).\n\n## CQL (Contextual Query Language)\n\nFurther [CQL](https://dev.folio.org/reference/glossary/#cql) information.\n\n### CQL2PgJSON: CQL to PostgreSQL JSON converter\n\nThe source code is at [./cql2pgjson](cql2pgjson) and [./cql2pgjson-cli](cql2pgjson-cli)\n\n### CQL2PgJSON: Usage\n\nInvoke like this:\n\n```java\n// users.user_data is a JSONB field in the users table.\nCQL2PgJSON cql2pgJson = new CQL2PgJSON(\"users.user_data\");\nString cql = \"name=Miller\";\nString where = cql2pgJson.cql2pgJson(cql);\nString sql = \"select * from users where \" + where;\n// select * from users\n// where CAST(users.user_data-\u003e'name' AS text)\n// ~ '(^|[[:punct:]]|[[:space:]])Miller($|[[:punct:]]|[[:space:]])'\n```\n\nOr use `toSql(String cql)` to get the `ORDER BY` clause separately:\n\n```java\nCQL2PgJSON cql2pgJson = new CQL2PgJSON(\"users.user_data\");\nString cql = \"name=Miller\";\nSqlSelect sqlSelect = cql2pgJson.toSql(cql);\nString sql = \"select * from users where \" + sqlSelect.getWhere()\n + \" order by \" + sqlSelect.getOrderBy();\n```\n\nSetting server choice indexes is possible, the next example searches `name=Miller or email=Miller`:\n\n```java\nCQL2PgJSON cql2pgJson = new CQL2PgJSON(\"users.user_data\", Arrays.asList(\"name\", \"email\"));\nString cql = \"Miller\";\nString where = cql2pgJson.cql2pgJson(cql);\nString sql = \"select * from users where \" + where;\n```\n\nSearching across multiple JSONB fields works like this. The _first_ json field specified\nin the constructor will be applied to any query arguments that aren't prefixed with the appropriate\nfield name:\n\n```java\n// Instantiation\nCQL2PgJSON cql2pgJson = new CQL2PgJSON(Arrays.asList(\"users.user_data\",\"users.group_data\"));\n\n// Query processing\nwhere = cql2pgJson.cql2pgJson( \"users.user_data.name=Miller\" );\nwhere = cql2pgJson.cql2pgJson( \"users.group_data.name==Students\" );\nwhere = cql2pgJson.cql2pgJson( \"name=Miller\" ); // implies users.user_data\n```\n\n### CQL: Field names\n\nThe field names (keys in JSON) are case sensitive. This is against the CQL specification of index names.\n\n### CQL: Relations\n\nOnly these relations have been implemented yet:\n\n* `=` (this is `==` for number matching and `adj` for a string matching.\n Examples 1: `height =/number 3.4` Example 2: `title = Potter`)\n* `==` (field match, for example `barcode == 883746123` or field prefix match `title == \"Harry Pott*\"`\n matching \"Harry Potter and the chamber of secrets\" but not \"Science of Harry Potter\";\n `==/number` matches any form: 3.4 = 3.400 = 0.34e1)\n* `all` (each word of the query string exists somewhere, `title all \"Potter Harry\"` matches \"Harry X. Potter\")\n* `any` (any word of the query string exists somewhere, `title any \"Potter Foo\"` matches \"Harry Potter\")\n* `adj` (substring phrase match: all words of the query string exist consecutively in that order, there may be any\n whitespace and punctuation in between, `title adj \"Harry Potter\"` matches \"The Harry - . - Potter Story\")\n* `\u003e` `\u003e=` `\u003c` `\u003c=` `\u003c\u003e` (comparison for both strings and numbers)\n\nNote to mask the CQL special characters by prepending a backslash: * ? ^ \" \\\n\nExample using [StringUtil](util/src/main/java/org/folio/util/StringUtil.java):\n\n```java\nString query = \"username==\" + StringUtil.cqlEncode(username);\nString url = \"https://example.com/users?query=\" + PercentCodec.encode(query);\n```\n\nUse quotes if the search string contains a space, for example `title = \"Harry Potter\"`.\n\nTo speed up the `==` field matching use a b-tree `\"index\"` or `\"uniqueIndex\"`.\n\nTo speed up the word matching use a `\"fullTextIndex\"`.\n\nThe `\"caseSensitive\"` and `\"removeAccents\"` configuration of the supporting index in schema.json is\nused for all string matching operators; `\"fullTextIndex\"` is always `\"caseSensitive\": false`.\n\nRefer to further explanation of [CQL string matching](https://dev.folio.org/faqs/explain-cql/)\nfor the \"Exact match operator\" and \"Word match operators\".\n\n### CQL: Modifiers\n\nMatching modifiers: Only `masked` is implemented, not `unmasked`, `regexp`,\n`honorWhitespace`, `substring`.\n\nWord begin and word end in JSON is only detected at whitespace and punctuation characters\nfrom the ASCII charset, not from other Unicode charsets.\n\n### CQL: Matching, comparing and sorting numbers\n\nAdd the /number modifier to enable number matching, comparing and sorting, for example `age ==/number 18`,\n`age \u003e=/number 21` and `sortBy age/number`.\n\n3.4, 3.400, and 0.34e1 match each other when using `==/number`, and 2 is smaller than 19\n(in contrast to string comparison where \"2\" \u003e \"19\").\n\nThis requires that the value has been stored as a JSONB number (`{\"age\": 19}`)\nand not as a JSONB string (`{\"age\": \"19\"}`).\n\n### CQL: Matching id and foreign key fields\n\nThe id field and any foreign key field declared in schema.json is a UUID field\nand is not searched in the JSONB but in an extracted proper database table field.\nAn index is automatically created for such a field,\ndo not add an index entry in schema.json.\n\n`=`, `==`, `\u003c\u003e`, `\u003e`, `\u003e=`, `\u003c`, and `\u003c=` relations are supported for comparison with a valid UUID and use the index.\n\n`=`, `==`, and `\u003c\u003e` relations allow `*` for right truncation with index support.\n\n`=` is interpreted as `==` for the id field and foreign key fields declared in schema.json; for other fields\nthat contain a UUID `=` results in a word full text match. Therefore `=` should be avoided and `==`\nbeen used for all UUID fields.\n\nModifiers are forbidden.\n\n### CQL: Matching full text\n\nSee [PostgreSQL's tsvector full text parser documentation](https://www.postgresql.org/docs/current/textsearch-parsers.html)\nhow word splitting works when using a full text index. Some notable consequences:\n\nCQL `field adj \"bar\"` matches `bar`, `bar-baz`, `foo-bar-baz`.\n\nCQL `field adj \"bar baz\"` matches `bar baz`, `bar-baz`, `foo-bar-baz`, `foo-bar baz`, `bar-baz-foo`.\n\nCQL `field adj \"bar-baz\"` matches `bar-baz`, but neither `bar baz` nor `foo-bar-baz` nor `foo-bar baz` nor `bar-baz-foo`.\n\nCQL `field adj \"123 456\"` matches `123 456`, but not `123-456`.\n\nCQL `field adj \"123-456\"` matches `123-456`, but not `123 456`.\n\n`foo/bar/baz` is a single word, while `foo//bar//baz`, `foo///bar///baz`, `foo////bar////baz`, etc.\nare split into the three words `foo`, `/bar`, and `/baz` (always reduced to a single slash).\n\n### CQL: Matching all records\n\nA search matching all records in the target index can be executed with a\n`cql.allRecords=1` (CQL standard) or a `id=*` (RMB specific) query. They can be used alone or as part of\na more complex query, for example\n`cql.allRecords=1 NOT name=Smith sortBy name/sort.ascending`\n\n* `cql.allRecords=1 NOT name=Smith` matches all records where name does not contain Smith\n as a word or where name is not defined.\n* `name=\"\" NOT name=Smith` matches all records where name is defined but does not contain\n Smith as a word.\n* For performance reasons, searching for `*` in any fulltext field will match all records as well,\n including records where that field does not exist.\n\n### CQL: Matching undefined or empty values\n\nA relation does not match if the value on the left-hand side is undefined. (but see the fulltext\n`*` case above).\nA negation (using NOT) of a relation matches if the value on the left-hand side is\nnot defined or if it is defined but doesn't match.\n\n* `name=\"\"` matches all records where name is defined.\n* `cql.allRecords=1 NOT name=\"\"` matches all records where name is not defined.\n* `name==\"\"` matches all records where name is defined and empty.\n* `cql.allRecords=1 NOT name==\"\"` matches all records where name is defined and not empty or\n where name is not defined.\n* `name=\"\" NOT name==\"\"` matches all records where name is defined and not empty.\n\n### CQL: Matching array elements\n\nFor matching the elements of an array use these queries (assuming that lang is either an array or not defined, and assuming\nan array element value does not contain double quotes):\n* `lang == \"[]\"` for matching records where lang is defined and an empty array\n* `cql.allRecords=1 NOT lang \u003c\u003e \"[]\"` for matching records where lang is not defined or an empty array\n* `lang == \"*\\\"en\\\"*\"` for matching records where lang is defined and contains the value en\n* `cql.allRecords=1 NOT lang == \"*\\\"en\\\"*\"` for matching records where lang does not\n contain the value en (including records where lang is not defined)\n* `lang = \"\" NOT lang == \"*\\\"en\\\"*\"` for matching records where lang is defined and\n and does not contain the value en\n* `lang = \"\"` for matching records where lang is defined\n* `cql.allRecords=1 NOT lang = \"\"` for matching records where lang is not defined\n\n### CQL: @-relation modifiers for array searches\n\nRMB 26 or later supports array searches with relation modifiers, that\nare particular suited for structures like:\n\n```json5\n\"property\" : [\n {\n \"type1\" : \"value1\",\n \"type2\" : \"value2\",\n \"subfield\": \"value\"\n },\n ...\n]\n```\n\nAn example of this kind of structure is `contributors ` (property) from\nmod-inventory-storage . `contributorTypeId` is the type of contributor\n(type1).\n\nWith CQL you can limit searches to `property` with regular match in\n`subfield`, with type1=value1 with\n\n```\nproperty =/@type1=value1 value\n```\n\nObserve that the relation modifier is preceded with the @-character to\navoid clash with other CQL relation modifiers.\n\nThe type1, type2 and subfield must all be defined in schema.json, because\nthe JSON schema is not known. And also because relation modifiers are\nunfortunately lower-cased by cqljava. To match value1 against the\nproperty contents of type1, full-text match is used.\n\nMultiple relation modifiers with value are ANDed together. So\n\n```\nproperty =/@type1=value1/@type2=value2 value\n```\n\nwill only give a hit if both type1 has value1 AND type2 has value2.\n\nIt is also possible to specify relation modifiers without value. This\nessentially is a way to override what subfield to search. In this case\nthe right hand side term is matched. Multiple relation modifiers\nare OR'ed together. For example:\n\n```\nproperty =/@type1 value\n```\n\nAnd to match any of the sub properties type1, type2, you could use:\n\n```\nproperty =/@type1/@type2 value\n```\n\nTo enable the `@`-relation modifiers for an array the module must define\nit in schema.json. The properties `arraySubfield` and `arrayModifiers`\nspecify the subfield and the list of modifiers respectively.\nThis can be applied to `ginIndex` or `fullTextIndex`.\nschema.json example:\n\n```json5\n{\n \"fieldName\": \"property\",\n \"tOps\": \"ADD\",\n \"caseSensitive\": false,\n \"removeAccents\": true,\n \"arraySubfield\": \"subfield\",\n \"arrayModifiers\": [\"type1\", \"type2\"]\n}\n```\n\nFor the identifiers example we could define things in schema.json with:\n```json5\n{\n \"fieldName\": \"identifiers\",\n \"tOps\": \"ADD\",\n \"arraySubfield\": \"value\",\n \"arrayModifiers\": [\"identifierTypeId\"]\n}\n```\nThis will allow you to perform searches, such as:\n```\nidentifiers = /@identifierTypeId=7e591197-f335-4afb-bc6d-a6d76ca3bace 6316800312\n```\n\n### CQL2PgJSON: Multi Field Index\n\nCQL2PGjson allows generating and querying indexes that contain multiple columns. The index json object now has support for the following properties:\n* `sqlExpression`\n\tAllows the user to explicitly define the expression they wish to use in the index\n\t```json5\n \"fieldName\": \"address\",\n \"sqlExpression\": \"concat_space_sql(jsonb-\u003e\u003e'city', jsonb-\u003e\u003e'state')\",\n\t```\n\n* `multiFieldNames`\n\tThis is a comma-separated list of json fields that are to be concatenated together via concat_ws with a space character.\n\texample:\n\t```json5\n\t\t\"fieldName\": \"address\",\n\t\t\"multiFieldNames\": \"city,state\",\n\t```\nThese 2 examples are equivalent and would be queried by using the fieldName such as:\n\n```\naddress = Boston MA\n```\n\nThese fields are optional but mutually exclusive, you only need one of them.\n\n\n### CQL2PgJSON: Foreign key cross table index queries\n\nCQL2PgJSON supports cross table joins via subquery based on foreign keys.\nThis allows arbitrary depth relationships in both child-to-parent (many-to-one) and parent-to-child (one-to-many) direction.\n\nExample relationship: item → holdings_record → instance\n\nJoin conditions of this example:\n* item.holdingsRecordId = holdings_record.id\n* holdings_record.instanceId = instance.id\n\nThe field in the child table points to the primary key `id` field of the parent table; the parent table is also called the target table.\n\n* Precede the index you want to search, with the table name in camelCase, e.g. `instance.title = \"bee\"`.\n* There is no change with child table fields. Use them in the regular way without table name prefix.\n* The `foreignKey` entry in schema.json automatically creates an index on the foreign key field.\n* For fast queries, declare an index on any other searched field like `title` in the schema.json file.\n* For a multi-table join, use `targetPath` instead of `fieldName` and put the list of field names into the `targetPath` array.\n The `targetPath` array must be in child-to-parent direction (many-to-one), e.g. item → holdings_record → instance, queries are possible in both directions.\n* When querying in parent → child direction each sub-expression may refer to a different child/grandchild/...:\nSearching for instances using `item.status.name==\"Missing\" and item.effectiveLocationId=\"fcd64ce1-6995-48f0-840e-89ffa2288371\"`\nwill look for an instance that has at least one item that is missing and this or some other item of the instance has the effective location.\nUse a child-to-parent query against the item endpoint if both conditions should apply to the same item record.\n* Use `= *` to check whether a join record exists. This runs a cross index join with no further restriction, e.g. `instance.id = *`.\n* The sortBy clause doesn't support foreign table fields. Use the API endpoint of the records with the field you want to sort on.\n* The schema for the above example:\n```json\n{\n \"tables\": [\n {\n \"tableName\": \"instance\",\n \"index\": [\n {\n \"fieldName\": \"title\",\n \"tOps\": \"ADD\",\n \"caseSensitive\": false,\n \"removeAccents\": true\n }\n ]\n },\n {\n \"tableName\": \"holdings_record\",\n \"foreignKeys\": [\n {\n \"fieldName\": \"instanceId\",\n \"targetTable\": \"instance\",\n \"targetTableAlias\": \"instance\",\n \"tableAlias\": \"holdingsRecord\",\n \"tOps\": \"ADD\"\n }\n ]\n },\n {\n \"tableName\": \"item\",\n \"foreignKeys\": [\n {\n \"fieldName\": \"holdingsRecordId\",\n \"targetTable\": \"holdings_record\",\n \"targetTableAlias\": \"holdingsRecord\",\n \"tableAlias\": \"item\",\n \"tOps\": \"ADD\"\n },\n {\n \"targetPath\": [\"holdingsRecordId\", \"instanceId\"],\n \"targetTable\": \"instance\",\n \"targetTableAlias\": \"instance\",\n \"tableAlias\": \"item\"\n }\n ]\n }\n ]\n}\n```\n\n### CQL2PgJSON: Foreign key tableAlias and targetTableAlias\n\nThe property `targetTableAlias` enables that parent table name in CQL queries against the current child table.\n\nThe property `tableAlias` enables that child table name in CQL queries against the target/parent table.\n\nIf any of these two properties is missing, then that respective foreign key join syntax is disabled.\n\nThe name may be different from the table name (`tableName`, `targetTable`). One use case is to change to camelCase, e.g.\n`\"targetTable\": \"holdings_record\"` and `\"targetTableAlias\": \"holdingsRecord\"`. Another use case is\nto resolve ambiguity when two foreign keys point to the same target table, example:\n```json\n{\n \"tableName\": \"item\",\n \"foreignKeys\": [\n {\n \"fieldName\": \"permanentLoanTypeId\",\n \"tableAlias\": \"itemWithPermanentLoanType\",\n \"targetTable\": \"loan_type\",\n \"targetTableAlias\": \"loanType\",\n \"tOps\": \"ADD\"\n },\n {\n \"fieldName\": \"temporaryLoanTypeId\",\n \"tableAlias\": \"itemWithTemporaryLoanType\",\n \"targetTable\": \"loan_type\",\n \"targetTableAlias\": \"temporaryLoanType\",\n \"tOps\": \"ADD\"\n }\n ]\n}\n```\nRunning CQL `loanType.name == \"Can circulate\"` against the item endpoint returns all items where the item's permanentLoanTypeId points to a loan_type where the loan_type's name equals \"Can circulate\".\n\nRunning CQL `temporaryLoanType.name == \"Can circulate\"` against the item endpoint returns all items where the item's temporaryLoanTypeId points to a loan_type where the loan_type's name equals \"Can circulate\".\n\nRunning CQL `itemWithPermanentLoanType.status == \"In transit\"` against the loan_type endpoint returns all loan_types where there exists an item that has this loan_type as a permanentLoanType and where the item's status equals \"In transit\".\n\nRunning CQL `itemWithTemporaryLoanType.status == \"In transit\"` against the loan_type endpoint returns all loan_types where there exists an item that has this loan_type as a temporaryLoanType and where the item's status equals \"In transit\".\n\n### CQL2PgJSON: Exceptions\n\nAll locally produced Exceptions are derived from a single parent so they can be caught collectively\nor individually. Methods that load a JSON data object model pass in the identity of the model as a\nresource file name, and may also throw a native `java.io.IOException`.\n\n```\nCQL2PgJSONException\n ├── FieldException\n ├── SchemaException\n ├── ServerChoiceIndexesException\n ├── CQLFeatureUnsupportedException\n └── QueryValidationException\n └── QueryAmbiguousException\n```\n\n### CQL2PgJSON: Unit tests\n\nTo run the unit tests in your IDE, the Unicode input files must have been produced by running maven.\nIn Eclipse you may use \"Run as ... Maven Build\" for doing so.\n\n## Tenant API\n\nThe Postgres Client support in the RMB is schema specific, meaning that\nit expects every tenant to be represented by its own schema. The Tenant\nAPI is asynchronous as of RMB 32 and later. The tenant job is initiated\nby a POST and may be inspected with GET and optionally be cleaned up\nwith DELETE.\n\nThe RAML defining the API:\n\n https://github.com/folio-org/raml/blob/tenant_v2_0/ramls/tenant.raml\n\nBy default RMB includes an implementation of the Tenant API which assumes\nPostgres being present.\nImplementation in\n [TenantAPI.java](https://github.com/folio-org/raml-module-builder/blob/master/domain-models-runtime/src/main/java/org/folio/rest/impl/TenantAPI.java) file. You might want to extend/override this because:\n\n1. You want to not call it at all (your module is not using Postgres).\n2. You want to provide further Tenant control, such as loading reference and/or sample data.\n\n#### Extending the Tenant API\n\nSince RMB 32, the postTenant that is provided is working in\nan asynchronous fashion. The handler provided will be called\nvery early in the process and before `schema.json` is fully processed.\nFor this reason doing anything in the handler is problematic as the\ndatabase is not fully populated with data. Furthermore, if there are\nany errors that could occur in your post handler, it seems natural to\nthrow an error, but the underlying RMB code does not know about that, and\nthus, continues its operation - including managing a job in the background.\n\nHere's an example of incorrect usage.\nDO NOT DO THIS:\n\n```java\n super.postTenant(entiry, headers, ar -\u003e {\n LiquibaseUtil.initializeSchemaForTenant(vertx, tenantId);\n if (someerror) {\n handler.handle(Future.succeededFuture(\n PostTenantResponse.respond400TextPlain(...)));\n }\n }, context);\n```\n\nThe `LiquibaseUtil.initializeSchemaForTenant`\nassumes that the database is configured. The handler is called when the module\nis invoked always (on purge, disable as well). Throwing error is problematic\nbecause postTenant has already done something. And, finally, the status\ncode of `ar` is not even inspected.\n\nFor these reasons, in most cases, it is discouraged to call\n`super.postTenant`. Instead extend the `loadData` method (see below).\n\nIf your module - based on RMB - does not use a storage then there is not\na problem - you must not call `super.postTenant` and you can just\nimplement an easy postTenant implementation in sync fashion -\nreturning 204 for OK/No Content.\n\nExtend the `loadData` method, to load sample/reference data for a module\nor do any other initialiation of a module - such as doing LiquibaseUtil\nwork. The loadData is called upon create/upgrade.\n\n\n```java\n@Validate\n@Override\nFuture\u003cInteger\u003e loadData(TenantAttributes attributes, String tenantId,\n Map\u003cString, String\u003e headers, Context vertxContext) {\n return super.loadData(attributes, tenantId, headers, vertxContext)\n .compose(superRecordsLoaded -\u003e {\n // load n records or any other initialization\n return Future.succeededFuture(superRecordsLoaded + n);\n });\n}\n```\n\nThe Integer here is the number of records loaded - just add 0 if\nno sample/ref is loaded.\n\nThere is no right way to load data, but consider that data load will be\nboth happening for first time tenant usage of the module and during an\nupgrade process. Your data loading should be idempotent. If files are stored\nas resources and as JSON files, you can use the TenantLoading utility.\n\n```java\nimport org.folio.rest.tools.utils.TenantLoading;\n\n@Validate\n@Override\nFuture\u003cInteger\u003e loadData(TenantAttributes attributes, String tenantId,\n Map\u003cString, String\u003e headers, Context vertxContext) {\n return super.loadData(attributes, tenantId, headers, vertxContext)\n .compose(recordsLoaded -\u003e new TenantLoading()\n // two sets of reference data files\n // resources ref-data/data1 and ref-data/data2 .. loaded to\n // okapi-url/instances and okapi-url/items respectively\n .withKey(\"loadReference\").withLead(\"ref-data\")\n .withIdContent()\n .add(\"data1\", \"instances\")\n .add(\"data2\", \"items\")\n .perform(attributes, headers, vertxContext, recordsLoaded));\n}\n```\n\nIf data is already in resources, then fine. If not, then copy it with maven-resource-plugin.\nFor example, to copy `reference-data` to `ref-data` in resources:\n\n```xml\n\u003cexecution\u003e\n \u003cid\u003ecopy-reference-data\u003c/id\u003e\n \u003cphase\u003eprocess-resources\u003c/phase\u003e\n \u003cgoals\u003e\n \u003cgoal\u003ecopy-resources\u003c/goal\u003e\n \u003c/goals\u003e\n \u003cconfiguration\u003e\n \u003coutputDirectory\u003e${basedir}/target/classes/ref-data\u003c/outputDirectory\u003e\n \u003cresources\u003e\n \u003cresource\u003e\n \u003cdirectory\u003e${basedir}/reference-data\u003c/directory\u003e\n \u003cfiltering\u003etrue\u003c/filtering\u003e\n \u003c/resource\u003e\n \u003c/resources\u003e\n \u003c/configuration\u003e\n\u003c/execution\u003e\n```\n\n#### Unit testing\n\nFor unit testing method `TenantAPI.postTenantSync` may be used\nto initialize the tenant. This method is a straight API call and is\nnot called via HTTP. Use it before/after using our own provided\ninterfaces in `org.folio.rest.impl`. Example:\n\n```java\n// create\nTenantAPI tenantAPI = new TenantAPI();\nTenantAttributes tenantAttributes = new TenantAttributes();\ntenantAttributes.setModuleTo(\"mod-2.0.0\");\ntenantAPI.postTenantSync(tenantAttributes, okapiHeaders, context.asyncAssertSuccess(result -\u003e\n assertThat(result.getStatus(), is(204))\n), vertx.getOrCreateContext());\n```\n\nA HTTP based alternatives is `TenantInit.exec` with a similar interface. Example:\n```java\nTenantClient client = new TenantClient(....);\nTenantAttributes tenantAttributes = new TenantAttributes().withModuleTo(\"mod-2.0.0\");\nTenantInit.exec(client, tenantAttributes, 60000).onComplete(context.asyncAssertSuccess());\n```\n\n#### The Post Tenant API\n\nThe Postgres based Tenant API implementation will look for a file at `/resources/templates/db_scripts/`\ncalled **schema.json**\n\nThe file contains an array of tables and views to create for a tenant on registration (tenant api post)\n\nAn example can be found here:\n\n - https://github.com/folio-org/raml-module-builder/blob/master/domain-models-runtime/src/main/resources/templates/db_scripts/examples/schema.json.example.json\n\nThe top level properties in schema.json (some of which are optional) are `scripts`, `tables`, `views` and `exactCount`.\n\nEntries in the json file to be aware of:\n\nFor each **table** in `tables` property:\n\n1. `tableName` - name of the table that will be generated - this is the table that should be referenced from the code. Maximum length is 49 characters.\n2. `generateId` - No longer supported. This functionality is not stable in Pgpool-II see https://www.pgpool.net/docs/latest/en/html/restrictions.html. The solution is to generate a UUID in java in the same manner as https://github.com/folio-org/raml-module-builder/blob/v23.11.0/domain-models-runtime/src/main/java/org/folio/rest/persist/PgUtil.java#L358\n3. `fromModuleVersion` - this optional field indicates the version in which the table was created / updated in. When a tenant update is requested - only versions older than the indicated version will generate the declared table. This ensures that if a module upgrades from an older version, the needed tables will be generated for it, however, subsequent upgrades from versions equal or later than the version indicated for the table will not re-generate the table.\n * Note that this is enforced for all tables, views, indexes, FK, triggers, etc. (via the `IF NOT EXISTS` sql Postgres statement)\n4. `mode` - should be used only to indicate `delete`\n5. `withMetadata` - will generate the needed triggers to populate the metadata section in the json on update / insert\n6. `likeIndex` - indicate which fields in the json will be queried using the `LIKE`. Needed for fields that will be faceted on.\n * `fieldName` the field name in the json for which to create the index, maximum name length is 49 characters.\n * the `tOps` indicates the table operation - ADD means to create this index, DELETE indicates this index should be removed\n * the `caseSensitive` allows you to create case insensitive indexes (boolean true / false), if you have a string field that may have different casings and you want the value to be unique no matter the case. Defaults to false.\n * `removeAccents` - normalize accents or leave accented chars as is. Defaults to true.\n * the `whereClause` allows you to create partial indexes, for example: \"whereClause\": \"WHERE (jsonb-\u003e\u003e'enabled')::boolean = true\"\n * `stringType` - defaults to true - if this is set to false than the assumption is that the field is not of type text therefore ignoring the removeAccents and caseSensitive parameters.\n * `arrayModifiers` - specifies array relation modifiers supported for some index. The modifiers must exactly match the name of the property in the JSON object within the array.\n * `arraySubfield` - is the key of the object that is used for the primary term when array relation modifiers are in use. This is typically also defined when `arrayModifiers` are also defined.\n * `multiFieldNames` - see [CQL2PgJSON: Multi Field Index](#cql2pgjson-multi-field-index) above\n * `sqlExpression` - see [CQL2PgJSON: Multi Field Index](#cql2pgjson-multi-field-index) above\n * `sqlExpressionQuery` - How to wrap the query string $ before matching against the index expression. This overwrites any `caseSensitive` and `removeAccents` wrapping. Example that manually implements case insensitive matching: `\"sqlExpression\": \"lower(jsonb-\u003e\u003e'name')\", \"sqlExpressionQuery\": \"lower($)\"`. Example for plain field, note the required braces: `\"sqlExpression\": \"(jsonb-\u003e\u003e'name')\", \"sqlExpressionQuery\": \"$\"`.\n * Do not manually add an index for an `id` field or a foreign key field. They get indexed automatically.\n7. `ginIndex` - generate an inverted index on the JSON using the `gin_trgm_ops` extension. Allows for left and right truncation LIKE queries and regex queries to run in an optimal manner (similar to a simple search engine). Note that the generated index is large and does not support the full field match (SQL `=` operator and CQL `==` operator without wildcards). See the `likeIndex` for available options.\n8. `uniqueIndex` - create a unique index on a field in the JSON\n * the `tOps` indicates the table operation - ADD means to create this index, DELETE indicates this index should be removed\n * the `whereClause` allows you to create partial indexes, for example: \"whereClause\": \"WHERE (jsonb-\u003e\u003e'enabled')::boolean = true\"\n * Adding a record will fail if the b-tree index byte size limit of 2712 is exceeded. Consider enforcing a length limit on the field, for example by adding a 600 character limit (multi-byte characters) to the RAML specification.\n * See additional options in the likeIndex section above\n9. `index` - create a btree index on a field in the JSON\n * the `tOps` indicates the table operation - ADD means to create this index, DELETE indicates this index should be removed\n * the `whereClause` allows you to create partial indexes, for example: \"whereClause\": \"WHERE (jsonb-\u003e\u003e'enabled')::boolean = true\"\n * See additional options in the likeIndex section above\n * The expression is wrapped into left(..., 600) to prevent exceeding the b-tree byte size limit of 2712. Special case: sqlExpression is not wrapped.\n10. `fullTextIndex` - create a full text index using the tsvector features of postgres.\n * `removeAccents` can be used, the default `caseSensitive: false` cannot be changed because tsvector always converts to lower case.\n * See [CQL: Matching full text](#cql-matching-full-text) to learn how word splitting works.\n * The `tOps` is optional (like for all indexes), and defaults to ADDing the index.\n * `whereClause` and `stringType` work as for `likeIndex` above.\n11. `withAuditing` - Creates an auditing table and a trigger that populates the audit table with the history of the table record whenever an insert, update, or delete occurs. `\"withAuditing\": true` for enabled, `false` or undefined for disabled.\n * `auditingTableName` The name of the audit table.\n * `auditingFieldName` The field (JSON property) in the audit record that contains the copy of the original record.\n * `\"withAuditing\": true` automatically creates the auditing table; an entry of the audit table in the \"tables\" section of schema.json is optional, for example to create indexes.\n * The `auditingSnippet` section allows some customizations to the auditing function with custom SQL in the declare section and the body (for either insert / update / delete).\n * The audit table jsonb column has three fields: `$auditingFieldName` contains the original record (jsonb from the original table), `id` contains a new unique id, `operation` contains `I`, `U`, `D` for insert, update, delete, and `createdDate` contains the time when the audit record was created.\n12. `foreignKeys` - adds / removes foreign keys (trigger populating data in a column based on a field in the JSON and creating a FK constraint)\n13. `customSnippetPath` - a relative path to a file with custom SQL commands for this specific table\n14. `deleteFields` / `addFields` - delete (or add with a default value), a field at the specified path for all JSON entries in the table\n15. `populateJsonWithId` - This schema.json entry and the disable option is no longer supported. The primary key is always copied into `jsonb-\u003e'id'` on each insert and update.\n16. `pkColumnName` - No longer supported. The name of the primary key column is always `id` and is copied into `jsonb-\u003e'id'` in each insert and update. The method PostgresClient.setIdField(String) no longer exists.\n17. `withOptimisticLocking` - `off` (default), `logOnConflict`, `failOnConflictUnlessSuppressed`, or `failOnConflict`, for details see [Optimistic Locking section](#optimistic-locking) below\n\nThe **views** section is a bit more self explanatory, as it indicates a viewName and the two tables (and a column per table) to join by. In addition to that, you can indicate the join type between the two tables. For example:\n```json\n\"views\": [\n {\n \"viewName\": \"items_mt_view\",\n \"join\": [\n {\n \"table\": {\n \"tableName\": \"item\",\n \"joinOnField\": \"materialTypeId\"\n },\n \"joinTable\": {\n \"tableName\": \"material_type\",\n \"joinOnField\": \"id\",\n \"jsonFieldAlias\": \"mt_jsonb\"\n }\n }\n ]\n }\n]\n```\nBehind the scenes this will produce the following statement which will be run as part of the schema creation:\n\n CREATE OR REPLACE VIEW ${tenantid}_${module_name}.items_mt_view AS\n SELECT u.id, u.jsonb as jsonb, g.jsonb as mt_jsonb\n FROM ${tenantid}_${module_name}.item u\n JOIN ${tenantid}_${module_name}.material_type g\n ON lower(f_unaccent(g.jsonb-\u003e\u003e'id')) = lower(f_unaccent(u.jsonb-\u003e\u003e'materialTypeId'))\n\nNotice the `lower(f_unaccent(` functions, currently, by default, all string fields will be wrapped in these functions (will change in the future).\n\nA three table join would look something like this:\n\n```json\n{\n \"viewName\": \"instance_holding_item_view\",\n \"join\": [\n {\n \"table\": {\n \"tableName\": \"instance\",\n \"joinOnField\": \"id\"\n },\n \"joinTable\": {\n \"tableName\": \"holdings_record\",\n \"joinOnField\": \"instanceId\",\n \"jsonFieldAlias\": \"ho_jsonb\"\n }\n },\n {\n \"table\": {\n \"tableName\": \"holdings_record\",\n \"joinOnField\": \"id\",\n \"jsonFieldAlias\": \"ho2_jsonb\"\n },\n \"joinTable\": {\n \"tableName\": \"item\",\n \"joinOnField\": \"holdingsRecordId\",\n \"jsonFieldAlias\": \"it_jsonb\"\n }\n }\n ]\n}\n```\n\nThe **script** section allows a module to run custom SQLs before table / view creation/updates and after all tables/views have been created/updated.\n\nThe fields in the **script** section include:\n\n1. `run` - either `before` or `after` the tables / views are generated\n2. `snippet` - the SQL to run. There is no variable replacement. Use `snippetPath` if variable replacement or line breaks for readability are needed.\n3. `snippetPath` - relative path to a file with SQL script to run. If `snippetPath` is set then `snippet` field will be ignored.\n4. `fromModuleVersion` - same as `fromModuleVersion` for table. If no fromModuleVersion is provided the SQL runs on each upgrade. This is reasonable if the SQL runs fast and is idempotent, for example ```CREATE OR REPLACE FUNCTION``` and you want to avoid the error-prone fromModuleVersion value maintenance needed when changing the function. The SQL also runs on the initial install of a tenant.\n\nA `snippetPath` SQL file (but not a `snippet` SQL statement) can make use of\n[FreeMarker template engine](https://freemarker.apache.org) that runs and evaluates on runtime. For examples, see RMB's\n[db_scripts directory](https://github.com/folio-org/raml-module-builder/tree/master/domain-models-runtime/src/main/resources/templates/db_scripts).\nRMB provides these variables:\n\n* `${myuniversity}` tenant id, for example `diku`\n* `${mymodule}` module name, for example `mod-inventory-storage`\n* `${mode}` either `CREATE`, `UPDATE`, or `DELETE`\n* `${version}` the previous module version on update (for example `mod-inventory-storage-18.0.0` or `18.0.0`),\n or `0.0` if not available (not provided by Okapi or in `CREATE` mode).\n* `${newVersion}` the new version that currently gets installed (`CREATE` mode) or is upgraded to (`UPDATE` mode),\n for example `mod-inventory-storage-18.1.0` or `18.1.0`.\n* `${rmbVersion}` the RMB version that the module currently uses, for example `29.3.2`.\n* `${exactCount}` the `exactCount` number from schema.json or the default (1000).\n\nIn addition the `tables`, `views` and `scripts` sections of schema.json are available.\n\nMaven's pom.xml file may contain `\u003cfiltering\u003etrue\u003c/filtering\u003e` to replace variables at build time,\nit also uses the `${}` syntax. Disable maven's filtering for the SQL script files or exclude them from filtering,\notherwise it replaces for example `${version}` by a wrong value (the current module version from pom.xml).\n\nThe **exactCount** section is optional and the value of the property is\na simple integer with a default value of 1000 for the\n[estimated totalRecords](#estimated-totalrecords) hit count calculation.\n\nThe tables / views will be generated in the schema named tenantid_modulename\n\nThe x-okapi-tenant header passed in to the API call will be used to get the tenant id.\nThe value used for the module name is the artifactId found in the pom.xml (the parent artifactId is used if one is found).\n\n#### Removing an index\n\nWhen upgrading a module via the Tenant API, an index is deleted if either `\"tOps\": \"DELETE\"` is set or the complete index entry is removed. Note that indexes are the only elements where removing the entry in `schema.json` removes them from the database.\n\n\"tOps\" example:\n\n```json\n\"index\": [\n {\n \"fieldName\": \"title\",\n \"tOps\": \"DELETE\",\n \"caseSensitive\": false,\n \"removeAccents\": true\n }\n]\n```\n\n#### Removing a table\n\nRMB removes a table and all related SQL functions on the next module upgrade if the table entry contains `\"mode\": \"DELETE\"`:\n\n```json\n {\n \"tableName\": \"holdings_record\",\n \"mode\": \"DELETE\"\n }\n```\n\n#### Posting information\n\nPosting a new tenant must include a body. The body should contain a JSON\nconforming to the\n[tenantAttributes](https://github.com/folio-org/raml/blob/master/schemas/tenantAttributes.schema)\nschema.\n\nTo enable a new module indicate the version module in `module_to` and omit `module_from`.\nTo upgrade a module indicate the existing version module in `module_from` and the new version module in `module_to`.\nTo disable a module indicate the existing version module in `module_from` and omit `module_to`.\n\nThe body may also hold a `parameters` property to specify per-tenant\nactions/info to be done during tenant creation/update.\n\n#### Encrypting Tenant passwords\n\nAs of now (this may change in the future), securing a tenant's connection to the database via an encrypted password can be accomplished in the following way:\n\n - Set the secret key (as described in the Securing DB Configuration file section)\n\n The PASSWORD will be replaced with the following:\n encrypt(tenant id with secret key) = **new tenant's password**\n The **new tenant's password** will replace the default PASSWORD value (which is the tenantid_modulename)\n The RMB Postgres client will use the secret key and the passed in tenant id to calculate the tenant's password when DB connections are needed for that tenant. Note that if you use the tenant API and set the secret key - the decrypting of the password will be done by the Postgres Client for each tenant connection.\n\n\nThe RMB comes with a TenantClient to facilitate calling the API via URL.\nTo post a tenant via the client:\n\n```java\nTenantClient tClient = null;\ntClient = new TenantClient(\"http://localhost:\" + port, \"mytenantid\", \"sometoken\");\ntClient.post( response -\u003e {\n response.bodyHandler( body -\u003e {\n System.out.println(body.toString());\n async.complete();\n });\n});\n```\n\n#### The Delete Tenant API\n\nWhen this API is called RMB will basically drop the schema for the tenant (CASCADE) as well as drop the user\n\n\n**Some Postgres Client examples**\n\n\nExamples:\n\nSaving a POJO within a transaction:\n\n```java\nPoLine poline = new PoLine();\n...\npostgresClient.withTrans(conn -\u003e {\n return conn.getByIdForUpdate(SOME_OTHER_TABLE, id)\n .compose(x -\u003e ...)\n .compose(x -\u003e conn.save(TABLE_NAME_POLINE, poline)\n .compose(x -\u003e ...)\n}).compose(res -\u003e ...\n```\n\nQuerying for similar POJOs in the DB (with or without additional criteria):\n\n```java\nCriterion c = new Criterion(new Criteria().addField(\"id\").setJSONB(false).setOperation(\"=\").setVal(entryId));\n\npostgresClient.get(TABLE_NAME_POLINE, PoLine.class, c)\n.compose(x -\u003e ...\n```\n\n## RAMLs API\n\nThe RAMLs API is a multiple interface which affords RMB modules to expose their RAML files in a machine readable way. To enable the interface the module must add the following to the provides array of its module descriptor:\n\n```JSON\n{\n \"id\": \"_ramls\",\n \"version\": \"1.0\",\n \"interfaceType\" : \"multiple\",\n \"handlers\" : [\n {\n \"methods\" : [ \"GET\" ],\n \"pathPattern\" : \"/_/ramls\",\n \"permissionsRequired\" : [ ]\n }\n ]\n}\n```\n\nThe interface has a single GET endpoint with an optional query parameter path. Without the path query parameter the response will be an application/json array of the available RAMLs. This will be the immediate RAMLs the module provides. If the query parameter path is provided it will return the RAML at the path if exists. The RAML will have HTTP resolvable references. These references are either to JSON Schemas or RAMLs the module provides or shared JSON Schemas and RAMLs. The shared JSON Schemas and RAMLs are included in each module via a git submodule under the path `raml_util`. These paths are resolvable using the path query parameter.\n\nThe RAML defining the API:\n\nhttps://github.com/folio-org/raml/blob/eda76de6db681076212e20c7f988c3913764b9b0/ramls/ramls.raml\n\n## JSON Schemas API\n\nThe JSON Schemas API is a multiple interface which affords RMB modules to expose their JSON Schema files in a machine readable way. To enable the interface the module must add the following to the provides array of its module descriptor:\n\n```JSON\n{\n \"id\": \"_jsonSchemas\",\n \"version\": \"1.0\",\n \"interfaceType\" : \"multiple\",\n \"handlers\" : [\n {\n \"methods\" : [ \"GET\" ],\n \"pathPattern\" : \"/_/jsonSchemas\",\n \"permissionsRequired\" : [ ]\n }\n ]\n}\n```\n\nThe interface has a single GET endpoint with an optional query parameter path.\nWithout the path query parameter the response will be an \"application/json\" array of the available JSON Schemas. By default this will be JSON Schemas that are stored in the root of ramls directory of the module. Returned list of schemas can be customized in modules pom.xml file.\nThe `ramls` subdirectory is the default to search for schema files. Add `\u003cramlDirs\u003e`\nto the domain-models-maven-plugin configuration in pom.xml to change it, for example:\n\n```xml\n\u003cconfiguration\u003e\n \u003cramlDirs\u003e\n \u003cramlDir\u003eramls\u003c/ramlDir\u003e\n \u003cramlDir\u003eramls/raml-util/ramls\u003c/ramlDir\u003e\n \u003c/ramlDirs\u003e\n\u003c/configuration\u003e\n```\n\nIf the query parameter path is provided it will return the JSON Schema at the path if exists. The JSON Schema will have HTTP resolvable references. These references are either to JSON Schemas or RAMLs the module provides or shared JSON Schemas and RAMLs. The shared JSON Schemas and RAMLs are included in each module via a git submodule under the path `raml_util`. These paths are resolvable using the path query parameter.\n\nThe RAML defining the API:\n\nhttps://github.com/folio-org/raml/blob/eda76de6db681076212e20c7f988c3913764b9b0/ramls/jsonSchemas.raml\n\n## Query Syntax\n\nThe RMB can receive parameters of different types. Modules can declare a query parameter and receive it as a string parameter in the generated API functions.\n\nThe RMB exposes an easy way to query, using [CQL (Contextual Query Language)](#cql-contextual-query-language).\nThis enables a seamless integration from the query parameters to a prepared \"where\" clause to query with.\n\n```java\n//create object on table.field\nCQL2PgJSON cql2pgJson = new CQL2PgJSON(\"tablename.jsonb\");\n//cql wrapper based on table.field and the cql query\nCQLWrapper cql = new CQLWrapper(cql2pgJson, query);\n//query the db with the cql wrapper object\nPostgresClient.getInstance(context.owner(), tenantId).get(CONFIG_COLLECTION, Config.class,\n cql, true,\n```\n\nThe CQLWrapper can also get an offset and limit:\n\n```java\nnew CQLWrapper(cql2pgJson, query).setLimit(new Limit(limit)).setOffset(new Offset(offset));\n```\n\nA CQL querying example:\n\n```sh\nhttp://localhost:\u003cport\u003e/configurations/entries?query=scope.institution_id=aaa%20sortBy%20enabled\n```\n\n## Estimated totalRecords\n\nRMB adds a `totalRecords` field to result sets. For `limit` = 0 it contains the exact value, otherwise an estimation how many records were matching if paging parameters were set to `offset` = 0 and `limit` = unlimited. For `totalRecords` = none the calculation and the field are suppressed.\n\nExamples (before percent encoding):\n\n/instance-storage/instances?query=title=garden\u0026limit=0\n/users?query=barcode==\"1234\"\u0026totalRecords=none\n\nFor estimation it uses this algorithm:\n\n1. Run \"EXPLAIN SELECT\" to get an estimation from PostgreSQL.\n2. If this is greater than 4\\*1000 return it and stop.\n3. Run \"SELECT COUNT(\\*) FROM query LIMIT 1000\".\n4. If this is less than 1000 return it (this is the exact number) and stop.\n5. If the result from 1. is less than 1000 return 1000 and stop.\n6. Return result from 1. (this is a value between 1000 and 4\\*1000).\n\nStep 2. contains the factor 4 that should be adjusted when there is empirical data for a better number.\nStep 3. may take long for full text queries with many hits, therefore we need steps 1.-2.\n\n1000 is configurable, see the [Post Tenant API](#the-post-tenant-api) how to set `exactCount` in schema.json.\n\nRMB adjusts `totalRecords` when the number of returned records in the current result set and the paging parameters `offset` and `limit` proves it wrong:\n\n* If no record is returned and `totalRecords \u003e offset` then adjust `totalRecords = offset`.\n* If the number of records returned equals `limit` and `totalRecords \u003c offset + limit` then adjust `totalRecords = offset + limit`.\n* If the number of records is at least one but less than `limit` then adjust `totalRecords = offset +` number of records returned (this is the exact number).\n\nNote that clients should **continue on the next page when `totalRecords = offset + limit`** because there may be more records.\n\nThis is the exact count guarantee:\n\nFor `limit` = 0 an exact count is returned without any records. Otherwise:\n\nIf a result set has a `totalRecords` value that is less than 1000 then it is the exact count; if it is 1000 or more it may be an estimate.\n\nIf the exact count is less than 1000 then `totalRecords` almost always contains the exact count; only when PostgreSQL estimates it to be more than 4\\*1000 then it contains that overestimation.\n\nReplace 1000 by `exactCount` if configured differently.\n\nPerformance warning: Depending on query and database indexes using `limit` = 0 may take long, may put load on the database and should run asynchronously in the front-end. If there isn't a supporting database index it requires a full table scan. In all cases PostgreSQL needs to check the visibility map, see [Slow counting](https://wiki.postgresql.org/wiki/Slow_Counting) and [Index-only_scans](https://wiki.postgresql.org/wiki/Index-only_scans).\n\n## Metadata\n\nRMB is aware of the [metadata.schema](https://github.com/folio-org/raml/blob/raml1.0/schemas/metadata.schema). When a request (POST / PUT / PATCH) comes into an RMB module, RMB will check if the passed-in JSON's schema declares a reference to the metadata schema. If so, RMB will populate the JSON with a metadata section with the current user and the current time. RMB will set both update and create values to the same date/time and to the same user, as accepting this information from the request may be unreliable. The module should persist the creation date and the created by values after the initial POST. For an example of this using SQL triggers see [metadata.ftl](https://github.com/folio-org/raml-module-builder/blob/master/domain-models-runtime/src/main/resources/templates/db_scripts/metadata.ftl). Add [withMetadata to the schema.json](https://github.com/folio-org/raml-module-builder#the-post-tenant-api) to create that trigger.\n\n## Optimistic Locking\n\nRMB supports optimistic locking. By default it is disabled. Module developers can enable it by adding attribute `withOptimisticLocking` to the table definition in schema.json. The available options are listed below. Unless `off` is specified, a database trigger will be created to auto populate the `_version` field in json on insert. On jsonb column update a database trigger will check that the `_version` field is the same in the incoming record and in the database, and will then increment the `_version` field before updating the database record. Note, `_version` field has to be defined in json to make this work. PgUtil reports a version conflict error as 409 HTTP response code if 409 is defined in RAML, otherwise, it will fall back to use 400 or 500 HTTP response code. The client should never increment the `_version` property; only the trigger should increment it to ensure that the check and the increment are in the same transaction to avoid any race condition.\n\n* `off` Optimistic Locking is disabled. The trigger is removed if it existed before.\n* `failOnConflict` Optimistic Locking is enabled. Version conflict will fail the transaction with a customized SQL error code 23F09.\n* `failOnConflictUnlessSuppressed` The same as `failOnConflict` but setting `_version` to `-1` is handled as if the current `_version` were provided. This must be enabled with the `DB_ALLOW_SUPPRESS_OPTIMISTIC_LOCKING` enviroment variable, see above. To have one API with this suppression and one without for the same database table add code to the latter that removes the `_version` property if it is `-1`. Suppressing optimistic locking is known to lead to data loss in some cases, don't use in production, you have been warned!\n* `logOnConflict` Optimistic Locking is enabled but the conflicting update succeeds, the version conflict info is logged with a customized SQL error code 23F09. Consult [Vertx logging](https://vertx.io/docs/vertx-core/java/#_logging) and [PostgreSQL Errors and Messages](https://www.postgresql.org/docs/current/plpgsql-errors-and-messages.html) if `logOnConflict` doesn't log.\n\nUpsert fails when using `INSERT ... ON CONFLICT (id) DO UPDATE` because the `INSERT` trigger overwrites the `_version` property that the `UPDATE` trigger uses to detect optimistic locking conflicts. Instead use [RMB's `upsert` plpgsql function](domain-models-runtime/src/main/resources/templates/db_scripts/general_functions.ftl) or implement upsert using a transaction and `UPDATE ...; IF NOT FOUND THEN INSERT ...`\n\n_Sample log entries:_\n\n```jsonp\n01:14:07 [] [] [] [] WARN ? Backend notice: severity='NOTICE', code='23F09', message='Ignoring optimistic locking conflict while overwriting changed record 57db089f-18e4-7815-55d5-4cc6607e9059: Stored _version is 2, _version of request is \"1\"' ...\n```\n\n```jsonp\n01:15:20 [] [] [] [] ERROR PostgresClient saveBatch size=2 { \"message\": \"version conflict\", \"severity\": \"ERROR\", \"code\": \"23F09\", \"where\": \"PL/pgSQL function raise_409() line 1 at RAISE\", \"file\": \"pl_exec.c\", \"line\": \"3337\", \"routine\": \"exec_stmt_raise\" }\n```\nUse mod-inventory-storage `instance` table as an example, do following to enable optimistic locking\n\n* in db `schema.json`, add `withOptimisticLocking` attribute for `instance` table definition\n\n```json5\n{\n \"tableName\": \"instance\",\n \"withOptimisticLocking\": \"logOnConflict\",\n ...\n}\n```\n\n* in `instance.json`, add `_version` field\n\n```json5\n{\n \"$schema\": \"http://json-schema.org/draft-04/schema#\",\n \"description\": \"An instance record\",\n \"type\": \"object\",\n \"properties\": {\n \"_version\": {\n \"type\": \"integer\",\n \"description\": \"Record version for optimistic locking\"\n }\n ...\n}\n```\n\n* update raml file to define 409 response code. For example in `instance-storage.raml` add below for `/{instanceId}` API\n\n```raml\nput:\n responses:\n 409:\n description: \"Conflict\"\n body:\n text/plain:\n example: \"Optimistic locking version has changed\"\n```\n\n#### API and Schema versioning for OptimisticLocking\n\nWhen enabling the `withOptimisticLocking` property on a table definition in `schema.json` an optional `_version` property MUST be added to the related entity definition in JSON Schema.\n\nWhen changing the value of the `withOptimisticLocking` property, there MUST be a corresponding change to the API version:\n\n* when changing from `off` to `logOnConflict` – the API version change should be minor (e.g 1.1 to 1.2)\n* when changing from `off` or `logOnConflict` to `failOnConflict` – the API version change should be major (e.g 1.1 to 2.0)\n* when changing from `failOnConflict` to `off` or `logConflict` – the API version change should be major (e.g 2.0 to 3.0)\n\n## Facet Support\n\nRMB also allows easy faceting of result sets. The grouping / faceting is done in the database.\nTo add faceting to your API.\n1. Add the [faceting RAML trait](https://github.com/folio-org/raml/blob/master/traits/facets.raml) to your RAML and reference it from the endpoint (using the is:[])\n - facet query parameter format: `facets=a.b.c` or `facets=a.b.c:10` (they are repeating). For example `?facets=active\u0026facets=personal.lastName`\n2. Add the [resultInfo.schema](https://github.com/folio-org/raml/blob/master/schemas/resultInfo.schema) to your RAML and reference it within your collection schemas.\nFor example:\n```json5\n\"type\": \"object\",\n\"properties\": {\n \"items\": {\n \"id\": \"items\",\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"$ref\" : \"item.json\"\n }\n },\n \"resultInfo\": {\n \"type\": \"object\",\n \"$ref\": \"raml-util/schemas/resultInfo.schema\"\n }\n}\n```\n3. When building your module, an additional parameter will be added to the generated interfaces of the faceted endpoints. `List\u003cString\u003e facets`. You can simply convert this list into a List of Facet objects using the RMB tool as follows: `List\u003cFacetField\u003e facetList = FacetManager.convertFacetStrings2FacetFields(facets, \"jsonb\");` and pass the `facetList` returned to the `postgresClient`'s `get()` methods.\n\nYou can set the amount of results to facet on by calling (defaults to 10,000) `FacetManager.setCalculateOnFirst(20000);`\nNote that higher numbers will potentially affect performance.\n\n4. Faceting on array fields can be done in the following manner:\n`personal.secondaryAddress[].postalCode`\n`personal.secondaryAddress[].address[].postalCode`\n\nNOTE: Creating an index on potential facet fields may be required so that performance is not greatly hindered\n\n## JSON Schema fields\n\nIt is possible to indicate that a field in the JSON is a readonly field when declaring th","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffolio-org%2Framl-module-builder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffolio-org%2Framl-module-builder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffolio-org%2Framl-module-builder/lists"}