{"id":23122769,"url":"https://github.com/folio-org/mod-inventory-update","last_synced_at":"2026-01-23T15:33:31.529Z","repository":{"id":37234776,"uuid":"273301068","full_name":"folio-org/mod-inventory-update","owner":"folio-org","description":"Provides different update schemes for populating a FOLIO Inventory Storage","archived":false,"fork":false,"pushed_at":"2026-01-16T09:20:40.000Z","size":789,"stargazers_count":5,"open_issues_count":3,"forks_count":4,"subscribers_count":13,"default_branch":"master","last_synced_at":"2026-01-16T23:45:09.092Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/folio-org.png","metadata":{"files":{"readme":"README.md","changelog":"NEWS.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-06-18T17:33:21.000Z","updated_at":"2026-01-07T11:04:47.000Z","dependencies_parsed_at":"2025-08-11T11:25:22.593Z","dependency_job_id":"14e14c36-d8a5-4fae-81a2-6b09af856e36","html_url":"https://github.com/folio-org/mod-inventory-update","commit_stats":null,"previous_names":[],"tags_count":32,"template":false,"template_full_name":null,"purl":"pkg:github/folio-org/mod-inventory-update","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Fmod-inventory-update","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Fmod-inventory-update/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Fmod-inventory-update/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Fmod-inventory-update/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/folio-org","download_url":"https://codeload.github.com/folio-org/mod-inventory-update/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Fmod-inventory-update/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28694660,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-23T14:15:13.573Z","status":"ssl_error","status_checked_at":"2026-01-23T14:09:05.534Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-17T07:30:19.446Z","updated_at":"2026-01-23T15:33:31.514Z","avatar_url":"https://github.com/folio-org.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# (under construction) mod-inventory-update (MIU)\n\nCopyright (C) 2019-2025 The Open Library Foundation\n\nThis software is distributed under the terms of the Apache License, Version 2.0. See the file \"[LICENSE](LICENSE)\" for\nmore information.\n\u003c!-- TOC --\u003e\n* [(under construction) mod-inventory-update (MIU)](#under-construction-mod-inventory-update-miu)\n  * [Purpose](#purpose)\n  * [How to use MIU](#how-to-use-miu)\n    * [The relevant \"upsert\" APIs](#the-relevant-upsert-apis)\n    * [What the APIs do with the inventory record set](#what-the-apis-do-with-the-inventory-record-set)\n      * [Detect if holdings records or items should be deleted](#detect-if-holdings-records-or-items-should-be-deleted)\n      * [Control record overlay on updates.](#control-record-overlay-on-updates)\n        * [Prevent MIU from override existing values](#prevent-miu-from-override-existing-values)\n        * [Instruct MIU to leave the item status unmodified under certain circumstance.](#instruct-miu-to-leave-the-item-status-unmodified-under-certain-circumstance)\n        * [Instruct MIU to avoid deleting items even though they are missing from the input](#instruct-miu-to-avoid-deleting-items-even-though-they-are-missing-from-the-input)\n        * [Retain omitted items if the status indicates that they are still circulating](#retain-omitted-items-if-the-status-indicates-that-they-are-still-circulating)\n      * [Provisional Instance created when related Instance doesn't exist yet](#provisional-instance-created-when-related-instance-doesnt-exist-yet)\n      * [Deletion of Instance-to-Instance relations](#deletion-of-instance-to-instance-relations)\n      * [Instance DELETE requests](#instance-delete-requests)\n        * [Protecting certain items or holdings records from deletion in DELETE requests](#protecting-certain-items-or-holdings-records-from-deletion-in-delete-requests)\n      * [Statistical coding of delete protection events](#statistical-coding-of-delete-protection-events)\n    * [Additional info about the batch version of upserts `/inventory-batch-upsert-hrid`](#additional-info-about-the-batch-version-of-upserts-inventory-batch-upsert-hrid)\n  * [Importing XML source files to Inventory Storage](#importing-xml-source-files-to-inventory-storage)\n  * [The importing component](#the-importing-component)\n    * [Channels](#channels)\n      * [Requests operating on a given channel](#requests-operating-on-a-given-channel)\n      * [Importing](#importing)\n      * [Request operating on multiple channels](#request-operating-on-multiple-channels)\n      * [Using `tag` for channel ID](#using-tag-for-channel-id)\n    * [The \"import job\"](#the-import-job)\n      * [Requests operating on jobs:](#requests-operating-on-jobs)\n  * [Miscellaneous](#miscellaneous)\n    * [`/shared-inventory-upsert-matchkey`](#shared-inventory-upsert-matchkey)\n    * [APIs for fetching an Inventory record set](#apis-for-fetching-an-inventory-record-set)\n      * [Fetching an Inventory record set from `inventory-upsert-hrid/fetch`](#fetching-an-inventory-record-set-from-inventory-upsert-hridfetch)\n      * [The _version fields and optimistic locking](#the-_version-fields-and-optimistic-locking)\n  * [Prerequisites](#prerequisites)\n  * [Git Submodules](#git-submodules)\n  * [Building](#building)\n  * [Additional information](#additional-information)\n    * [Other documentation](#other-documentation)\n    * [Code of Conduct](#code-of-conduct)\n    * [Issue tracker](#issue-tracker)\n    * [ModuleDescriptor](#moduledescriptor)\n    * [API documentation](#api-documentation)\n    * [Code analysis](#code-analysis)\n    * [Download and configuration](#download-and-configuration)\n\u003c!-- TOC --\u003e\n\n## Purpose\n\nMod-inventory-update (MIU) is module for importing XML and JSON sources files into Inventory\nStorage; creating, updating or deleting instances, holdings records, and items in the storage.\n\n## How to use MIU\n\nMod-inventory-update operates with a dedicated, module specific JSON object containing Inventory records like instances,\nholdings and items, called an \"inventory record set\" (IRS) .\n\nOnly the JSON based APIs take this format as input but even if using the module's XML import APIs, it is\nimportant to know the format. That is because the XML importing works by transforming the incoming XML of\nan (almost) arbitrary format into XML versions of that same structure and then converting the XML to JSON for further processing by the upsert APIs.\nSince the stylesheets implementing the XML transformations are custom provided, the XSLT author must know what\nXML structure, and ultimately JSON structure, the transformations are aiming for.\n\nThe JSON based upsert APIs are synchronous, processing the input and returning a response right away, once done.\nThe XML based import APIs on the other hand work asynchronously, and will put the file in queue and return a response that\nthis has been done. But feedback regarding the processing and its outcomes is logged in the module, to be retrieved later\nthrough the APIs.\n\n\u003cimg alt=\"Diagram of MIU APIs\" src=\"doc/diagram-of-miu-import.jpg\" width=\"1123\" title=\"Upsert and import APIs\"/\u003e\n\nThe high level structure of an inventory record set is\n```\n - Inventory instance {hrid and other properties}\n - holdings records (optional)\n   - Inventory holdings record {hrid and other properties}\n     - items (optional)\n       - Inventory item {hrid and other properties}\n       - item\n       - ...\n   - Holdings record\n     - items\n       - item\n       - item\n       - ...\n   - ...\n - instance relations (optional)\n - processing (optional)\n```\n\nThe details of this structure are described below. See also the OpenAPI spec for more details:\n[Upsert APIs](src/main/resources/openapi/inventory-update-5.0.yaml).\n\n### The relevant \"upsert\" APIs\n\nThey main JSON based APIs of interest are `inventory-upsert-hrid` and `inventory-batch-upsert-hrid`. They consume JSON files\ncontaining one or more inventory record sets.\n\nOnly the instance object and the hrid properties are required by the upsert API itself, but in order for the\nrecords to be successfully created or updated in storage, they must comply with the schema for Inventory Storage.\n\nUpserts can be done in batches, but deletion of instances is done one instance at a time through the API\nDELETE /inventory-upsert-hrid with a JSON payload of  { \"hrid\": \"the id\" }\n\nThere are a few more APIs listed in the spec. The ones named something with `matchkey` are supported but no longer in use.\n\n### What the APIs do with the inventory record set\n\nThe following is a walk-through of what happens to the records in Inventory Storage when an inventory record set is\npushed to mod-inventory-update. This applies directly to the upsert APIs, but also indirectly to the XML importing\nAPIs since they will utilize the exact same mechanics after the incoming XML has been transformed to XML IRS and\nconverted to JSON IRS.\n\nThe upsert APIs will update an Instance as well as its associated holdings and items based on incoming HRIDs on all three record types. If\nan instance with the incoming HRID does not exist in storage already, the new Instance is inserted, otherwise the\nexisting instance is updated - thus the term 'upsert'.\n\nThis means that HRIDs are required to be present from the client side in all three record types.\n\n#### Detect if holdings records or items should be deleted\n\nThe API will detect if holdings and/or items have disappeared from the Instance since last update and in that case\nremove them from storage. Note, however, that there is a distinction between a request with no `holdingsRecords`\nproperty and a request with an empty `holdingsRecords` property. If existing holdings and items should not be touched,\nfor example if holdings and items are maintained manually in Inventory, then no `holdingsRecords` property should appear\nin the request JSON and existing records would be ignored. Providing an empty `holdingsRecords` property, on the other\nhand, would cause all existing holdings and items to be deleted. The API will also detect if new holdings or items on\nthe Instance existed already on a different Instance in storage and then move them over to the incoming Instance. The\nIDs (UUIDs) on any pre-existing Instances, holdings records and items will be preserved in this process, thus avoiding\nbreaking any external UUID based references to these records.\n\nThe Inventory Record Set, that is PUT to the end point, may contain relations to other Instances, for example the kind\nof relationships that tie multipart monographs together or relations pointing to preceding or succeeding titles. Based\non a comparison with the set of relationships that may be registered for the Instance in storage already, relationships\nwill be created and/or deleted (updating relationships is obsolete).\n\n#### Control record overlay on updates.\n\nThe default behavior of MIU is to simply replace the entire record on updates, for example override the entire\nholdingsRecord, with the input JSON it receives from the client, except for the ID (UUID) and version.\n\nThe default behavior can be changed per request using structures\nin the processing element.\n\n##### Prevent MIU from override existing values\n\nMIU can be instructed to leave certain properties in place when updating Instances, holdings records, and Items.\n\nFor example, to retain all existing Item properties that are not included in the request body to MIU, use\nthe `retainExistingValues`.\n\n```\n\"processing\": {\n   \"item\": {\n     \"retainExistingValues\": {\n       \"forOmittedProperties\": true\n     }\n   }\n}\n```\n\nThis is a way to have MIU only update the set of properties it should be concerned with and let other properties be the\nresponsibility of other processes if required.\n\nIf MIU sets certain properties on insert but should not touch them in subsequent updates -- even though they are\nprovided in the request body to MIU -- those properties can be explicitly turned off in updates:\n\n```\n\"processing\": {\n   \"item\": {\n     \"retainExistingValues\": {\n       \"forTheseProperties\": [ \"a\", \"b\", \"c\" ]\n     }\n   }\n}\n```\n\nThe two settings can be combined to not touch neither omitted properties nor the explicitly listed properties.\n\nIf `forOmittedProperties` is used it requires the client to distinguish between sending an empty property vs not sending\nthe property at all. Say an Instance had `contributors` before, but now they were removed in the source catalog. If this\nis communicated to MIU by an empty `contributors` property, then it's fine, it will become empty in Inventory Storage\ntoo, but if the property is simply removed from the request body altogether, then the existing value of `contributors`\nwill be retained in storage if `forOmittedProperties` is set to true.\n\n##### Instruct MIU to leave the item status unmodified under certain circumstance.\n\nIn case the Item status is being managed outside the MIU update process, likely by FOLIO Circulation, MIU can be\ninstructed to only touch the status under certain circumstances.\n\nFor example, to only overwrite a status of \"On order\" and retain any other statuses, do:\n\n```\nprocessing\": {\n   \"item\": {\n     \"status\": {\n       \"policy\": \"overwrite\",\n       \"ifStatusWas\": [\n         {\"name\": \"On order\"}\n       ]\n     }\n   }\n}\n```\n\nThe default behavior is to overwrite all statuses.\n\n##### Instruct MIU to avoid deleting items even though they are missing from the input\n\nWhen MIU receives an Instance update, it will look for existing items on the holdings record that are not present in the\nupdate and then delete them.\n\nWith this instruction, deletion can be prevented based on a regular expression matched against a specified property.\nThis could be used to preserve items created from other sources, provided that a regular expression can be written that\nwill identify such items\nwithout at the same time matching items of the current data feed.\n\nFor example, to retain items that have HRIDs starting with non-digit characters:\n\n```\nprocessing\": {\n   \"item\": {\n     \"retainOmittedRecord\": {\n       \"ifField\": \"hrid\",\n       \"matchesPattern\": \"\\\\D+.*\"\n     }\n   }\n}\n```\n\n##### Retain omitted items if the status indicates that they are still circulating\n\nUsually items that are omitted from the holdings record in the upsert will be removed from storage by MIU. The\nexceptions are if they are delete protected as described in the previous section, or if they have a status that\nindicates they might still be circulating.\n\nMIU will avoid deleting items with following item statuses\n\n* Awaiting delivery\n* Awaiting pickup\n* Checked out\n* Aged to lost\n* Claimed returned\n* Declared lost\n* Paged\n* In transit\n\n#### Provisional Instance created when related Instance doesn't exist yet\n\nIf an upsert request comes in with a relation to an Instance that doesn't already exist in storage, a provisional\nInstance will be created provided that the request contains sufficient data as required for creating the provisional\nInstance - like any mandatory Instance properties.\n\n#### Deletion of Instance-to-Instance relations\n\nOnly existing relationships that are explicitly omitted in the request will be deleted. In FOLIO Inventory, a relation\nwill appear on both Instances of the relation, say, one Instance will have a parent relation, and the other will have a\nchild relation.\n\nThis may not be the case in the source system where, perhaps, the child record may declare its parent, but the parent\nwill not mention its child records.\n\nTo support deletion of relations for these scenarios, and not implicitly but unintentionally delete too many, following\nrules apply:\n\nIncluding an empty array of child instances will tell the API that if the Instance has any existing child relations,\nthey should be deleted.\n\n```\n\"instanceRelations\": {\n  \"childInstances\": []\n}\n```\n\nLeaving out any reference to child instances -- or as in this sample, any references to any related Instances at all --\nmeans that any existing relationships will be left untouched by the update request.\n\n```\n\"instanceRelations\": {\n}\n```\n\n#### Instance DELETE requests\n\nThe API supports DELETE requests, which would delete the Instance with all of its associated holdings records and items\nand any relations it might have to other Instances.\n\nTo delete an instance record with all its holdings and items, send a DELETE request to `/inventory-upsert-hrid` with a payload like this:\n\n```\n{\n  \"hrid\": \"001\"\n}\n```\n\nNote that deleting any relations that the Instance had to other instances only cuts those links between them but does not otherwise affect those other instances.\n\n##### Protecting certain items or holdings records from deletion in DELETE requests\n\nDelete requests can be extended with a processing instruction that blocks deletion of holdings and/or items based on\npattern matching in select properties.\n\nThis can be used to avoid deletion of items that are created outside the MIU pipeline (for example through the\nInventory UI) provided that there is a pattern that can be applied to a property value of those records to identify\nwhich items to protect.\n\nFor example, to protect items that have HRIDs starting with non-digit characters, following delete body for deletion of\nthe Instance with HRID \"123456\" could be used:\n\n```\n{\n    \"hrid\":  \"1234567\",\n    \"processing\": {\n       \"item\": {\n         \"blockDeletion\": {\n           \"ifField\": \"hrid\",\n           \"matchesPattern\": \"\\\\D+.*\"\n         }\n       }\n    }\n}\n```\n\nWhen a delete request is sent for an Instance that has protected Items, the deletion of the Instance, as well as the\nholdings record for the Item, will be blocked as well. Other holdings records or Items that do not match the block\ncriteria will be deleted.\n\nDeletion will likewise be blocked when one or more items under the instance have one of the statuses also listed in\n\"Retain omitted items if the status indicates that they are still circulating\":\n\n* Awaiting delivery\n* Awaiting pickup\n* Checked out\n* Aged to lost\n* Claimed returned\n* Declared lost\n* Paged\n* In transit\n\n#### Statistical coding of delete protection events\n\nWhen records are prevented from being deleted, the delete or the upsert request can be configured to set specified\nstatistical codes on the record, which was up for deletion, and then update the existing record with those. This will not\ncount as a record update, and if the update fails -- for example due to use of invalid UUIDs -- it will write an error\nto the module log but will not fail the overall request. If non-existing statistical codes are specified the storage\nmodule will silently avoid setting them.\n\nHere are some examples of statistical coding of skipped deletes, first in a delete requests\n\nSet all available codes on all record types. This means not just setting the code on the item that was not delete but also marking it on the instance that consequently could also not be deleted:\n```\n{\n  \"hrid\": \"in001\",\n  \"processing\": {\n    \"item\": {\n      \"blockDeletion\": {\n        \"ifField\": \"hrid\",\n        \"matchesPattern\": \"it.*\"\n      },\n      \"statisticalCoding\": [\n        {\"if\": \"deleteSkipped\", \"becauseOf\":  \"ITEM_STATUS\", \"setCode\": \"1ce0a775-286f-45ae-8446-e26ba0687b61\"},\n        {\"if\": \"deleteSkipped\", \"becauseOf\":  \"ITEM_PATTERN_MATCH\", \"setCode\": \"42b735fa-eb6f-4c53-b5d5-2d98500868c5\"}\n      ]\n    },\n    \"holdingsRecord\": {\n      \"blockDeletion\": {\n        \"ifField\": \"hrid\",\n        \"matchesPattern\": \"ho.*\"\n      },\n      \"statisticalCoding\": [\n        {\"if\": \"deleteSkipped\", \"becauseOf\":  \"HOLDINGS_RECORD_PATTERN_MATCH\", \"setCode\": \"d11fd9d8-b234-4159-b7b1-61b531bb1405\"},\n        {\"if\": \"deleteSkipped\", \"becauseOf\":  \"ITEM_STATUS\", \"setCode\": \"1ce0a775-286f-45ae-8446-e26ba0687b61\"},\n        {\"if\": \"deleteSkipped\", \"becauseOf\":  \"ITEM_PATTERN_MATCH\", \"setCode\": \"42b735fa-eb6f-4c53-b5d5-2d98500868c5\"}\n      ]\n    },\n    \"instance\": {\n      \"statisticalCoding\": [\n        {\"if\": \"deleteSkipped\", \"becauseOf\":  \"PO_LINE_REFERENCE\", \"setCode\": \"98993c92-c5c9-414b-b12c-a82836b0dbf6\"},\n        {\"if\": \"deleteSkipped\", \"becauseOf\":  \"HOLDINGS_RECORD_PATTERN_MATCH\", \"setCode\": \"d11fd9d8-b234-4159-b7b1-61b531bb1405\"},\n        {\"if\": \"deleteSkipped\", \"becauseOf\":  \"ITEM_STATUS\", \"setCode\": \"1ce0a775-286f-45ae-8446-e26ba0687b61\"},\n        {\"if\": \"deleteSkipped\", \"becauseOf\":  \"ITEM_PATTERN_MATCH\", \"setCode\": \"42b735fa-eb6f-4c53-b5d5-2d98500868c5\"}\n      ]\n    }\n  }\n}\n```\n\nLift all codes up on the instance level, even if the primarily protected record was a holdings record or an item:\n\n```\n{\n  \"hrid\": \"in001\",\n  \"processing\": {\n    \"item\": {\n      \"blockDeletion\": {\n        \"ifField\": \"hrid\",\n        \"matchesPattern\": \"it.*\"\n      }\n    },\n    \"holdingsRecord\": {\n      \"blockDeletion\": {\n        \"ifField\": \"hrid\",\n        \"matchesPattern\": \"ho.*\"\n      }\n    },\n    \"instance\": {\n      \"statisticalCoding\": [\n        {\"if\":  \"deleteSkipped\", \"becauseOf\":  \"PO_LINE_REFERENCE\", \"setCode\": \"98993c92-c5c9-414b-b12c-a82836b0dbf6\"},\n        {\"if\":  \"deleteSkipped\", \"becauseOf\":  \"HOLDINGS_RECORD_PATTERN_MATCH\", \"setCode\": \"d11fd9d8-b234-4159-b7b1-61b531bb1405\"},\n        {\"if\":  \"deleteSkipped\", \"becauseOf\":  \"ITEM_STATUS\", \"setCode\": \"1ce0a775-286f-45ae-8446-e26ba0687b61\"},\n        {\"if\":  \"deleteSkipped\", \"becauseOf\":  \"ITEM_PATTERN_MATCH\", \"setCode\": \"42b735fa-eb6f-4c53-b5d5-2d98500868c5\"}\n      ]\n    }\n  }\n}\n```\n\nOnly set the code on the record that was directly protected from deletion (not on the indirectly delete protected records) :\n\n```\n{\n  \"hrid\": \"in001\",\n  \"processing\": {\n    \"item\": {\n      \"blockDeletion\": {\n        \"ifField\": \"hrid\",\n        \"matchesPattern\": \"it.*\"\n      },\n      \"statisticalCoding\": [\n        {\"if\": \"deleteSkipped\", \"becauseOf\": \"ITEM_STATUS\", \"setCode\": \"1ce0a775-286f-45ae-8446-e26ba0687b61\"},\n        {\"if\": \"deleteSkipped\", \"becauseOf\": \"ITEM_PATTERN_MATCH\", \"setCode\": \"42b735fa-eb6f-4c53-b5d5-2d98500868c5\"}\n      ]\n    },\n    \"holdingsRecord\": {\n      \"blockDeletion\": {\n        \"ifField\": \"hrid\",\n        \"matchesPattern\": \"ho.*\"\n      },\n      \"statisticalCoding\": [\n        {\"if\": \"deleteSkipped\", \"becauseOf\": \"HOLDINGS_RECORD_PATTERN_MATCH\", \"setCode\": \"d11fd9d8-b234-4159-b7b1-61b531bb1405\"}\n      ]\n    },\n    \"instance\": {\n      \"statisticalCoding\": [\n        {\"if\": \"deleteSkipped\", \"becauseOf\": \"PO_LINE_REFERENCE\", \"setCode\": \"98993c92-c5c9-414b-b12c-a82836b0dbf6\"}\n      ]\n    }\n  }\n}\n```\n\nStatistical codes can be set on holdings and items in upserts (deleting the instance is not an option in the upsert,\nand preventing delete due to PO_LINE_REFERENCE does not apply).\n\nFor example:\n```\n\"processing\": {\n  \"item\": {\n    \"retainOmittedRecord\": {\n      \"ifField\": \"hrid\",\n      \"matchesPattern\": \"it.*\"\n    },\n    \"statisticalCoding\": [\n      { \"if\": \"deleteSkipped\", \"becauseOf\": \"ITEM_STATUS\", \"setCode\": \"2b750461-5368-4a4e-9484-4e1eea2bc384\" },\n      { \"if\": \"deleteSkipped\", \"becauseOf\": \"ITEM_PATTERN_MATCH\", \"setCode\": \"6f143d4c-75fe-4987-ae3e-3d7c2a4ccca2\" },\n    ]\n  },\n  \"holdingsRecord\": {\n    \"retainOmittedRecord\": {\n      \"ifField\": \"hrid\",\n      \"matchesPattern\": \"ho.*\"\n    }\n    \"statisticalCoding\": [\n      { \"if\": \"deleteSkipped\", \"becauseOf\": \"ITEM_STATUS\", \"setCode\": \"b2452cc2-7024-41fa-a5c7-b1736280d781\" },\n      { \"if\": \"deleteSkipped\", \"becauseOf\": \"ITEM_PATTERN_MATCH\", \"setCode\": \"a8e70d5e-2861-4a89-93cc-88679c74e592\" },\n      { \"if\": \"deleteSkipped\", \"becauseOf\": \"HOLDINGS_RECORD_PATTERN_MATCH\", \"setCode\": \"e24f4a4e-8d8f-4ca2-ab05-dd497a8379e3\" }\n    ]\n  },\n  \"instance\": {\n  }\n}\n```\n\n### Additional info about the batch version of upserts `/inventory-batch-upsert-hrid`\n\nA client can send arrays of inventory record sets to the batch APIs with significant improvement to overall throughput\ncompared to the single record APIs.\n\nThese APIs utilise Inventory Storage's batch upsert APIs for Instances, holdings records and Items.\n\nIn case of data problems in either type of entity (referential constraint errors, missing mandatory properties, etc),\nthe entire batch of the given entity will fail. For example if an Item fails all Items fail. At this point all the\nInstances and holdings records are presumably persisted. The module aims to recover from such inconsistent states by\nswitching from batch processing to record-by-record updates in case of errors so that, in this example, all the good\nItems can be persisted. The response on a request with one or more errors, that didn't prevent the entire request from being processed, will be\na `207` `Multi-Status`, and the one or more error that were found will be returned with the\nresponse JSON, together with the summarized update statistics (\"metrics).\n\nIn a feed with many errors the throughput will be close to that of the single record APIs since many batches will be\nprocessed record-by-record.\n\nIf the client needs to pair up error reports in the response with any original records it holds itself, the client can\nset an identifier in the inventory record set property \"processing\". The name and content of that property is entirely\nup to the client, MIU will simply return the data it gets, so it could be a simple sequence number for the\nbatch, like\n\n```\n{\n \"inventoryRecordSets\":\n  [\n   {\n    \"instances\": ...\n    \"holdingsRecords\": ...\n    \"processing\": {\n      ...\n      \"batchIndex\": 1\n    }\n   },\n    \"instances\": ...\n    \"holdingsRecords\": ...\n    \"processing\": {\n      ...\n      \"batchIndex\": 2\n    }\n}\n```\n\nA response of a batch upsert of 100 Instances where the 50th failed, the error message is tagged with the clients batch\nindex.\n\n```\n{\n    \"metrics\": {\n        \"INSTANCE\": {\n            \"CREATE\": {\n                \"COMPLETED\": 99,\n                \"FAILED\": 1,\n                \"SKIPPED\": 0,\n                \"PENDING\": 0\n            },\n            \"UPDATE\": {\n                \"COMPLETED\": 0,\n                \"FAILED\": 0,\n                \"SKIPPED\": 0,\n                \"PENDING\": 0\n            },\n            \"DELETE\": {\n                \"COMPLETED\": 0,\n                \"FAILED\": 0,\n                \"SKIPPED\": 0,\n                \"PENDING\": 0\n            }\n        },\n        \"HOLDINGS_RECORD\": {\n            \"CREATE\": {\n                \"COMPLETED\": 0,\n                \"FAILED\": 0,\n                \"SKIPPED\": 0,\n                \"PENDING\": 0\n            },\n            \"UPDATE\": {\n                \"COMPLETED\": 0,\n                \"FAILED\": 0,\n                \"SKIPPED\": 0,\n                \"PENDING\": 0\n            },\n            \"DELETE\": {\n                \"COMPLETED\": 0,\n                \"FAILED\": 0,\n                \"SKIPPED\": 0,\n                \"PENDING\": 0\n            }\n        },\n        \"ITEM\": {\n            \"CREATE\": {\n                \"COMPLETED\": 0,\n                ...\n                ...\n                ...\n            }\n        }\n    },\n    \"errors\": [\n        {\n            \"category\": \"STORAGE\",\n            \"message\": {\n                \"message\": {\n                    \"errors\": [\n                        {\n                            \"message\": \"must not be null\",\n                            \"type\": \"1\",\n                            \"code\": \"javax.validation.constraints.NotNull.message\",\n                            \"parameters\": [\n                                {\n                                    \"key\": \"source\",\n                                    \"value\": \"null\"\n                                }\n                            ]\n                        }\n                    ]\n                }\n            },\n            \"shortMessage\": \"One or more errors occurred updating Inventory records\",\n            \"entityType\": \"INSTANCE\",\n            \"entity\": {\n                \"title\": \"New title 50\",\n                \"instanceTypeId\": \"12345\",\n                \"matchKey\": \"new_title_50__________________________________________________________0000_____________________________________________________________________________________________p\",\n                \"id\": \"782d5015-147d-433d-beaf-06a47bde6be5\"\n            },\n            \"statusCode\": 422,\n            \"requestJson\": {\n                \"instance\": {\n                    \"title\": \"New title 50\",\n                    \"instanceTypeId\": \"12345\",\n                    \"matchKey\": \"new_title_50__________________________________________________________0000_____________________________________________________________________________________________p\"\n                },\n                \"processing\": {\n                    \"batchIndex\": 50\n                }\n            },\n            \"details\": {\n\n            }\n        }\n    ]\n}\n```\n\nNote that any given batch cannot touch the same records twice, since that would require a certain order of processing,\nsomething that batching will not be able to guarantee. For the upsert by HRID for example, if any HRID appears twice in\nthe\nbatch, the module will fall back to record-by-record updates and process the record sets in the order it receives them.\nSame\nfor the upsert by match-key, if any match-key appears twice in the batch.\n\n\n\n## Importing XML source files to Inventory Storage\n\nThe importing component of MIU consists of import \"channels\" equipped with file queues and processing pipelines.\n\nThe only existing pipeline implementation is an XSLT transformation pipeline. It takes an XML 'collection' of 'record'\nelements and -- through custom provided XSLT style-sheets -- transforms them into inventory record set XML. The\nXML is converted to batches of inventory record set JSON records and processed through MIU's inventory upsert APIs.\n\nTo use the XML importing API, the channels must first be configured with a pipeline, which happens through the provided\nconfiguration APIs.\n\n## The importing component\n\nThe main elements of the importing component are\n\n- a \"channel\" with an associated file queue\n- a processing pipeline, called a \"transformation\"\n- the \"transformation\"  has an ordered set of \"transformation steps\" with XSLT style-sheets, that ends with a crosswalk of the XML to JSON.\n- the \"transformation\" also has a \"target\" for its JSON results, which is a component that will batch and persist the results to inventory storage.\n\nSee also the OpenAPI spec for further details [XML importing APIs](src/main/resources/openapi/inventory-import-1.0.yaml).\n\n### Channels\n\nThe static parts of the channel itself are\n- a name for channel\n- a reference to the \"transformation\" that processes the incoming source files\n- two flags that indicates if the channel is deployed (or is to be deployed after an interruption), and is actively listening\n\nThe dynamic parts of a channel are\n\n- a dedicated process (a worker verticle in Vert.x terms) that listens for incoming files\n- file queue: a set of filesystem directories that acts as a queue for incoming files\n- \"importJob\":  if there are any incoming files, and the channel is actively listening, it will automatically launch an \"import job\",\n  using its associated \"transformation\", and the progress of that process will be logged in the objects \"importJob\",\n  \"logLines\", and \"failedRecords\"\n- when the last file in the queue is processed, the job will finish, but the channel will keep listening\n  until paused or decommissioned or until MIU is uninstalled or redeployed.\n\n#### Requests operating on a given channel\n\n| API                                                           | Feature                                                                                                                                                                                                                                                                                                                                          |\n|---------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| POST `/inventory-import/channels`                             | Create a channel, with `enabled` and `listening` settings                                                                                                                                                                                                                                                                                        |\n| PUT `/inventory-import/channels/\u003cchannel uuid\u003e`               | Update properties of a channel, like `name`, `tag`, `enabled`, and `listening`                                                                                                                                                                                                                                                                   |\n| POST `/inventory-import/channels/\u003cchannel id\u003e/commission`     | Launch a channel, marking it enabled. \u003cbr/\u003eThis has the same effect as setting the channel property `channel.enabled`=`true`. However, the command takes an additional parameter `?retainQueue`=[true,false] (default `false`). When `true`, any filesystem queue that was left behind from a previous deployment for this channel will be kept. |\n| POST `/inventory-import/channels/\u003cchannel id\u003e/decommission`   | Undeploy (disable) the channel, has the same effect as setting the channel property `channel.enabled`=`false`, but the command takes an extra parameter `?retainQueue`=[true,false] (default `false`). When set to `true`, the file system queue for this channel is kept and potentially still available if the channel is deployed again.      |\n| POST `/inventory-import/channels/\u003cchannel id\u003e/listen`         | Listen for source files in queue, same effect as setting `channel.listening`=`true`                                                                                                                                                                                                                                                              |\n| POST `/inventory-import/channels/\u003cchannel id\u003e/no-listen`      | Ignore source files in queue, same effect as setting `channel.listening`=`false`                                                                                                                                                                                                                                                                 |\n| POST `/inventory-import/channels/\u003cchannel id\u003e/init-queue`     | Delete all the source files in a queue (or re-establish an empty queue structure, in case the previous queue was deleted directly in the file system for example).                                                                                                                                                                               |\n| DELETE `/inventory-import/channels/\u003cchannel uuid\u003e`            | Delete the channel configuration, including the file queue but not the channel's job history                                                                                                                                                                                                                                                     |\n| POST `/inventory-import/channels/\u003cchannel id\u003e/upload`         | Push a source file to the channel, currently set to accept files up to a size of 100 MB                                                                                                                                                                                                                                                          |\n\n#### Importing\n\n| API                                                    | Feature                                                                                                                                                                                                                                                                                                                                                                     |\n|--------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| POST `/inventory-import/channels/\u003cchannel id\u003e/upload`  | Push a source file to the channel. \u003cbr/\u003eIf the channel is not enabled the upload is refused. If the channel is enabled but not listening, the file will enter the queue. If the channel is listening the file will enter the queue and be imported to Inventory in turn, barring any errors. \u003cbr/\u003eThe service is currently hardcoded to accept files up to a size of 100 MB |\n\n#### Request operating on multiple channels\n\n| API                                                    | Feature                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |\n|--------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| POST `/inventory-import/recover-interrupted-channels`  | Deploy (\"commission\") all channels that are marked `enabled` but are not actually running. This is a scenario that would presumably only occur if the module was restarted while channels were enabled. The command takes a paramter `?listening=false` to enable the channels but not start actual importing right away. \u003cbr/\u003e Notice that when posting a source file to a channel that is `enabled` but not deployed (not \"commissioned\") then the channel will be implicitly commissioned by that post. Explicitly recovering channels by this command is merely to ensure completion of already existing file queues for which processing was interrupted by a module restart. |\n\n\n#### Using `tag` for channel ID\n\nThe \u003cchannel id\u003e in the paths can either be the UUID of the channel record (`channel.id`) or the value of the property\n`channel.tag`. The tag is an optional, unique, max 24 character long string without spaces. If it's set on a channel,\nthat channel can be referenced by the tag in the various channel commands. The basic REST requests (GET, PUT, DELETE channel)\nuse the UUID like standard FOLIO APIs.\n\n### The \"import job\"\n\nA \"job\" is a continuous processing of source files that starts when a channel with an empty queue receives a new source\nfile, and the job ends when the file queue is once again empty. When the job ends, some metrics are calculated\ncovering the span of the job. There can thus only be zero or one job in a channel at any time.\n\nA job is in other words a mostly automatic entity that primarily exists in order to organise the counting of record processes.\nThere are some ways for the operator to handle jobs, though. The start of a job can be controlled, if desired, by pausing the\nchannel listener while uploading source files to the channel. When the listener is restarted, this will trigger a\nnew job. A running job can also be paused and resumed if needed.\n\nIf a fatal error occurs while a job is running, the module will attempt to gracefully pause the job, so that it can potentially\nbe resumed.\n\nFinally a job can be cancelled. This command will include\n\n- stop the channel listener to prevent more files from entering the job\n- calculating metrics for the duration of the job and mark the job cancelled\n- empty the file queue\n\nAfter cancelling the job, the operator must reactivate the listener to allow a new job to start. If some external process is still\nuploading files to the queue, the queue will be populated when the listener is started even though the file queue was emptied\nwhen cancelling the job.\n\n#### Requests operating on jobs:\n\nChannel operations will affect a current job in the channel. Besides that, there are following requests that act on an\nongoing import job directly.\n\n- POST `/inventory-import/channels/\u003cchannel id\u003e/pause-job`\n- POST `/inventory-import/channels/\u003cchannel id\u003e/resume-job`\n- wip - POST `/inventory-import/channels/\u003cchannel id\u003e/cancel-job`\n\n\n\n## Miscellaneous\n\n### `/shared-inventory-upsert-matchkey`\n\nCurrently obsolete.\n\nInserts or updates an Instance based on whether an Instance with the same matchKey exists in storage already. The\nmatchKey is typically generated from a combination of metadata in the bibliographic record, and the API has logic for\nthat, but if an Instance comes in with a ready-made `matchKey`, the end-point will use that instead.\n\n### APIs for fetching an Inventory record set\n\nThere are two REST paths for retrieving single Inventory record sets by ID: `/inventory-upsert-hrid/fetch/{id}`\nand `/shared-inventory-upsert-matchkey/fetch/{id}`. Both APIs will return a record set with an Instance record,\npotentially an array of holdings records, each holdings-record potentially with an array of Item records, and finally a\nset of arrays of external relations that the Instance has with other Instances.\n\n#### Fetching an Inventory record set from `inventory-upsert-hrid/fetch`\n\nThe ID provided on the API path is the Instance HRID. A request like\n`GET /inventory-upsert-hrid/fetch/inst000000000017` would give a response like this (shortened):\n\n```\n{\n  \"instance\" : {\n    \"_version\" : 1,\n    \"hrid\" : \"inst000000000017\",\n    \"source\" : \"FOLIO\",\n    \"title\" : \"Interesting Times\",\n    \"identifiers\" : [ {\n      \"value\" : \"0552142352\",\n      \"identifierTypeId\" : \"8261054f-be78-422d-bd51-4ed9f33c3422\"\n    } ],\n    \"contributors\" : [ {\n      \"name\" : \"Pratchett, Terry\",\n      \"contributorNameTypeId\" : \"2b94c631-fca9-4892-a730-03ee529ffe2a\"\n    } ],\n    \"subjects\" : [ ],\n    ... etc\n    \"statusUpdatedDate\" : \"2021-11-01T23:31:36.026+0100\",\n    \"metadata\" : {\n      \"createdDate\" : \"2021-11-01T22:31:36.025+00:00\",\n      \"updatedDate\" : \"2021-11-01T22:31:36.025+00:00\"\n    },\n  },\n  \"holdingsRecords\" : [ {\n    \"_version\" : 1,\n    \"hrid\" : \"hold000000000007\",\n    \"permanentLocationId\" : \"f34d27c6-a8eb-461b-acd6-5dea81771e70\",\n    ... etc\n    \"metadata\" : {\n      \"createdDate\" : \"2021-11-01T22:31:38.030+00:00\",\n      \"updatedDate\" : \"2021-11-01T22:31:38.030+00:00\"\n    },\n    \"items\" : [ {\n      \"_version\" : 1,\n      \"hrid\" : \"item000000000012\",\n      \"barcode\" : \"326547658598\",\n      ... etc\n      \"status\" : {\n        \"name\" : \"Checked out\",\n        \"date\" : \"2021-11-01T22:31:38.587+00:00\"\n      },\n      \"materialTypeId\" : \"1a54b431-2e4f-452d-9cae-9cee66c9a892\",\n      \"metadata\" : {\n        \"createdDate\" : \"2021-11-01T22:31:38.587+00:00\",\n        \"updatedDate\" : \"2021-11-01T22:31:38.587+00:00\"\n      }\n    } ]\n  } ],\n  \"instanceRelations\" : {\n    \"parentInstances\" : [ ],\n    \"childInstances\" : [ ],\n    \"precedingTitles\" : [ ],\n    \"succeedingTitles\" : [ ]\n  }\n}\n(Note: it's possible to use the Instance UUID instead of the HRID in the GET request)\n```\n\nIt's possible to take the response from the `/inventory-upsert-hrid/fetch` and PUT it back to\nthe `/inventory-upsert-hrid` API.\n\nThere may not be obvious use cases for it but for what it's worth, the response JSON can be edited by, say, setting\n\"editions\" to [\"First edition\"] or adding one more Item, and the record set JSON can then be PUT back\nto `/inventory-upsert-hrid` to perform the updates.\n\nThe response JSON above contains none of the primary key fields, `id`, or referential fields,\n`instanceId` and `holdingsRecordId`, for the three main entities of the Inventory record set. This is because the\n`inventory-upsert-hrid` API is entirely HRID based (at least when viewed from the outside. Internally the module of\ncourse deals with the UUIDs).\n\nThe client of the API is responsible for knowing what the HRIDs for the records are and for ensuring that the\nprovided IDs are indeed unique.\n\n#### The _version fields and optimistic locking\n\nThe `_version` fields for optimistic locking can be seen in the output above. These values would have no effect in a PUT\nto the upsert API. As the service receives the record set JSON in a PUT request, it will pull new versions of the\nentities from storage and get the latest version numbers from that anyway.\n\n       | 2.2.0                          |\n\n## Prerequisites\n\n- Java 21 JDK\n- Maven 3.3.9\n\n## Git Submodules\n\nThere are some common RAML definitions that are shared between FOLIO projects via Git submodules.\n\nTo initialise these please run `git submodule init \u0026\u0026 git submodule update` in the root directory.\n\nIf these are not initialised, the module will fail to build correctly, and other operations may also fail.\n\nMore information is available on\nthe [FOLIO developer site](https://dev.folio.org/guides/developer-setup/#update-git-submodules).\n\n## Building\n\nrun `mvn install` from the root directory.\n\n## Additional information\n\n### Other documentation\n\nOther [modules](https://dev.folio.org/source-code/#server-side) are described, with further FOLIO Developer\ndocumentation at [dev.folio.org](https://dev.folio.org/)\n\n### Code of Conduct\n\nRefer to the Wiki [FOLIO Code of Conduct](https://wiki.folio.org/display/COMMUNITY/FOLIO+Code+of+Conduct).\n\n### Issue tracker\n\nSee project [MODINVUP](https://issues.folio.org/browse/MODINVUP)\nat the [FOLIO issue tracker](https://dev.folio.org/guidelines/issue-tracker).\n\n### ModuleDescriptor\n\nSee the [ModuleDescriptor](descriptors/ModuleDescriptor-template.json)\nfor the interfaces that this module requires and provides, the permissions, and the additional module metadata.\n\n### API documentation\n\n* [Updating (OpenAPI)](src/main/resources/openapi/inventory-update-5.0.yaml)\n* [Importing (OpenAPI)](src/main/resources/openapi/inventory-import-1.0.yaml)\n\nGenerated [API documentation](https://dev.folio.org/reference/api/#mod-inventory-update).\n\n### Code analysis\n\n[SonarQube analysis](https://sonarcloud.io/dashboard?id=org.folio%3Amod-inventory-update).\n\n### Download and configuration\n\nThe built artifacts for this module are available. See [configuration](https://dev.folio.org/download/artifacts) for\nrepository access, and the [Docker image](https://hub.docker.com/r/folioorg/mod-inventory-update/).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffolio-org%2Fmod-inventory-update","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffolio-org%2Fmod-inventory-update","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffolio-org%2Fmod-inventory-update/lists"}