{"id":22281214,"url":"https://github.com/folio-org/mod-data-export-spring","last_synced_at":"2026-04-16T20:02:22.068Z","repository":{"id":38198458,"uuid":"333973935","full_name":"folio-org/mod-data-export-spring","owner":"folio-org","description":null,"archived":false,"fork":false,"pushed_at":"2024-12-02T14:35:27.000Z","size":941,"stargazers_count":3,"open_issues_count":0,"forks_count":5,"subscribers_count":26,"default_branch":"master","last_synced_at":"2024-12-02T15:37:17.203Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/folio-org.png","metadata":{"files":{"readme":"README.md","changelog":"NEWS.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-28T22:33:58.000Z","updated_at":"2024-11-29T14:42:26.000Z","dependencies_parsed_at":"2023-12-21T22:29:10.608Z","dependency_job_id":"bcebb70a-6d47-479e-b4bc-95fae0b0b414","html_url":"https://github.com/folio-org/mod-data-export-spring","commit_stats":null,"previous_names":[],"tags_count":50,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Fmod-data-export-spring","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Fmod-data-export-spring/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Fmod-data-export-spring/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/folio-org%2Fmod-data-export-spring/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/folio-org","download_url":"https://codeload.github.com/folio-org/mod-data-export-spring/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227952056,"owners_count":17846350,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-03T16:16:16.478Z","updated_at":"2026-04-16T20:02:17.013Z","avatar_url":"https://github.com/folio-org.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mod-data-export-spring\n\nCopyright (C) 2021-2023 The Open Library Foundation\n\nThis software is distributed under the terms of the Apache License,\nVersion 2.0. See the file [LICENSE](LICENSE) for more information.\n\n## Introduction\nAPI for Data Export Spring module.\n\n## Environment variables:\n\n| Name                                    |      Default value      | Description                                                                                                                                            |\n|:----------------------------------------|:-----------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------|\n| DB_HOST                                 |        postgres         | Postgres hostname                                                                                                                                      |\n| DB_PORT                                 |          5432           | Postgres port                                                                                                                                          |\n| DB_USERNAME                             |       folio_admin       | Postgres username                                                                                                                                      |\n| DB_PASSWORD                             |            -            | Postgres username password                                                                                                                             |\n| DB_DATABASE                             |      okapi_modules      | Postgres database name                                                                                                                                 |\n| KAFKA_HOST                              |          kafka          | Kafka broker hostname                                                                                                                                  |\n| KAFKA_PORT                              |          9092           | Kafka broker port                                                                                                                                      |\n| OKAPI_URL                               |    http://okapi:9130    | Okapi url                                                                                                                                              |\n| SYSTEM\\_USER\\_NAME                      | data-export-system-user | Username of the system user                                                                                                                            |\n| SYSTEM\\_USER\\_PASSWORD                  |            -            | Password of the system user                                                                                                                            |\n| SYSTEM\\_USER\\_ENABLED                   |          true           | Defines if system user must be created at service tenant initialization or used for egress service requests                                            |\n| ENV                                     |          folio          | Logical name of the deployment, must be set if Kafka/Elasticsearch are shared for environments, `a-z (any case)`, `0-9`, `-`, `_` symbols only allowed |\n| JOB_EXPIRATION_PERIOD_DAYS              |            7            | Expiration period of export jobs, acquisition jobs have no expiration, bulk edit jobs use own rules to define it with using mod-configuration          |\n| JOB_DOWNLOAD_FILE_CONNECTION_TIMEOUT_MS |          5000           | Connection timeout to download exported file from storage                                                                                              |\n\n\n## Additional information\nData Export Spring API provides the following URLs:\n\n|  Method | URL| Permissions  | Description  | \n|---|---|---|---|\n| GET  | /data-export-spring/jobs/        | data-export.job.collection.get    | Gets jobs                                |\n| GET  | /data-export-spring/jobs/{id}    | data-export.job.item.get          | Gets a job by the job ID                 |\n| POST | /data-export-spring/jobs/        | data-export.job.item.post         | Upserts a job                            |\n| GET  | /data-export-spring/configs/     | data-export.config.collection.get | Get a list of data export configurations |\n| PUT  | /data-export-spring/configs/{id} | data-export.config.item.put       | Change an export configuration           |\n| POST | /data-export-spring/configs/     | data-export.config.item.post      | Add an export configuration              |\n\nMore detail can be found on Data Export Spring wiki-page: [WIKI Data Export Spring](https://wiki.folio.org/pages/viewpage.action?pageId=52134948).\n\n### Required Permissions\nInstitutional users should be granted the following permissions in order to use this Data Export Spring API:\n- data-export.config.all\n- data-export.job.all\n\n### Deployment information\n#### Before Poppy release\n![](https://img.shields.io/static/v1?label=\u0026message=!WARNING\u0026color=orange)\n Only ONE instance should be running until the issues described below are fixed: \n\n1. If more than one instance of the module is running, then the same export tasks will be launched by all instances.\nAs a result, information will be duplicated.\n\nMore details:\n[Prevents execution of the same scheduled export task from another node](https://issues.folio.org/browse/MODEXPS-75)\n\n2. If instance of the module will be restarted by Kubernetes or manually and no need to register modules again,\nbecause version is not changed. As a result, mandatory information for scheduling (Okapi headers, system user, tenant information)\nwill not be stored in memory in a FolioExecutionContext.\n\nMore details:\n[Export scheduling doesn't support work in the cluster and after restarting docker container](https://issues.folio.org/browse/MODEXPS-81)\n\n##### Short overview\nBefore running scheduled task(job) there is check, that module is registered for the Okapi tenant.\n\nTenant information need to define DB schema for storing information about Job and etc.\n\nThe `data-export-system-user` system user for running scheduled export tasks is created in the post tenant API controller. The password must be set using the `SYSTEM_USER_PASSWORD` environment variable. Permissions are defined in `src/main/resources/permissions/system-user-permissions.csv`.\n\nAlso Okapi headers, system user, tenant information are s-tored in memory in a FolioExecutionContext.\n\n#### Since Poppy release\nScheduling was changed to quartz: [Quartz Scheduling Implementation](https://wiki.folio.org/display/DD/Quartz+Scheduling+Implementation+in+mod-data-export-spring).\nThe issues above were fixed, there's no need to reenable module after restarts and it can be scaled.\n##### Migration to quartz scheduling\n1. Migration is done once automatically on module upgrade from version which does not support quartz to version supporting quartz (based on `moduleFrom` and `moduleTo` versions in TenantAttributes).\n2. In case reloading of existing schedules needs to be forced, it can be done with setting `forceSchedulesReload=true` parameter in TenantAttributes in module enable request. [Example](https://wiki.folio.org/display/DD/Quartz+Scheduling+Implementation+in+mod-data-export-spring#QuartzSchedulingImplementationinmoddataexportspring-Migrationtoquartz) \n3. After new version supporting quartz is deployed and enabled for tenants, old module version has to be stopped, otherwise jobs will be executed by both old version with spring scheduler and new version with quartz.\n##### Module disabling for tenant\nTenant's schedules deletion is done in scope of [module disable with purge](https://github.com/folio-org/okapi/blob/master/doc/guide.md#purge-module-data).\nIf disabling with purge is not invoked for mod-data-export-spring, tenant's scheduled jobs will continue to run in the background even after tenant itself is deleted. [Details](https://wiki.folio.org/display/DD/Quartz+Scheduling+Implementation+in+mod-data-export-spring)\n\n### Issue tracker\nSee project [MODEXPS](https://issues.folio.org/browse/MODEXPS)\nat the [FOLIO issue tracker](https://dev.folio.org/guidelines/issue-tracker).\n\n### Other documentation\nOther [modules](https://dev.folio.org/source-code/#server-side) are described,\nwith further FOLIO Developer documentation at\n[dev.folio.org](https://dev.folio.org/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffolio-org%2Fmod-data-export-spring","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffolio-org%2Fmod-data-export-spring","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffolio-org%2Fmod-data-export-spring/lists"}