{"id":21660126,"url":"https://github.com/slub/ocrd_manager","last_synced_at":"2025-10-24T10:40:21.219Z","repository":{"id":38241648,"uuid":"460003899","full_name":"slub/ocrd_manager","owner":"slub","description":"frontend for ocrd_controller and adapter towards ocrd_kitodo","archived":false,"fork":false,"pushed_at":"2024-03-11T17:00:44.000Z","size":6748,"stargazers_count":10,"open_issues_count":6,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-25T18:41:16.515Z","etag":null,"topics":["ocr-d"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/slub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-02-16T12:55:10.000Z","updated_at":"2024-08-02T02:23:18.000Z","dependencies_parsed_at":"2023-12-18T18:24:49.345Z","dependency_job_id":"fadfa7d7-19af-492e-87c5-e5d4f2a96bd4","html_url":"https://github.com/slub/ocrd_manager","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Focrd_manager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Focrd_manager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Focrd_manager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Focrd_manager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/slub","download_url":"https://codeload.github.com/slub/ocrd_manager/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248493022,"owners_count":21113159,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ocr-d"],"created_at":"2024-11-25T09:32:17.308Z","updated_at":"2025-10-24T10:40:16.165Z","avatar_url":"https://github.com/slub.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OCR-D Manager\n\nOCR-D Manager is a server that mediates between [Kitodo](https://github.com/kitodo)\nand [OCR-D](https://ocr-d.de). It resides on the site of the Kitodo installation\n(so the actual OCR server can be managed independently) but runs in its own container\n(so Kitodo can be managed independently).\n\nSpecifically, it gets called by [Kitodo.Production](https://github.com/kitodo/kitodo-production)\nor [Kitodo.Presentation](https://github.com/kitodo-presentation) to handle OCR for a document,\nand in turn calls the [OCR-D Controller](https://github.com/slub/ocrd_controller) for workflow processing.\n\nFor an integration as a **service container**, orchestrated with other containers (Kitodo+Controller+Monitor),\nsee [this meta-repo](https://github.com/slub/ocrd_kitodo).\n\nOCR-D Manager is responsible for\n- data transfer from Kitodo to Manager to Controller and back,\n- delegation to Controller,\n- signalling/reporting,\n- result validation,\n- result extraction (putting ALTO files in the process directory where Kitodo.Production expects them,\n  or updating the METS for Kitodo.Presentation).\n\nIt is currently implemented as SSH login server with an installation of [OCR-D core](https://github.com/OCR-D/core)\nand an SSH client to connect to the Controller.\n\n * [Usage](#usage)\n   * [Building](#building)\n   * [Starting and mounting](#starting-and-mounting)\n   * [General management](#general-management)\n   * [Processing](#processing)\n     * [From image to ALTO files](#from-image-to-alto-files)\n     * [From METS to METS file](#from-mets-to-mets-file)\n   * [Data transfer](#data-transfer)\n   * [Logging](#logging)\n * [Testing](#testing)\n\n## Usage\n\n### Building\n\nBuild or pull the Docker image:\n\n    make build # or docker pull ghcr.io/slub/ocrd_manager\n\n### Starting and mounting\n\nThen run the container – providing a **host-side directory** for the volumes …\n\n * `DATA`: directory for data processing (including images or existing workspaces),  \n   defaults to current working directory\n * `WORKFLOWS`: directory for scripts (preconfigured workflows),  \n   defaults to `./workflows` in current working directory\n\n… but also files …\n\n * `KEYS`: public key **credentials** for log-in to the manager\n * `PRIVATE`: private key **credentials** for log-in to the controller …\n\n… and (optionally) some **environment variables** …\n\n * `UID`: numerical user identifier to be used by programs in the container  \n    (will affect the files modified/created); defaults to current user\n * `GID`: numerical group identifier to be used by programs in the container  \n    (will affect the files modified/created); defaults to current group\n * `UMASK`: numerical user mask to be used by programs in the container  \n    (will affect the files modified/created); defaults to 0002\n * `PORT`: numerical TCP port to expose the SSH server on the host side  \n    defaults to 9022 (for non-priviledged access)\n * `CONTROLLER` network address:port for the controller client\n\t\t\t(must be reachable from the container network)\n * `ACTIVEMQ` network address:port of ActiveMQ server listening to result status\n\t\t\t(must be reachable from the container network)\n * `NETWORK` name of the Docker network to use  \n    defaults to `bridge` (the default Docker network)\n\n… thus, for **example**:\n\n    make run DATA=/mnt/workspaces WORKFLOWS=/mnt/workflows KEYS=~/.ssh/id_rsa.pub PORT=9022 PRIVATE=~/.ssh/id_rsa\n\n(You can also run the service via `docker-compose` manually – just `cp .env.example .env` and edit to your needs.)\n\n### General management\n\nThen you can **log in** as user `ocrd` from remote (but let's use `manager` in the following –\nwithout loss of generality):\n\n    ssh -p 9022 ocrd@manager bash -i\n\n(Typically though, you will run a non-interactive script, see next section.)\n\n### Processing\n\nIn the Manager, you can run shell scripts that do\n- data management and validation via `ocrd` CLIs\n- OCR processing by running workflows in the controller via `ssh ocrd@ocrd_controller` log-ins\n\nThe data management will depend on which Kitodo context you want to integrate into (Production 2 / 3 or Presentation).\n\n#### From image to ALTO files\n\nFor **Kitodo.Production**, there is a preconfigured script `process_images.sh` (or `for_production.sh`)\nwhich takes the following arguments:\n\n```sh\nSYNOPSIS:\n\nprocess_images.sh [OPTIONS] DIRECTORY\n\nwhere OPTIONS can be any/all of:\n --lang LANGUAGE    overall language of the material to process via OCR\n --script SCRIPT    overall script of the material to process via OCR\n --workflow FILE    workflow file to use for processing, default:\n                    ocr-workflow-default.sh\n --no-validate      skip comprehensive validation of workflow results\n --img-subdir IMG   name of the subdirectory to read images from, default:\n                    images\n --ocr-subdir OCR   name of the subdirectory to write OCR results to, default:\n                    ocr/alto\n --proc-id ID       process ID to communicate in ActiveMQ callback\n --task-id ID       task ID to communicate in ActiveMQ callback\n --help             show this message and exit\n\nand DIRECTORY is the local path to process. The script will import\nthe images from DIRECTORY/IMG into a new (temporary) METS and\ntransfer this to the Controller for processing. After resyncing back\nto the Manager, it will then extract OCR results and export them to\nDIRECTORY/OCR.\n\nIf ActiveMQ is used, the script will exit directly after initialization,\nand run processing in the background. Completion will then be signalled\nvia ActiveMQ network protocol (using the proc and task ID as message).\n\nENVIRONMENT VARIABLES:\n\n CONTROLLER: host name and port of OCR-D Controller for processing\n ACTIVEMQ: URL of ActiveMQ server for result callback (optional)\n ACTIVEMQ_CLIENT: path to ActiveMQ client library JAR file (optional)\n```\n\nThe `workflow` parameter is optional and defaults to the preconfigured script `ocr-workflow-default.sh`\nwhich contains a trivial workflow:\n- import of the images into a new OCR-D workspace\n- preprocessing, layout analysis and text recognition with a single Tesseract processor call\n- format conversion of the result from PAGE-XML to ALTO-XML\n\nIt can be replaced with the (path) name of any workflow script mounted under `/workflows` or `/data`.\n\nFor example (assuming `testdata` is a directory with image files mounted under `/data`):\n\n    ssh -T -p 9022 ocrd@manager process_images.sh --proc-id 1 --task-id 3 --lang deu --script Fraktur --workflow myocr.sh testdata\n\n\n#### From METS to METS file\n\nFor **Kitodo.Presentation**, there is a preconfigured script `process_mets.sh` (or `for_presentation.sh`)\nwhich takes the following arguments:\n\n```sh\nSYNOPSIS:\n\nprocess_mets.sh [OPTIONS] METS\n\nwhere OPTIONS can be any/all of:\n --workflow FILE    workflow file to use for processing, default:\n                    ocr-workflow-default.sh\n --no-validate      skip comprehensive validation of workflow results\n --pages RANGE      selection of physical page range to process\n --img-grp GRP      fileGrp to read input images from, default:\n                    DEFAULT\n --ocr-grp GRP      fileGrp to write output OCR text to, default:\n                    FULLTEXT\n --url-prefix URL   convert result text file refs from local to URL\n                    and prefix them\n --help             show this message and exit\n\nand METS is the path of the METS file to process. The script will copy\nthe METS into a new (temporary) workspace and transfer this to the\nController for processing. After resyncing back, it will then extract\nOCR results and copy them to METS (adding file references to the file\nand copying files to the parent directory).\n\nENVIRONMENT VARIABLES:\n\n CONTROLLER: host name and port of OCR-D Controller for processing\n```\n\nFor the `workflow` parameter, the same goes here as [above](#from-image-to-alto-files).\n\nFor example (assuming `testdata` is a directory with image files mounted under `/data`):\n\n    ssh -T -p 9022 ocrd@manager process_mets.sh --lang deu --script Fraktur --workflow myocr.sh testdata/mets.xml\n\n\n### Data transfer\n\nFor sharing data between the Manager and Controller, it is recommended to transfer files _explicitly_\n(as this will make the costs more measurable and controllable).\n\n(This is currently implemented via `rsync`.)\n\nThe data lifecycle should be:\n- on Controller: short-lived\n- on Manager: as long as process is active in Production\n\n(This is currently not managed.)\n\n### Logging\n\nAll logs are accumulated on standard output, which can be inspected via Docker:\n\n    docker logs ocrd_manager\n\nLogs for all services can also be viewed on the [Monitor web server](https://slub.github.io/ocrd_kitodo/usage/ocrd-monitor).\n\n## Testing\n\nAfter [building](#building) and [starting](#starting-and-mounting), you can use the `test` target\nfor a round-trip:\n\n    make test DATA=/mnt/workspaces\n\nThis will download sample data and run the default workflow on them.\n(All logging is still accumulated on the Docker output, so the shell\nitself will not print any. See [above](#logging))\n\n(If the Manager has been started externally already, make sure to pass the correct value\n for the `NETWORK` variable – the makefile will then attempt to use `docker exec` instead of\n `ssh ocrd@localhost` to connect.)\n\nTo clean up the results, use:\n\n    make clean-testdata\n\n## Maintainers\n\nIf you have any questions or encounter any problems, please do not hesitate to contact us.\n\n- [Robert Sachunsky](https://github.com/bertsky)\n- [Markus Weigelt](https://github.com/markusweigelt)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslub%2Focrd_manager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fslub%2Focrd_manager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslub%2Focrd_manager/lists"}