{"id":21198867,"url":"https://github.com/natlibfi/ekirjasto-circulation","last_synced_at":"2025-03-14T22:21:08.544Z","repository":{"id":219238445,"uuid":"730986936","full_name":"NatLibFi/ekirjasto-circulation","owner":"NatLibFi","description":null,"archived":false,"fork":false,"pushed_at":"2024-04-12T22:14:32.000Z","size":62511,"stargazers_count":1,"open_issues_count":5,"forks_count":0,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-04-14T11:41:09.228Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NatLibFi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-12-13T05:35:59.000Z","updated_at":"2024-04-17T21:28:07.484Z","dependencies_parsed_at":"2024-04-17T21:27:41.659Z","dependency_job_id":"41e83c52-d72d-4c28-8438-041cb5e7d5f9","html_url":"https://github.com/NatLibFi/ekirjasto-circulation","commit_stats":null,"previous_names":["natlibfi/ekirjasto-circulation"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NatLibFi%2Fekirjasto-circulation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NatLibFi%2Fekirjasto-circulation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NatLibFi%2Fekirjasto-circulation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NatLibFi%2Fekirjasto-circulation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NatLibFi","download_url":"https://codeload.github.com/NatLibFi/ekirjasto-circulation/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243653392,"owners_count":20325734,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-20T19:53:51.039Z","updated_at":"2025-03-14T22:21:08.511Z","avatar_url":"https://github.com/NatLibFi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# E-kirjasto Circulation Manager\n\n[![Test \u0026 Build](https://github.com/NatLibFi/ekirjasto-circulation/actions/workflows/test-build.yml/badge.svg)](https://github.com/NatLibFi/ekirjasto-circulation/actions/workflows/test-build.yml)\n\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat\u0026labelColor=ef8336)](https://pycqa.github.io/isort/)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white)](https://github.com/pre-commit/pre-commit)\n![Python: 3.10,3.11](https://img.shields.io/badge/Python-3.10%20|%203.11-blue)\nThis is the E-kirjasto fork of the [The Palace Project](https://thepalaceproject.org) Palace Manager (which is a fork of\n[Library Simplified](http://www.librarysimplified.org/) Circulation Manager).\n\n## Installation\n\nDocker images created from this code will be available at:\n\n- [ekirjasto-circ-webapp](https://github.com/NatLibFi/circulation/pkgs/container/ekirjasto-circ-webapp)\n- [ekirjasto-circ-scripts](https://github.com/NatLibFi/circulation/pkgs/container/ekirjasto-circ-scripts)\n- [ekirjasto-circ-exec](https://github.com/NatLibFi/circulation/pkgs/container/ekirjasto-circ-exec)\n\nDocker images are the preferred way to deploy this code in a production environment.\n\n## Git Branch Workflow\n\nThe default branch is `main` and that's the working branch that should be used when branching off for bug fixes or new\nfeatures.\n\n## Set Up\n\n### Using Makefile to set up everything for development\n\nBefore running the commands in the Makefile, you should have `docker-compose-dev.yml` file in your directory. Also, you\nshould change the url of `EKIRJASTO_AUTHENTICATION_URL` in the Makefile's environment variables section to the real one.\n\nTo install dependencies and packages, run:\n\n```shell\nmake install\n```\n\nFor pyenv to work, add the following to your `.zshrc`:\n\n```sh\nexport PYENV_ROOT=\"$HOME/.pyenv\"\ncommand -v pyenv \u003e/dev/null || export PATH=\"$PYENV_ROOT/bin:$PATH\"\neval \"$(pyenv init -)\"\neval \"$(pyenv virtualenv-init -)\"\n```\n\nand source it with:\n\n```shell\nsource ~/.zshrc\n```\n\nCreate a virtual environment with Python 3.11.1, run:\n\n```shell\nmake venv\n```\n\nActivate the virtual environment and install dependencies with Poetry:\n\n```shell\npyenv activate circ-311\npoetry install\n```\n\nIf all went well, you should see the activated virtual environment `(circ-311)` in your shell and all packages\ninstalled. You can verify this by checking the Python version:\n\n```shell\npython3 --version\n```\n\nNow, all you need to do to start docker containers and run the application is:\n\n```shell\nmake run\n```\n\nWhen you need to stop the application and containers and delete everything to start fresh, first check the path of the\npostgres docker container volume in `docker-compose-dev.yml`. Update the `POSTGRES_DATA` in the Makefile to match it\nand then run:\n\n```shell\nmake clean\n```\n\nYou can modify the Makefile to also create a virtual enviroment for Python 3.10.1. You can switch between the virtual\nenvironments by activating and deactivating them with e.g. `pyenv activate circ-311` and `pyenv deactivate circ-311`.\n\n### Docker Compose\n\nIn order to help quickly set up a development environment, we include a [docker-compose.yml](./docker-compose.yml)\nfile. This docker-compose file, will build the webapp and scripts containers from your local repository, and start\nthose containers as well as all the necessary service containers.\n\nYou can give this a try by running the following command:\n\n```shell\ndocker-compose up --build\n```\n\n### Python Set Up\n\n#### Homebrew (OSX)\n\nIf you do not have Python 3 installed, you can use [Homebrew](https://brew.sh/) to install it by running the command\n`brew install python3`.\n\nIf you do not yet have Homebrew, you can install it by running the following:\n\n```sh\n/bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"\n```\n\nWhile you're at it, go ahead and install the following required dependencies:\n\n```sh\nbrew install pkg-config libffi\nbrew install libxmlsec1\nbrew install libjpeg\n```\n\n#### Linux\n\nMost distributions will offer Python packages. On Arch Linux, the following command is sufficient:\n\n```sh\npacman -S python\n```\n\nYou need to install dependencies: \u003chttps://devguide.python.org/getting-started/setup-building/#build-dependencies\u003e\n\nEnable Source Packages:\nUncomment a deb-src in `/etc/apt/sources.list` e.g. `jammy main`\n\nInstall build dependencies:\n\n```sh\nsudo apt-get update\nsudo apt-get build-dep python3\nsudo apt-get install pkg-config\nsudo apt install libxmlsec1 libxmlsec1-dev\n```\n\n#### pyenv (Optional)\n\n[pyenv](https://github.com/pyenv/pyenv) pyenv lets you easily switch between multiple versions of Python. It can be\n[installed](https://github.com/pyenv/pyenv-installer) using the command `curl https://pyenv.run | bash`. You can then\ninstall the version of Python you want to work with.\n\nCheck if you already have pyenv-virtualenv as a plugin with your pyenv:\n\n```sh\nls $PYENV_ROOT/plugins/pyenv-virtualenv/\n```\n\nIf you have it installed already you can skip the next part.\n\nIt is recommended that [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv) be used to allow `pyenv`\nto manage _virtual environments_ in a manner that can be used by the [poetry](#poetry) tool. The `pyenv-virtualenv`\nplugin can be installed by cloning the relevant repository into the `plugins` subdirectory of your `$PYENV_ROOT`:\n\n```sh\nmkdir -p $PYENV_ROOT/plugins\ncd $PYENV_ROOT/plugins\ngit clone https://github.com/pyenv/pyenv-virtualenv\n```\n\nAfter cloning the repository, `pyenv` now has a new `virtualenv` command:\n\n```sh\n$ pyenv virtualenv\npyenv-virtualenv: no virtualenv name given.\n```\n\n#### Poetry\n\nYou will need to set up a local virtual environment to install packages and run the project. This project uses\n[poetry](https://python-poetry.org/) for dependency management.\n\nPoetry can be installed using the command `curl -sSL https://install.python-poetry.org | python3 -`.\n\nMore information about installation options can be found in the\n[poetry documentation](https://python-poetry.org/docs/master/#installation).\n\n### OpenSearch\n\nPalace now supports OpenSearch: please use it instead of Elasticsearch.\nElasticsearch is no longer supported.\n\n#### Docker\n\nWe recommend that you run OpenSearch with docker using the following docker commands:\n\n```sh\ndocker run --name opensearch -d --rm -p 9200:9200 -e \"discovery.type=single-node\" -e \"plugins.security.disabled=true\" \"opensearchproject/opensearch:1\"\ndocker exec opensearch opensearch-plugin -s install analysis-icu\ndocker restart opensearch\n```\n\n### Database\n\n#### Docker\n\n```sh\ndocker run -d --name pg -e POSTGRES_USER=palace -e POSTGRES_PASSWORD=test -e POSTGRES_DB=circ -p 5432:5432 postgres:12\n```\n\nYou can run `psql` in the container using the command\n\n```sh\ndocker exec -it pg psql -U palace circ\n```\n\n#### Local\n\n1. Download and install [Postgres](https://www.postgresql.org/download/) if you don't have it already.\n2. Use the command `psql` to access the Postgresql client.\n3. Within the session, run the following commands:\n\n```sh\nCREATE DATABASE circ;\nCREATE USER palace with password 'test';\ngrant all privileges on database circ to palace;\n```\n\n### Environment variables\n\n#### Database\n\nTo let the application know which database to use, set the `SIMPLIFIED_PRODUCTION_DATABASE` environment variable.\n\n```sh\nexport SIMPLIFIED_PRODUCTION_DATABASE=\"postgresql://palace:test@localhost:5432/circ\"\n```\n\n#### Opensearch\n\nTo let the application know which Opensearch instance to use, you can set the following environment variables:\n\n- `PALACE_SEARCH_URL`: The url of the Opensearch instance (**required**).\n- `PALACE_SEARCH_INDEX_PREFIX`: The prefix to use for the Opensearch indices. The default is `circulation-works`.\n    This is useful if you want to use the same Opensearch instance for multiple CM (optional).\n- `PALACE_SEARCH_TIMEOUT`: The timeout in seconds to use when connecting to the Opensearch instance. The default is `20`\n  (optional).\n- `PALACE_SEARCH_MAXSIZE`: The maximum size of the connection pool to use when connecting to the Opensearch instance.\n  (optional).\n\n```sh\nexport PALACE_SEARCH_URL=\"http://localhost:9200\"\n```\n\n#### Storage Service\n\nThe application optionally uses a s3 compatible storage service to store files. To configure the application to use\na storage service, you can set the following environment variables:\n\n- `PALACE_STORAGE_PUBLIC_ACCESS_BUCKET`: Required if you want to use the storage service to serve files directly to\n  users. This is the name of the bucket that will be used to serve files. This bucket should be configured to allow\n  public access to the files.\n- `PALACE_STORAGE_ANALYTICS_BUCKET`: Required if you want to use the storage service to store analytics data.\n- `PALACE_STORAGE_ACCESS_KEY`: The access key (optional).\n    - If this key is set it will be passed to boto3 when connecting to the storage service.\n    - If it is not set boto3 will attempt to find credentials as outlined in their\n    [documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials).\n- `PALACE_STORAGE_SECRET_KEY`: The secret key (optional).\n- `PALACE_STORAGE_REGION`: The AWS region of the storage service (optional).\n- `PALACE_STORAGE_ENDPOINT_URL`: The endpoint of the storage service (optional). This is used if you are using a\n  s3 compatible storage service like [minio](https://min.io/).\n- `PALACE_STORAGE_URL_TEMPLATE`: The url template to use when generating urls for files stored in the storage service\n  (optional).\n    - The default value is `https://{bucket}.s3.{region}.amazonaws.com/{key}`.\n    - The following variables can be used in the template:\n        - `{bucket}`: The name of the bucket.\n        - `{key}`: The key of the file.\n        - `{region}`: The region of the storage service.\n\n#### Reporting\n\n- `PALACE_REPORTING_NAME`: (Optional) A name used to identify the CM instance associated with generated reports.\n- `SIMPLIFIED_REPORTING_EMAIL`: (Required) Email address of recipient of reports.\n\n#### Logging\n\nThe application uses the [Python logging](https://docs.python.org/3/library/logging.html) module for logging. Optionally\nlogs can be configured to be sent to AWS CloudWatch logs. The following environment variables can be used to configure\nthe logging:\n\n- `PALACE_LOG_LEVEL`: The log level to use for the application. The default is `INFO`.\n- `PALACE_LOG_VERBOSE_LEVEL`: The log level to use for particularly verbose loggers. Keeping these loggers at a\n  higher log level by default makes it easier to troubleshoot issues. The default is `WARNING`.\n- `PALACE_LOG_CLOUDWATCH_ENABLED`: Enable / disable sending logs to CloudWatch. The default is `false`.\n- `PALACE_LOG_CLOUDWATCH_REGION`: The AWS region of the CloudWatch logs. This must be set if using CloudWatch logs.\n- `PALACE_LOG_CLOUDWATCH_GROUP`: The name of the CloudWatch log group to send logs to. Default is `palace`.\n- `PALACE_LOG_CLOUDWATCH_STREAM`: The name of the CloudWatch log stream to send logs to. Default is\n  `{machine_name}/{program_name}/{logger_name}/{process_id}`. See\n  [watchtower docs](https://github.com/kislyuk/watchtower#log-stream-naming) for details.\n- `PALACE_LOG_CLOUDWATCH_INTERVAL`: The interval in seconds to send logs to CloudWatch. Default is `60`.\n- `PALACE_LOG_CLOUDWATCH_CREATE_GROUP`: Whether to create the log group if it does not exist. Default is `true`.\n- `PALACE_LOG_CLOUDWATCH_ACCESS_KEY`: The access key to use when sending logs to CloudWatch. This is optional.\n    - If this key is set it will be passed to boto3 when connecting to CloudWatch.\n    - If it is not set boto3 will attempt to find credentials as outlined in their\n    [documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials).\n- `PALACE_LOG_CLOUDWATCH_SECRET_KEY`: The secret key to use when sending logs to CloudWatch. This is optional.\n\n#### Patron `Basic Token` authentication\n\nEnables/disables patron \"basic token\" authentication through setting the designated environment variable to any\n(case-insensitive) value of \"true\"/\"yes\"/\"on\"/\"1\" or \"false\"/\"no\"/\"off\"/\"0\", respectively.\nIf the value is the empty string or the variable is not present in the environment, it is disabled by default.\n\n- `SIMPLIFIED_ENABLE_BASIC_TOKEN_AUTH`\n\n```sh\nexport SIMPLIFIED_ENABLE_BASIC_TOKEN_AUTH=true\n```\n\n#### Firebase Cloud Messaging\n\nFor Firebase Cloud Messaging (FCM) support (e.g., for notifications), `one` (and only one) of the following should be set:\n\n- `SIMPLIFIED_FCM_CREDENTIALS_JSON` - the JSON-format Google Cloud Platform (GCP) service account key or\n- `SIMPLIFIED_FCM_CREDENTIALS_FILE` - the name of the file containing that key.\n\n```sh\nexport SIMPLIFIED_FCM_CREDENTIALS_JSON='{\"type\":\"service_account\",\"project_id\":\"\u003cid\u003e\", \"private_key_id\":\"f8...d1\", ...}'\n```\n\n...or...\n\n```sh\nexport SIMPLIFIED_FCM_CREDENTIALS_FILE=\"/opt/credentials/fcm_credentials.json\"\n```\n\nThe FCM credentials can be downloaded once a Google Service account has been created.\nMore details in the [FCM documentation](https://firebase.google.com/docs/admin/setup#set-up-project-and-service-account)\n\n#### Quicksight Dashboards\n\nFor generating quicksight dashboard links the following environment variable is required\n`QUICKSIGHT_AUTHORIZED_ARNS` - A dictionary of the format `\"\u003cdashboard name\u003e\": [\"arn:aws:quicksight:...\",...]`\nwhere each quicksight dashboard gets treated with an arbitrary \"name\", and a list of \"authorized arns\".\nThe first the \"authorized arns\" is always considered as the `InitialDashboardID` when creating an embed URL\nfor the respective \"dashboard name\".\n\n#### Analytics\n\nLocal analytics are enabled by default. S3 analytics can be enabled via the following environment variable:\n\n- PALACE_S3_ANALYTICS_ENABLED: A boolean value to disable or enable s3 analytics. The default is false.\n\n#### OpenSearch Analytics (E-Kirjasto, Finland)\n\nOpenSearch analytics can be enabled via the following environment variables:\n\n- PALACE_OPENSEARCH_ANALYTICS_ENABLED: A boolean value to disable or enable OpenSearch analytics. The default is false.\n- PALACE_OPENSEARCH_ANALYTICS_URL: The url of your OpenSearch instance, eg. \"http://localhost:9200\"\n- PALACE_OPENSEARCH_ANALYTICS_INDEX_PREFIX: The prefix of the event index name, eg. \"circulation-events\"\n\n#### Email\n\nTo use the features that require sending emails, for example to reset the password for logged-out users, you will need\nto have a working SMTP server and set some environment variables:\n\n```sh\nexport SIMPLIFIED_MAIL_SERVER=example.smtp.com\nexport SIMPLIFIED_MAIL_PORT=465\nexport SIMPLIFIED_MAIL_USERNAME=username\nexport SIMPLIFIED_MAIL_PASSWORD=password\nexport SIMPLIFIED_MAIL_SENDER=sender@example.com\n```\n\n## Running the Application\n\nAs mentioned in the [pyenv](#pyenv) section, the `poetry` tool should be executed under a virtual environment\nin order to guarantee that it will use the Python version you expect. To use a particular Python version,\nyou should create a local virtual environment in the cloned `circulation` repository directory. Assuming that\nyou want to use, for example, Python 3.11.1:\n\n```sh\npyenv virtualenv 3.11.1 circ\n```\n\nThis will create a new local virtual environment called `circ` that uses Python 3.11.1. Switch to that environment:\n\n```sh\npyenv local circ\n```\n\nOn most systems, using `pyenv` will adjust your shell prompt to indicate which virtual environment you\nare now in. For example, the version of Python installed in your operating system might be `3.10.1`, but\nusing a virtual environment can substitute, for example, `3.11.1`:\n\n```sh\n$ python --version\nPython 3.10.1\n\n$ pyenv local circ\n(circ) $ python --version\nPython 3.11.1\n```\n\nFor brevity, these instructions assume that all shell commands will be executed within a virtual environment.\n\nInstall the dependencies (including dev and CI):\n\n```sh\npoetry install\n```\n\nInstall only the production dependencies:\n\n```sh\npoetry install --only main,pg\n```\n\nRun the application with:\n\n```sh\npoetry run python app.py\n```\n\nCheck that there is now a web server listening on port `6500`:\n\n```sh\ncurl http://localhost:6500/\n```\n\n### The Admin Interface\n\n#### Access\n\nBy default, the application is configured to provide a built-in version of the [admin web interface](https://github.com/NatLibFi/ekirjasto-circulation-admin).\nThe admin interface can be accessed by visiting the `/admin` endpoint:\n\n```sh\n# On Linux\nxdg-open http://localhost:6500/admin/\n\n# On MacOS\nopen http://localhost:6500/admin/\n```\n\nIf no existing users are configured (which will be the case if this is a fresh instance of the application), the\nadmin interface will prompt you to specify an email address and password that will be used for subsequent logins.\nExtra users can be configured later.\n\n#### Creating A Library\n\nNavigate to `System Configuration → Libraries` and click _Create new library_. You will be prompted to enter various\ndetails such as the name of the library, a URL, and more. For example, the configuration for a hypothetical\nlibrary, _Hazelnut Peak_, might look like this:\n\n![.github/readme/library.png](.github/readme/library.png)\n\nNote that the _Patron support email address_ will appear in OPDS feeds served by the application, so make sure\nthat it is an email address you are happy to make public.\n\nAt this point, the _library_ exists but does not contain any _collections_ and therefore won't be of much use to anyone.\n\n#### Adding Collections\n\nNavigate to `System Configuration → Collections` and click _Create new collection_. You will prompted to enter\ndetails that will be used to source the data for the collection. A good starting point, for testing purposes,\nis to use an open access OPDS feed as a data source. The\n[Open Bookshelf](https://palace-bookshelf-opds2.dp.la/v1/publications) is a good example of such a feed. Enter the\nfollowing details:\n\n![.github/readme/collection.png](.github/readme/collection.png)\n\nNote that we associate the collection with the _Hazelnut Peak_ library by selecting it in the `Libraries` drop-down.\nA collection can be associated with any number of libraries.\n\n##### Importing\n\nAt this point, we have a library named _Hazelnut Peak_ configured to use the _Palace Bookshelf_ collection we created.\nIt's now necessary to tell the application to start importing books from the OPDS feed. When the application is\nrunning inside a Docker image, the image is typically configured to execute various import operations on a regular\nschedule using `cron`. Because we're running the application from the command-line for development purposes, we\nneed to execute these operations ourselves manually. In this particular case, we need to execute the `opds_import_monitor`:\n\n```sh\n(circ) $ ./bin/opds_import_monitor\n{\"host\": \"hazelnut\",\n \"app\": \"simplified\",\n \"name\": \"OPDS Import Monitor\",\n \"level\": \"INFO\",\n \"filename\": \"opds_import.py\",\n \"message\": \"[Palace Bookshelf] Following next link: http://openbookshelf.dp.la/lists/Open%20Bookshelf/crawlable\",\n \"timestamp\": \"2022-01-17T11:52:35.839978+00:00\"}\n...\n```\n\nThe command will cause the application to crawl the configured OPDS feed and import every book in it. At the time\nof writing, this command will take around an hour to run the first time it is executed, but subsequent executions\ncomplete in seconds. Please wait for the import to complete before continuing!\n\nWhen the import has completed, the books are imported but no OPDS feeds will have been generated, and no search\nservice has been configured.\n\n#### Configuring Search\n\nNavigate to `System Configuration → Search` and add a new search configuration. The required URL is\nthe URL of the [OpenSearch instance configured earlier](#opensearch):\n\n![OpenSearch](.github/readme/search.png)\n\n#### Generating Search Indices\n\nAs with the collection [configured earlier](#adding-collections), the application depends upon various operations\nbeing executed on a regular schedule to generate search indices. Because we're running the application from\nthe local command-line, we need to execute those operations manually:\n\n```sh\n./bin/search_index_clear\n./bin/search_index_refresh\n```\n\nNeither of the commands will produce any output if the operations succeed.\n\n#### Generating OPDS Feeds\n\nWhen the collection has finished [importing](#importing), we are required to generate OPDS feeds. Again,\nthis operation is configured to execute on a regular schedule in the Docker image, but we'll need to execute\nit manually here:\n\n```sh\n./bin/opds_entry_coverage\n```\n\nThe command will produce output indicating any errors.\n\nNavigating to `http://localhost:6500/` should show an OPDS feed containing various books:\n\n![Feed](.github/readme/feed.png)\n\n#### Troubleshooting\n\nThe `./bin/repair/where_are_my_books` command can produce output that may indicate why books are not appearing\nin OPDS feeds. A working, correctly configured installation, at the time of writing, produces output such as this:\n\n```sh\n(circ) $ ./bin/repair/where_are_my_books\nChecking library Hazelnut Peak\n Associated with collection Palace Bookshelf.\n Associated with 171 lanes.\n\n0 feeds in cachedfeeds table, not counting grouped feeds.\n\nExamining collection \"Palace Bookshelf\"\n 7838 presentation-ready works.\n 0 works not presentation-ready.\n 7824 works in the search index, expected around 7838.\n```\n\nWe can see from the above output that the vast majority of the books in the _Open Bookshelf_ collection\nwere indexed correctly.\n\n### Sitewide Settings\n\nSome settings have been provided in the admin UI that configure or toggle various functions of the Circulation Manager.\nThese can be found at `/admin/web/config/SitewideSettings` in the admin interface.\n\n#### Push Notification Status\n\nThis setting is a toggle that may be used to turn on or off the ability for the the system\nto send the Loan and Hold reminders to the mobile applications.\n\n### Installation Issues\n\nWhen running the `poetry install ...` command, you may run into installation issues. On newer macos machines, you may\nencounter an error such as:\n\n```sh\nerror: command '/usr/bin/clang' failed with exit code 1\n  ----------------------------------------\n  ERROR: Failed building wheel for xmlsec\nFailed to build xmlsec\nERROR: Could not build wheels for xmlsec which use PEP 517 and cannot be installed directly\n```\n\nThis typically happens after installing packages through brew and then running the `pip install` command.\n\nThis [blog post](https://mbbroberg.fun/clang-error-in-pip/) explains and shows a fix for this issue. Start by trying\nthe `xcode-select --install` command. If it does not work, you can try adding the following to your `~/.zshrc` or\n`~/.bashrc` file, depending on what you use:\n\n```sh\nexport CPPFLAGS=\"-DXMLSEC_NO_XKMS=1\"\n```\n\n## Scheduled Jobs\n\nAll jobs are scheduled via `cron`, as specified in the `docker/services/simplified_crontab` file.\nThis includes all the import and reaper jobs, as well as other necessary background tasks, such as maintaining\nthe search index and feed caches.\n\n### Job Requirements\n\n#### hold_notifications\n\nRequires one of [the Firebase Cloud Messaging credentials environment variables (described above)](#firebase-cloud-messaging)\nto be present and non-empty.\nIn addition, the site-wide `PUSH_NOTIFICATIONS_STATUS` setting must be either `unset` or `true`.\n\n#### loan_notifications\n\nRequires one of [the Firebase Cloud Messaging credentials environment variables (described above](#firebase-cloud-messaging)\nto be present and non-empty.\nIn addition, the site-wide `PUSH_NOTIFICATIONS_STATUS` setting must be either `unset` or `true`.\n\n## Code Style\n\nCode style on this project is linted using [pre-commit](https://pre-commit.com/). This python application is included\nin our `pyproject.toml` file, so if you have the applications requirements installed it should be available. pre-commit\nis run automatically on each push and PR by our [CI System](#continuous-integration).\n\nYou can run it manually on all files with the command: `pre-commit run --all-files`.\n\nYou can also set it up, so that it runs automatically for you on each commit. Running the command `pre-commit install`\nwill install the pre-commit script in your local repositories git hooks folder, so that pre-commit is run automatically\non each commit.\n\n### Configuration\n\nThe pre-commit configuration file is named [`.pre-commit-config.yaml`](.pre-commit-config.yaml). This file configures\nthe different lints that pre-commit runs.\n\n### Linters\n\n#### Built in\n\nPre-commit ships with a [number of lints](https://pre-commit.com/hooks.html) out of the box, we are configured to use:\n\n- `trailing-whitespace` - trims trailing whitespace.\n- `end-of-file-fixer` - ensures that a file is either empty, or ends with one newline.\n- `check-yaml` - checks yaml files for parseable syntax.\n- `check-json` - checks json files for parseable syntax.\n- `check-ast` - simply checks whether the files parse as valid python.\n- `check-shebang-scripts-are-executable` - ensures that (non-binary) files with a shebang are executable.\n- `check-executables-have-shebangs` - ensures that (non-binary) executables have a shebang.\n- `check-merge-conflict` - checks for files that contain merge conflict strings.\n- `check-added-large-files` - prevents giant files from being committed.\n- `mixed-line-ending` - replaces or checks mixed line ending.\n\n#### Black\n\nWe lint using the [black](https://black.readthedocs.io/en/stable/) code formatter, so that all of our code is formatted\nconsistently.\n\n#### isort\n\nWe lint to make sure our imports are sorted and correctly formatted using [isort](https://pycqa.github.io/isort/). Our\nisort configuration is stored in our [tox.ini](tox.ini) which isort automatically detects.\n\n#### autoflake\n\nWe lint using [autoflake](https://pypi.org/project/autoflake/) to flag and remove any unused import statement. If an\nunused import is needed for some reason it can be ignored with a `#noqa` comment in the code.\n\n## Internationalization (i18n, l10n, flask-pybabel, managing translations)\n\nWhen adding new translations (with `gettext()` or alias `_()`), make sure to\nupdate the translation files with:\n\n```bash\nbin/util/collect_translations\n```\n\nHere's what the script does:\n\n1) Collects and generates translations from source files with a custom script.\n2) Creates new translation templates (`*.pot`) with `pybabel extract`\n3) Updates existing translation files (`*.po`) with `pybabel update`\n\n## Continuous Integration\n\nThis project runs all the unit tests through Github Actions for new pull requests and when merging into the default\n`main` branch. The relevant file can be found in `.github/workflows/test-build.yml`. When contributing updates or\nfixes, it's required for the test Github Action to pass for all Python 3 environments. Run the `tox` command locally\nbefore pushing changes to make sure you find any failing tests before committing them.\n\nFor each push to a branch, CI also creates a docker image for the code in the branch. These images can be used for\ntesting the branch, or deploying hotfixes.\n\nTo install the tools used by CI run:\n\n```sh\npoetry install --only ci\n```\n\n## Testing\n\nThe Github Actions CI service runs the unit tests against Python 3.10, and 3.11 automatically using\n[tox](https://tox.readthedocs.io/en/latest/).\n\nTox has an environment for each python version, the module being tested, and an optional `-docker` factor that will\nautomatically use docker to deploy service containers used for the tests. You can select the environment you would like\nto test with the tox `-e` flag.\n\n### Factors\n\nWhen running tox without an environment specified, it tests `circulation` and `core` using all supported Python versions\nwith service dependencies running in docker containers.\n\n#### Python version\n\n| Factor | Python Version |\n| ------ | -------------- |\n| py310  | Python 3.10    |\n| py311  | Python 3.11    |\n\nAll of these environments are tested by default when running tox. To test one specific environment you can use the `-e`\nflag.\n\nTest Python 3.8\n\n```sh\ntox -e py38\n```\n\nYou need to have the Python versions you are testing against installed on your local system. `tox` searches the system\nfor installed Python versions, but does not install new Python versions. If `tox` doesn't find the Python version its\nlooking for it will give an `InterpreterNotFound` errror.\n\n[Pyenv](#pyenv) is a useful tool to install multiple Python versions, if you need to install\nmissing Python versions in your system for local testing.\n\n#### Module\n\n| Factor | Module     |\n| ------ | ---------- |\n| core   | core tests |\n| api    | api tests  |\n\n#### Docker\n\nIf you install `tox-docker` tox will take care of setting up all the service containers necessary to run the unit tests\nand pass the correct environment variables to configure the tests to use these services. Using `tox-docker` is not\nrequired, but it is the recommended way to run the tests locally, since it runs the tests in the same way they are run\non the Github Actions CI server. `tox-docker` is automatically included when installing the `ci` dependency group.\n\nThe docker functionality is included in a `docker` factor that can be added to the environment. To run an environment\nwith a particular factor you add it to the end of the environment.\n\nTest with Python 3.8 using docker containers for the services.\n\n```sh\ntox -e \"py38-{api,core}-docker\"\n```\n\n### Local services\n\nIf you already have elastic search or postgres running locally, you can run them instead by setting the\nfollowing environment variables:\n\n- `SIMPLIFIED_TEST_DATABASE`\n- `PALACE_TEST_SEARCH_URL`\n\nMake sure the ports and usernames are updated to reflect the local configuration.\n\n```sh\n# Set environment variables\nexport SIMPLIFIED_TEST_DATABASE=\"postgresql://simplified_test:test@localhost:9005/simplified_circulation_test\"\nexport SIMPLIFIED_TEST_OPENSEARCH=\"http://localhost:9200\"\n\n# Run tox\ntox -e \"py38-{api,core}\"\n```\n\n### Override `pytest` arguments\n\nIf you wish to pass additional arguments to `pytest` you can do so through `tox`. Every argument passed after a `--` to\nthe `tox` command line will the passed to `pytest`, overriding the default.\n\nOnly run the `test_google_analytics_provider` tests with Python 3.8 using docker.\n\n```sh\ntox -e \"py38-api-docker\" -- tests/api/test_google_analytics_provider.py\n```\n\n### Coverage Reports\n\nCode coverage is automatically tracked with [`pytest-cov`](https://pypi.org/project/pytest-cov/) when tests are run.\nWhen the tests are run with github actions, the coverage report is automatically uploaded to\n[codecov](https://about.codecov.io/) and the results are added to the relevant pull request.\n\nWhen running locally, the results from each individual run can be collected and combined into an HTML report using\nthe `report` tox environment. This can be run on its own after running the tests, or as part of the tox environment\nselection.\n\n```shell\n# Run core and api tests under Python 3.8, using docker\n# containers for dependencies, and generate code coverage report\ntox -e \"py38-{core,api}-docker,report\"\n```\n\n## Usage with Docker\n\nCheck out the [Docker README](/docker/README.md) in the `/docker` directory for in-depth information on optionally\nrunning and developing the Circulation Manager locally with Docker, or for deploying the Circulation Manager with\nDocker.\n\n## Performance Profiling\n\nThere are three different profilers included to help measure the performance of the application. They can each be\nenabled by setting environment variables while starting the application.\n\n### AWS XRay\n\n#### Environment Variables\n\n- `PALACE_XRAY`: Set to enable X-Ray tracing on the application.\n- `PALACE_XRAY_NAME`: The name of the service shown in x-ray for these traces.\n- `PALACE_XRAY_ANNOTATE_`: Any environment variable starting with this prefix will be added to to the trace as an\n  annotation.\n    - For example setting `PALACE_XRAY_ANNOTATE_KEY=value` will set the annotation `key=value` on all xray traces sent\n    from the application.\n- `PALACE_XRAY_INCLUDE_BARCODE`: If this environment variable is set to `true` then the tracing code will try to include\n  the patrons barcode in the user parameter of the trace, if a barcode is available.\n\nAdditional environment variables are provided by the\n[X-Ray Python SDK](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-python-configuration.html#xray-sdk-python-configuration-envvars).\n\n### cProfile\n\nThis profiler uses the\n[werkzeug `ProfilerMiddleware`](https://werkzeug.palletsprojects.com/en/2.0.x/middleware/profiler/)\nto profile the code. This uses the\n[cProfile](https://docs.python.org/3/library/profile.html#module-cProfile)\nmodule under the hood to do the profiling.\n\n#### Environment Variables\n\n- `PALACE_CPROFILE`: Profiling will the enabled if this variable is set. The saved profile data will be available at\n  path specified in the environment variable.\n- The profile data will have the extension `.prof`.\n- The data can be accessed using the\n  [`pstats.Stats` class](https://docs.python.org/3/library/profile.html#the-stats-class).\n- Example code to print details of the gathered statistics:\n\n  ```python\n  import os\n  from pathlib import Path\n  from pstats import SortKey, Stats\n\n  path = Path(os.environ.get(\"PALACE_CPROFILE\"))\n  for file in path.glob(\"*.prof\"):\n      stats = Stats(str(file))\n      stats.sort_stats(SortKey.CUMULATIVE, SortKey.CALLS)\n      stats.print_stats()\n  ```\n\n### PyInstrument\n\nThis profiler uses [PyInstrument](https://pyinstrument.readthedocs.io/en/latest/) to profile the code.\n\n#### Profiling tests suite\n\nPyInstrument can also be used to profile the test suite. This can be useful to identify slow tests, or to identify\nperformance regressions.\n\nTo profile the core test suite, run the following command:\n\n```sh\npyinstrument -m pytest --no-cov tests/core/\n```\n\nTo profile the API test suite, run the following command:\n\n```sh\npyinstrument -m pytest --no-cov tests/api/\n```\n\n#### Environment Variables\n\n- `PALACE_PYINSTRUMENT`: Profiling will the enabled if this variable is set. The saved profile data will be available at\n  path specified in the environment variable.\n\n    - The profile data will have the extension `.pyisession`.\n    - The data can be accessed with the\n      [`pyinstrument.session.Session` class](https://pyinstrument.readthedocs.io/en/latest/reference.html#pyinstrument.session.Session).\n    - Example code to print details of the gathered statistics:\n\n    ```python\n    import os\n    from pathlib import Path\n\n    from pyinstrument.renderers import HTMLRenderer\n    from pyinstrument.session import Session\n\n    path = Path(os.environ.get(\"PALACE_PYINSTRUMENT\"))\n    for file in path.glob(\"*.pyisession\"):\n        session = Session.load(file)\n        renderer = HTMLRenderer()\n        renderer.open_in_browser(session)\n    ```\n\n### Other Environment Variables\n\n- `SIMPLIFIED_SIRSI_DYNIX_APP_ID`: The Application ID for the SirsiDynix Authentication API (optional)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnatlibfi%2Fekirjasto-circulation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnatlibfi%2Fekirjasto-circulation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnatlibfi%2Fekirjasto-circulation/lists"}