{"id":29687296,"url":"https://github.com/thepalaceproject/circulation-lcp-test","last_synced_at":"2026-04-28T12:06:24.239Z","repository":{"id":103609245,"uuid":"291999292","full_name":"ThePalaceProject/circulation-lcp-test","owner":"ThePalaceProject","description":"Testbed for testing Readium LCP integration in Circulation Manager (part of https://librarysimplified.org/)","archived":false,"fork":false,"pushed_at":"2020-09-10T15:15:27.000Z","size":8876,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-10-04T07:56:30.470Z","etag":null,"topics":["confd","docker","docker-compose","lcp","python"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ThePalaceProject.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-09-01T13:01:59.000Z","updated_at":"2021-07-29T19:57:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"ef96bde4-0bda-4225-9f32-669939f5f62e","html_url":"https://github.com/ThePalaceProject/circulation-lcp-test","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ThePalaceProject/circulation-lcp-test","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThePalaceProject%2Fcirculation-lcp-test","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThePalaceProject%2Fcirculation-lcp-test/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThePalaceProject%2Fcirculation-lcp-test/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThePalaceProject%2Fcirculation-lcp-test/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ThePalaceProject","download_url":"https://codeload.github.com/ThePalaceProject/circulation-lcp-test/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThePalaceProject%2Fcirculation-lcp-test/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32379679,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-28T11:25:28.583Z","status":"ssl_error","status_checked_at":"2026-04-28T11:25:05.435Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["confd","docker","docker-compose","lcp","python"],"created_at":"2025-07-23T04:36:36.287Z","updated_at":"2026-04-28T12:06:24.226Z","avatar_url":"https://github.com/ThePalaceProject.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# circulation-lcp-test\nTestbed for testing Readium LCP integration in [Circulation Manager](https://github.com/NYPL-Simplified/circulation).\n\n## Architecture\nProject consists of the following modules:\n- [elasticsearch](./elasticsearch) is a Dockerized version of Elasticsearch with pre-installed analysis-icu plugin required by Circulation Manager\n- [lcp-collection](./lcp-collection) is an ONIX collection of 3 books used for testing\n- [lcp-docker](./lcp-docker) is a [Docker-based implementation of Readium LCP](https://github.com/amigoslibraryservices/lcp-docker)\n- [lcp-docker-conf](./lcp-docker-conf) is a Dockerized application generating LCP configuration for LCP License and Status servers using confd\n- [lcp-import](./lcp-import) is Docker image based on [nypl/circ-exec](https://hub.docker.com/r/nypl/circ-exec/) including a bash script for importing the LCP collection into Circulation Manager\n- [proxy](./proxy) is a Dockerized nginx reversed proxy based on [docker-nginx-with-confd](https://github.com/sysboss/docker-nginx-with-confd) GitHub project\n\n## Usage\n\n### Preparing the local environment\n1. Update all the submodules:\n```bash\ngit submodule init\ngit submodule update --remote --recursive\n```\n\n2. Update the following host names in [.env](./.env) file:\n- READIUM_LSDSERVER_HOSTNAME\n- READIUM_FRONTEND_HOSTNAME\n- MINIO_HOSTNAME\n- CM_HOSTNAME\n\n3. Replace all the host names with `127.0.0.1` in `etc/hosts` file:\n```\n127.0.0.1     lsdserver.lcp.hilbertteam.net\n127.0.0.1     testfrontend.lcp.hilbertteam.net\n127.0.0.1     minio.lcp.hilbertteam.net\n127.0.0.1     cm.lcp.hilbertteam.net\n```\n\n3. Run `lcp-conf` first to generate configuration required by `lcpserver`, `lsdserver`, and `testfrontend`:\n```bash\ndocker-compose run lcp-conf\n```\n\n4. Build the images:\n```bash\ndocker-compose build\n```\n\n\u003e :warning: Please note that this testbed uses *development* version of `nypl/circ-webapp` and [custom version](https://hub.docker.com/r/viacheslavbessonov/circ-exec) of `circ-exec` Docker images.\n\n6. Run all the containers:\n```bash\ndocker-compose up -d\n```\n\n7. Use `docker-compose ps` to confirm that all the containers started successfully. It may take some time for `mariadb` to start which can negatively affect `lcpserver`, `lsdserver`, and `testfrontend`. In this case wait until `mariadb` finishes the initialization process (you can check the logs using `docker-compose logs mariadb`) and then start all the remaining containers:\n```bash\ndocker-compose up -d\n```\n\n8. Make sure that Elasticsearch started correctly. Sometimes when the disk capacity is low, Elasticsearch marks shards as read-only which doesn't allow to use it properly.\nCheck the logs using `docker-compose logs es` and if you see something suspicious execute the following requests to fix it:\n```bash\ndocker-compose exec es bash\ncurl -XPUT -H \"Content-Type: application/json\" http://localhost:9200/_cluster/settings -d '{ \"transient\": { \"cluster.routing.allocation.disk.threshold_enabled\": false } }'\ncurl -XPUT -H \"Content-Type: application/json\" http://localhost:9200/_all/_settings -d '{\"index.blocks.read_only_allow_delete\": null}'\n```\nAlso, sometimes Elasticsearch [keeps failing because of OOM even though there is enough RAM](https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html). In this case please try to increase `vm.max_map_count`:\n```bash\nsysctl -w vm.max_map_count=262144\n```\n\n### Setting up a MinIO instance\n1. Log into MinIO's administrative interface located at `MINIO_HOSTNAME` using `AWS_S3_KEY` and `AWS_S3_SECRET` defined in [.env](./.env) file as credentials:\n\n2. Create the following buckets as it's shown on the picture below:\n- `covers` - public access bucket containig book covers\n- `encrypted-books` (`READIUM_S3_BUCKET`) - public access bucket containing encrypted books\n  ![Creating buckets in MinIO](docs/01-Creating-buckets-in-MinIO.png \"Creating buckets in MinIO\")\n\n3. Grant public access to the buckets created before:\n```bash\n# Start the MinIO's command line client\ndocker run -it --entrypoint=/bin/sh --network container:circulation-lcp-test_minio_1 minio/mc\n\n# Authenticate against the running MinIO instance\nmc alias set local http://minio:9000 minioadmin minioadmin # Please use the credentials set in .env file\n\n# Grant public access to covers bucket\nmc policy set public local/covers\n\n# Grant public access to covers encrypted-books\nmc policy set public local/encrypted-books\n```\n\n### Setting up Circulation Manager\n1. Open Circulation Manager's administrative interface located at `CM_HOSTNAME`\n\n2. Set up a new administrative account\n\n3. Create a new library using `LCP` as its short name as it's shown on the picture below:\n\u003e _`LCP` is used in LCP configuration and shouldn't be changed_\n\n  ![Creating a new library](docs/02-Creating-a-new-library.png \"Creating a new library\")\n\n4. Set up a search service using `http://es:9200` as Elasticsearch's URL:\n  ![Creating a search service](docs/03-Creating-a-search-service.png \"Creating a search service\")\n\n5. Set up a new MinIO storage as it's shown on the pictures below and using `AWS_S3_KEY` and `AWS_S3_SECRET` as credentials:\n  ![Creating a MinIO storage service](docs/04-Creating-a-MinIO-storage-service.png \"Creating a MinIO storage service\")\n\n  ![Creating a MinIO storage service](docs/05-Creating-a-MinIO-storage-service.png \"Creating a MinIO storage service\")\n\n6. Set up a new LCP storage service:\n  ![Creating an LCP storage service](docs/06-Creating-an-LCP-storage-service.png \"Creating an LCP storage service\")\n\n  ![Creating an LCP storage service](docs/07-Creating-an-LCP-storage-service.png \"Creating an LCP storage service\")\n\n7. Set up a new LCP collection:\n\u003e - *LCP License Server's input directory* has to be `/opt/readium/files` because it's defined in Dockerfile\n\u003e - *lcpencrypt's output directory* has to be a value of `CM_REPOSITORY`, it points to the *intermediate_repository*\n\n  ![Creating an LCP collection](docs/08-Creating-a-new-LCP-collection.png \"Creating an LCP collection\")\n\n  ![Creating an LCP collection](docs/09-Creating-a-new-LCP-collection.png \"Creating an LCP collection\")\n\n8. Set up a patron authentication provider (for example, a basic authentication):\n  ![Creating a basic authentication provider](docs/10-Creating-a-basic-authentication-provider.png \"Creating a basic authentication provider\")\n\n### Import the LCP collection\n1. Run [the import script](./lcp-import/run.sh) via in `nypl/circ-exec` Docker image:\n```bash\ndocker-compose -f docker-compose.yml -f docker-compose.import.yml run import\n```\n\n### Test\n1. Go to the Circulation Manager dashboard, select `A Dictionary in Hindi and English` and borrow it:\n  ![Borrowing an LCP book](docs/11-Borrowing-an-LCP-book.png \"Borrowing an LCP book\")\n\n2. Authenticate using the test patron credentials:\n  ![Authenticating](docs/12-Authenticating-a-patron.png \"Authenticating\")\n\n3. Download the book:\n  ![Downloading an LCP book](docs/13-Downloading-an-LCP-book.png \"Downloading an LCP book\")\n\n4. Find the downloaded file.\n\u003e Please note that this file must have `.lcpl` extension (this should be fixed in [this PR](https://github.com/NYPL-Simplified/opds-web-client/pull/279)). Until it's fixed please change its extension manually\n  ![Accessing the downloaded LCP book](docs/14-Accessing-the-LCP-book.png \"Accessing the downloaded LCP book\")\n\n5. Open the `.lcpl` file in Thorium Reader:\n  ![Opening the LCP book in Thorium Reader](docs/15-Opening-the-LCP-book-in-Thorium-Reader.png \"Opening the LCP book in Thorium Reader\")\n\n6. Get generated passphrase from Circulation Manager:\n  ![Getting a passphrase from Circulation Manager](docs/16-Getting-passphrase-from-Circulation-Manager.png \"Getting a passphrase from Circulation Manager\")\n\n7. Enjoy the book in Thorium Reader:\n  ![LCP book opened in Thorium Reader](docs/17-LCP-book-opened-in-Thorium-Reader.png \"LCP book opened in Thorium Reader\")\n\n\n## Troubleshooting\nIf a list of the books you see in Circulation Manager is outdated or simply wrong you may have to truncate cached feeds in CM's database. To do that please execute the following steps:\n```bash\ndocker exec -it circulation-lcp-test_postgres_1 bash\npsql -U simplified simplified_circulation_dev  # Please use the credentials from .env file\ntruncate cachedfeeds;\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthepalaceproject%2Fcirculation-lcp-test","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthepalaceproject%2Fcirculation-lcp-test","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthepalaceproject%2Fcirculation-lcp-test/lists"}