{"id":39825091,"url":"https://github.com/iiasa/accelerator_service","last_synced_at":"2026-01-18T13:03:49.623Z","repository":{"id":201199382,"uuid":"620856880","full_name":"iiasa/accelerator_service","owner":"iiasa","description":"Docker compose for services and jobs related to scenario explorer and data processing","archived":false,"fork":false,"pushed_at":"2025-11-28T13:37:30.000Z","size":119,"stargazers_count":0,"open_issues_count":40,"forks_count":1,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-11-30T20:18:29.983Z","etag":null,"topics":["accelerator"],"latest_commit_sha":null,"homepage":"","language":"Berry","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iiasa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-03-29T14:03:50.000Z","updated_at":"2025-11-28T13:37:34.000Z","dependencies_parsed_at":"2024-08-08T18:12:07.945Z","dependency_job_id":"96bacb7c-62bd-4ed4-9db8-1bd8fb447e0d","html_url":"https://github.com/iiasa/accelerator_service","commit_stats":null,"previous_names":["iiasa/services_docker_compose_for_dev","iiasa/accelerator_service"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/iiasa/accelerator_service","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iiasa%2Faccelerator_service","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iiasa%2Faccelerator_service/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iiasa%2Faccelerator_service/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iiasa%2Faccelerator_service/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iiasa","download_url":"https://codeload.github.com/iiasa/accelerator_service/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iiasa%2Faccelerator_service/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28536688,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T10:13:46.436Z","status":"ssl_error","status_checked_at":"2026-01-18T10:13:11.045Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accelerator"],"created_at":"2026-01-18T13:03:49.533Z","updated_at":"2026-01-18T13:03:49.612Z","avatar_url":"https://github.com/iiasa.png","language":"Berry","funding_links":[],"categories":[],"sub_categories":[],"readme":"# How to configure and start Accelerator services for local development\n\n## Prerequisites\n\n1. A Docker runtime on your development machine.\n2. A Kubernetes cluster, either on your development machine or remotely.\n    * **Warning:** Docker Desktop on Windows has issues with building and Wine.\n    * On Linux, success has been had with Rancher desktop.\n\n## Services\n\nTo set up the services, copy the default Docker compose `.env*` files from the `sample_dotenv/` folder into the working\ndirectory root (containing this `README.md`). Beware: these files have a leading dot and are normally hidden. Customize\nthe copied `.env*` files as described below, taking into account that default values enclosed by angular brackets (\n`\u003csomething\u003e`) must be overridden.\n\nThe current working directory when issuing `docker compose` commands must be the `accelerator_service` working directory\nroot. For development work, at a minimum run the frontend, backend, scheduler, TiTiler, and MinIO.\n\n### `.env`\n\nSets various paths pointing to Accelerator subsystems accessible on your development machine. The `*PROJECT_FOLDER` and\n`TITILER_FOLDER` settings should point to the working directories of Accelerator-related Git repositories. The default\nvalues indicate the name of the repository. For example,\n\n```\nACCMS_PROJECT_FOLDER='\u003cpath to\u003e/accms'\n```\n\nindicates that you have to locate the `accms` repository by [searching for repositories marked with the\n`accelerator` topic under the IIASA Github organization](https://github.com/search?q=org%3Aiiasa%20topic%3Aaccelerator\u0026type=repositories),\nclone it somewhere convenient, and point `ACCMS_PROJECT_FOLDER` at the resulting working directory.\n\n\u003e [!CAUTION]\n\u003e Relative paths (relative to the `accelerator_service` working directory root) should start with `./` to avoid being\n\u003e mistaken for a volume name.\n\n### `.env.web.be` (backend)\n\nAside from the self-explanatory settings...\n\nInsert the IPv4 address of your dev machine in the value of `INITIAL_S3_ENDPOINT` before the port number.\n\nSet `JOB_SECRET_ENCRYPTION_KEY` to the base64-encoded representation of random 256-bit key values which you can obtain\nas follows:\n\n```\nhead \u003c/dev/random -c32 | base64\n```\n\nThis key encrypts secrets required by jobs.\n\nSet `BUCKET_DETAILS_ENCRYPTION_KEY` to the base64-encoded representation of random 256-bit key values which you can\nobtain as follows:\n\n```\nhead \u003c/dev/random -c32 | base64\n```\n\nThis key encrypts bucket credentials.\n\nNeed a public/private keypair. Tokens are signed with the private key by the backend, and can be verified with the\npublic key. This is useful for example for the gateway for interactive containers: the gateway simply verifies the token\nvia the public key as obtained via the `GET` method at\n`https://accelerator-api.iiasa.ac.at/docs#/.well-known/jwks.json`.\n\nUse OpenSSL to generate the keypair and extract the public key:\n\n```\nopenssl ecparam -genkey -name prime256v1 -noout -out private_key.pem\nopenssl ec -in private_key.pem -pubout -out public_key.pem\n```\n\nSet the following:\n\n```\nJWT_BASE64_PRIVATE_KEY=\"$(base64 -w0 private_key.pem)\"\nJWT_BASE64_PUBLIC_KEY=\"$(base64 -w0 public_key.pem)\"\n```\n\n### `.env.web.fe` (frontend)\n\nMust use https with TiTiler and hence set `https://...` in `VITE_TITILER_API_BASE_URL`. Therefore need to generate\nself-signed certificate for TiTiler. Configuration details pending. Query an LLM on how to obtain a self-signed\ncertificate that also works for `localhost`.\n\n### `.env.scheduler` (job dispatcher)\n\nIn `.env.scheduler`, aside from the obvious settings:\n\n1. In `.env.scheduler` configure `IMAGE_REGISTRY_*`. `IMAGE_REGISTRY_TAG_PREFIX` is needed when the registry is\n   subdivided in namespaces. For example Harbor uses projects. If so, set the name of your space/project followed by a\n   slash as value.\n    - When the registry service is running, you should be able to login via  `docker login \u003cregistry\u003e:8443` and the\n      configured username and password.\n2. Set `JOBSTORE_*` values to point to an S3 bucket for transient file storage when launching WKube jobs.\n3. Convert `~/.kube/config` to JSON and then a base64 string:\n   ```\n   kubectl config view --output json --raw \u003ekubeconfig.json\n   ```\n   Edit the JSON to remove irrelevant contexts / credentials.\n   ```\n   base64 -w0 kubeconfig.json \u003ekubeconfig.b64\n   ```\n4. Set  `WKUBE_SECRET_JSON_B64` to the content of `kubeconfig.b64`.\n    - Or use command\n      `python3 -c \"import sys, yaml, json; print(json.dumps(yaml.safe_load(sys.stdin), indent=2))\" \u003c ~/.kube/config \u003e config.json`\n      to convert the kubernetes config to JSON.\n5. Set `ACCELERATOR_APP_TOKEN` by obtaining a token as follows:\n    - Startup the backend service:  \n      `docker compose up web_be`\n        - This also starts the integrated frontend.\n    - Create an account for yourself by signing in to the front end at `https:\\\\localhost:8080`.\n        - Press the \"Login with IIASA\" button.\n    - With `docker ps`, determine the container ID of the backend and shell into the container:  \n      `docker ps | grep web_be`  \n      `docker exec -it \u003cbackend container ID\u003e /bin/bash`\n    - Grant your account superuser rights:  \n      `python apply.py add_role \u003cyour email\u003e APP__SUPERUSER`\n    - Obtain an access token with superuser rights:  \n      `python apply.py get_access_token \u003cyour email\u003e \u003cseconds to expiry\u003e`\n    - Copy and paste the token as the value of `ACCELERATOR_APP_TOKEN`.\n\n### [TiTiler](https://developmentseed.org/titiler/) (tile server)\n\n1. Clone the repo `https://github.com/iiasa/meta-titiler`\n2. Point `TITILER_FOLDER` in `.env` at the resulting working directory.\n3. Check that a certificate is present in `certs`. If absent, create a self-signed certificate expiring in `$DAYS` for\n   TiTiler by issuing:\n   ```\n   cd meta-titiler\n   mkdir certs\n   cd certs\n   openssl req -x509 -newkey rsa:2048 -keyout private.key -out public.crt -days $DAYS -nodes -subj \"/CN=localip\"\n   cp public.crt ../dockerfiles/minio-cert.crt\n   cd ..\n   ```\n\n### MinIO (block storage, S3)\n\n1. Create a self-signed certificate:\n   ```\n   cd minio_certs\n   openssl req -x509 -newkey rsa:2048 -keyout private.key -out public.crt -days $DAYS -nodes -subj \"/CN=localhost\"\n   cd ..\n   ```\n   where `$DAYS` is the number of days before the certificate expires. Use a big number to effectively have no expiry.\n2. Start the service:\n   ```\n   docker compose -f docker-compose.dev.yml up minio [--build]\n   ```\n   Optionally use the `--build` flag to allow image to be rebuilt if needed. This ensures the containers pick up code\n   changes, updated dependencies, and any modifications to the build specification (`Dockerfile`).\n3. Access MinIO container via `docker exec` or GUI.\n4. Register the local instance.\n   ```\n   mc alias set local https://localhost:9000 admin adminpassword --insecure\n   ```\n5. Create required buckets.\n   ```\n   mc mb local/accelerator --insecure\n   mc mb local/jobstore --insecure\n   ```\n6. Generate access key and secret key.\n   ```\n   mc admin accesskey create local/ --insecure\n   ```\n7. In `.env.web.be` and `.env.scheduler`, set these as values of the `*_S3_API_KEY=` and `*_S3_SECRET_KEY=` entries.\n8. Add the MinIO endpoint to your system's known hosts.\n\n### Registry\n\nGenerate `htpasswd` file:\n\n1. `docker pull httpd:2`\n2. `docker run --rm --entrypoint htpasswd httpd:2 -Bbn myregistry myregistrypassword \u003e registry_auth/htpasswd`\n\n### Database\n\n1. Execute `docker compose -f docker-compose.dev.yml up db [--build]` to start the service and optionally (re)build the\n   image.\n2. Enter the db container with `docker exec -it \u003cdb container ID\u003e /bin/bash`\n3. Create databases inside the container with:\n    - `su -- postgres -c \"createdb accelerator\"`\n    - `su -- postgres -c \"createdb acceleratortest\"`\n    - `su -- postgres -c \"createdb accms\"`\n    - `su -- postgres -c \"createdb thrd\"`\n4. When a database already exists, you may wish to drop it first to start with a clean slate:\n    - `su -- postgres -c \"dropdb accelerator\"`\n    - `su -- postgres -c \"dropdb acceleratortest\"`\n    - `su -- postgres -c \"dropdb accms\"`\n    - `su -- postgres -c \"dropdb thrd\"`\n\n## Further configuration\n\nCreate local IP entries in your `/etc/hosts`\n\n```\n# Accelerator\nxxx.xxx.xxx.xxx localip registry web_be\n```\n\nwhere `xxx.xxx.xxx.xxx` is your IP address on the IIASA network.\n\n\u003e [!NOTE]\n\u003e When changing the network environment, for example by taking a dev laptop home, you will need to change this.\n\n## Startup the project\n\n`docker compose -f docker-compose.dev.yml up [--build]`\n\n## Browse\n\nBrowse to the backend at `https://localhost:8000`. In case of a security warning on account of the self-signed\ncertificate, add an exception in your browser.\n\nBrowse to the frontend at `https://localhost:8080`. In case of a security warning on account of the self-signed\ncertificate, add an exception in your browser. Then log in via the `Login with IIASA` button.\n\n## Additional notes\n\n- Inside `control_services_backend` ignore the `.env.sample`, the configs are passed down as the containers are\n  orchestrated.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiiasa%2Faccelerator_service","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiiasa%2Faccelerator_service","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiiasa%2Faccelerator_service/lists"}