{"id":26321818,"url":"https://github.com/terradue/calrissian-session","last_synced_at":"2026-01-01T22:05:46.189Z","repository":{"id":142078524,"uuid":"460018803","full_name":"Terradue/calrissian-session","owner":"Terradue","description":"From zero to CWL on K8s with Calrissian","archived":false,"fork":false,"pushed_at":"2023-11-08T06:08:33.000Z","size":55,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-04-16T11:14:30.713Z","etag":null,"topics":["cwl","helm-chart","kubernetes","visual-studio-code"],"latest_commit_sha":null,"homepage":"","language":"Smarty","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Terradue.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-02-16T13:36:23.000Z","updated_at":"2023-01-19T09:44:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"7d4ad0bb-a2a6-44c2-810b-6d11269455f7","html_url":"https://github.com/Terradue/calrissian-session","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Terradue%2Fcalrissian-session","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Terradue%2Fcalrissian-session/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Terradue%2Fcalrissian-session/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Terradue%2Fcalrissian-session/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Terradue","download_url":"https://codeload.github.com/Terradue/calrissian-session/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243753976,"owners_count":20342543,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cwl","helm-chart","kubernetes","visual-studio-code"],"created_at":"2025-03-15T16:18:34.686Z","updated_at":"2026-01-01T22:05:46.116Z","avatar_url":"https://github.com/Terradue.png","language":"Smarty","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Calrissian session: a Helm template to create a calrissian kubernetes session\n\nCalrissian is a CWL implementation designed to run inside a Kubernetes cluster. Its goal is to be highly efficient and scalable, taking advantage of high capacity clusters to run many steps in parallel.  \n\nCalrissian requires a Kubernetes cluster, configured to provision PersistentVolumes with the `ReadWriteMany` access mode. \n\nThis Helm chart deploys and configures:\n\n- a ReadWriteMany `PersistentVolumeClaim`\n- `configMap`s setting:\n    - the access to an S3 bucket \n    - the access to container registries\n- `Role`s to create pods and access pod logs\n- `RoleBinding`s to associate the roles to the namespace default service account\n- a `Deployment` with a pod that includes `calrissian` and typical development tools (the pod can be attached to a Visual Studio Code session)\n- an optional `ServiceAccount` \n- a `Secret` to pull containers from container registries\n\n## Values\n\n| Key | Type | Default | Description |\n|-----|------|---------|-------------|\n| affinity | object | `{}` | node affinity |\n| image | object | `{\"pullPolicy\":\"Always\",\"repository\":\"ghcr.io/terradue/calrissian-session/calrissian-session\",\"tag\":\"latest\"}` | Calrissian container image for running the Calrissian pod |\n| imageCredentials | list | `[{\"auth\":\"bXNhZ2....RVURt\",\"registry\":\"ghcr.io\"},{\"auth\":\"ZmFi...mlRTldqZw==\",\"https://index.docker.io/v1/\":null}]` | container registries credentials |\n| imageCredentials[0] | object | `{\"auth\":\"bXNhZ2....RVURt\",\"registry\":\"ghcr.io\"}` | registry is the container registry |\n| imageCredentials[0].auth | string | `\"bXNhZ2....RVURt\"` | auth is the base64 auth string (see your ~/.docker/config.json file) |\n| nodeSelector | object | `{\"k8s.scaleway.com/pool-name\":\"processing-node-pool-iride-xl\"}` | specify the node selector for the Calrissian pod and the Calrissian worker pods |\n| podAnnotations | object | `{}` | optional pod annotations |\n| podSecurityContext | object | `{}` | additional settings for the pod security context |\n| replicaCount | int | `1` | number of pods, one is usually enough |\n| resources | object | `{\"limits\":{\"cpu\":\"4\",\"memory\":\"12Gi\"},\"requests\":{\"cpu\":\"4\",\"memory\":\"8Gi\"}}` | specify the resources for the Calrissian pod |\n| s3 | object | `{\"access_key_id\":\"SC...8Z\",\"bucket_pattern\":\"s3:\\\\/\\\\/ir....tplace\\\\/.*\",\"enabled\":true,\"endpoint_url\":\"https://s3.....cloud\",\"region\":\"...\",\"secret_access_key\":\"bf...dc6\",\"signature_version\":\"s3v4\"}` | use s3, if true, configMaps are mounted to access the S3 bucket |\n| securityContext | object | `{\"privileged\":true}` | running with privileged set to true allows running podman in the Calrissian pod |\n| serviceAccount | object | `{\"annotations\":{},\"create\":true,\"name\":\"calrissian-sa\"}` | Service account to use |\n| serviceAccount.annotations | object | `{}` | Annotations to add to the service account |\n| serviceAccount.create | bool | `true` | Specifies whether a service account should be created |\n| serviceAccount.name | string | `\"calrissian-sa\"` | The name of the service account to use. |\n| storageClass | string | `\"openebs-kernel-nfs-scw\"` | ReadWriteMany storage class for Calrissian worker  |\n| tolerations | list | `[]` | tolerations |\n| volumeSize | string | `\"10Gi\"` | size of the ReadWriteMany volume for Calrissian executions |\n\n## Requirements\n\n- a kubeconfig file to access a kubernetes cluster\n- an environment variable named `KUBECONFIG` exporting that config file\n- [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) and [helm](https://helm.sh/docs/intro/install/) installed\n\n### Setting the KUBECONFIG environment variable\n\nExample: \n\n```console\nexport KUBECONFIG=~/Downloads/kubeconfig.yaml\n```\n\n### A kubernetes namespace for the Calrissian session\n\nExport an environment variable exporting the namespace for the Calrissian session, e.g.:\n\n```console\nexport NAMESPACE_NAME=calrissian-session\n```\n\nTip: You can create a namespace with:\n\n```\nkubectl create namespace \"$NAMESPACE_NAME\"\n```\n\n## Install or upgrade a calrissian session\n\nAdd the helm repo with:\n\n```\nhelm repo add calrissian-session https://terradue.github.io/calrissian-session\nhelm repo update\n```\n\nDeploy the calrissian session with:\n\n```\nhelm upgrade --install my-calrissian-session calrissian-session/calrissian-session --namespace \"$NAMESPACE_NAME\" --create-namespace \\\n    --set imageCredentials[0].registry=docker.terradue.com \\\n    --set imageCredentials[0].auth=\"aaa....bbbb\" \\\n    --set s3.enabled=true \\\n    --set s3.access_key_id=\"cc..dd\" \\\n    --set s3.secret_access_key=\"ee...ff\" \\\n    --set s3.region=a_region \\\n    --set s3.endpoint_url=https://s3.somedomain.com \\\n    --set s3.signature_version=s3v4\n```\n\nOr create a `values.yaml` file and then:\n\n```console\nhelm upgrade --install calrissian-session calrissian-session --namespace \"$NAMESPACE_NAME\" --values calrissian-session/values.yaml\n```\n\n## Access the calrissian session pod in Visual Studio Code\n\n`~/.kube/config` must be a symbolic link to the `$KUBECONFIG` file to use the pod with a Visual Studio Code attached to:\n\n```\nrm -f ~/.kube/config\nln -s ~/Downloads/kubeconfig.yaml ~/.kube/config\n```\n\n**Visual Studio requirements** \n\n- [Kubernetes extension installed](https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools)\n- [Remote Development extension installed](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack)\n\n**Attach the pod to a VS Code session**\n\nProcedure:\n\n1. Connect the kubernetes cluster to VS Code:\n    - Open the Command Palette and use `Kubernetes: Set Kubeconfig`\n    - Select `Add new Kubeconfig` to select a new kubeconfig file **OR** select a kubeconfig out of the list \n2. Click on the `Kubernetes` icon \n3. Navigate to the namespace `$NAMESPACE` (defined above) and right-click it and select \"Use Namespace\" (you might have to refresh the list of namespaces)\n4. Expand \"Workloads\" and \"Pods\"\n5. Right-click the pod named \"calrissian-session-nnnn\" where nnnn is an uid created by kubernetes and select \"Attach Visual Studio Code\". This will open a new VS Code window\n6. Monitor the progress on the new Visual Studio Code window\n\nOnce ready, proceed with the next step.\n\n**Clone Repository** \n\nGo to the \"Explorer\" and select \"Clone Repository\"\n\n1. Clone the repo https://gitlab.com/app-packages/terradue/dnbr-sentinel-2-cog.git \n2. Select /home as the target repo\n3. Open the cloned repository folder (bottom right question)\n\n**Run a CWL Workflow** \n\n1. Open a new terminal\n2. Run the application with: \n\n```console\ncalrissian --stdout /calrissian/results.json \\\n           --stderr  /calrissian/app.log \\\n           --max-ram 16G \\\n           --max-cores \"8\" \\\n           --tmp-outdir-prefix /calrissian/tmp \\\n           --outdir /calrissian/output \\\n           --usage-report /calrissian/usage.json \\\n           app-package.cwl#dnbr \\\n           params.yaml\n```\n\nThe expected output is:\n\n```\nINFO calrissian 0.10.0 (cwltool 3.1.20211004060744)\nINFO Resolved 'app-package.cwl#dnbr' to 'file:///home/app-package-04/dnbr-sentinel-2-cog/app-package.cwl#dnbr'\nINFO [workflow ] starting step node_nbr\nINFO [step node_nbr] start\nINFO [workflow node_nbr] starting step node_stac_2\nINFO [step node_stac_2] start\nINFO [step node_stac_2] start\nINFO [step node_stac_2] start\nINFO [step node_nbr] start\nINFO [workflow node_nbr_2] starting step node_stac_3\nINFO [step node_stac_3] start\nINFO [step node_stac_3] start\nINFO [step node_stac_3] start\nINFO [workflow ] start\nINFO [workflow node_nbr] start\nINFO [workflow node_nbr_2] start\nINFO [step node_stac_3] completed success\nINFO [workflow node_nbr_2] starting step node_subset_2\nINFO [step node_subset_2] start\nINFO [step node_subset_2] start\nINFO [step node_stac_2] completed success\nINFO [step node_subset_2] start\nINFO [workflow node_nbr] starting step node_subset\nINFO [step node_subset] start\nINFO [step node_subset] start\nINFO [step node_subset] start\nINFO [step node_subset] completed success\nINFO [workflow node_nbr] starting step node_nbr_2\nINFO [step node_nbr_2] start\nINFO [step node_subset_2] completed success\nINFO [workflow node_nbr_2] starting step node_nbr_3\nINFO [step node_nbr_3] start\nINFO [step node_nbr_2] completed success\nINFO [workflow node_nbr] starting step node_cog_2\nINFO [step node_cog_2] start\nINFO [step node_nbr_3] completed success\nINFO [workflow node_nbr_2] starting step node_cog_3\nINFO [step node_cog_3] start\nINFO [step node_cog_2] completed success\nINFO [workflow node_nbr] completed success\nINFO [step node_cog_3] completed success\nINFO [workflow node_nbr_2] completed success\nINFO [step node_nbr] completed success\nINFO [workflow ] starting step node_dnbr\nINFO [step node_dnbr] start\nINFO [step node_dnbr] completed success\nINFO [workflow ] starting step node_cog\nINFO [step node_cog] start\nINFO [step node_cog] completed success\nINFO [workflow ] starting step node_stac\nINFO [step node_stac] start\nINFO [step node_stac] completed success\nINFO [workflow ] completed success\n{\n    \"stac\": {\n        \"location\": \"file:///calrissian-output/r30kfi8f\",\n        \"basename\": \"r30kfi8f\",\n        \"class\": \"Directory\",\n        \"listing\": [\n            {\n                \"class\": \"File\",\n                \"location\": \"file:///calrissian-output/r30kfi8f/catalog.json\",\n                \"basename\": \"catalog.json\",\n                \"checksum\": \"sha1$a5d1d9821e889aa125778e4f2e14a788ff1512ce\",\n                \"size\": 225,\n                \"path\": \"/calrissian-output/r30kfi8f/catalog.json\"\n            },\n            {\n                \"class\": \"File\",\n                \"location\": \"file:///calrissian-output/r30kfi8f/dnbr-item.json\",\n                \"basename\": \"dnbr-item.json\",\n                \"checksum\": \"sha1$1c0a635ad501c599ab258019d05c7b276515c565\",\n                \"size\": 818,\n                \"path\": \"/calrissian-output/r30kfi8f/dnbr-item.json\"\n            },\n            {\n                \"class\": \"File\",\n                \"location\": \"file:///calrissian-output/r30kfi8f/dnbr.tif\",\n                \"basename\": \"dnbr.tif\",\n                \"checksum\": \"sha1$df335a81cc67198654951d0a0fb98c5b89db67fe\",\n                \"size\": 1407480,\n                \"path\": \"/calrissian-output/r30kfi8f/dnbr.tif\"\n            }\n        ],\n        \"path\": \"/calrissian-output/r30kfi8f\"\n    }\n}\nINFO Final process status is success\n```\n\n## Clean-up\n\n### Uninstall a calrissian session\n\n```console\nexport NAMESPACE_NAME=calrissian-session\nhelm uninstall my-calrissian-session --namespace \"$NAMESPACE_NAME\" \n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fterradue%2Fcalrissian-session","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fterradue%2Fcalrissian-session","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fterradue%2Fcalrissian-session/lists"}