{"id":24562597,"url":"https://github.com/humanbrainproject/img-svc-tasks","last_synced_at":"2025-09-06T05:41:55.688Z","repository":{"id":74588889,"uuid":"239733567","full_name":"HumanBrainProject/img-svc-tasks","owner":"HumanBrainProject","description":null,"archived":false,"fork":false,"pushed_at":"2020-05-18T12:21:47.000Z","size":42,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-08-25T09:27:29.608Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HumanBrainProject.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-02-11T10:21:15.000Z","updated_at":"2020-05-18T12:21:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"986177fd-cefc-404b-9b51-aeda71970e75","html_url":"https://github.com/HumanBrainProject/img-svc-tasks","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/HumanBrainProject/img-svc-tasks","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HumanBrainProject%2Fimg-svc-tasks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HumanBrainProject%2Fimg-svc-tasks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HumanBrainProject%2Fimg-svc-tasks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HumanBrainProject%2Fimg-svc-tasks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HumanBrainProject","download_url":"https://codeload.github.com/HumanBrainProject/img-svc-tasks/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HumanBrainProject%2Fimg-svc-tasks/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273861339,"owners_count":25181322,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-06T02:00:13.247Z","response_time":2576,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-23T09:16:32.823Z","updated_at":"2025-09-06T05:41:55.668Z","avatar_url":"https://github.com/HumanBrainProject.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tasks that can be invoked by the Image Service on the HPC workers\n\n# Scripts in the package\n\n## `ingest`\n\nCombines arguments from the other sub-modules and does all steps in one\nCommand Line Interface.\n\nIt has the advantage of executing a complete process with a single Unicore job\ndefinition.\n\nExample\n\n```\nSWIFT_SETTINGS='../.os_settings' ingest \\\n  --source https://object.cscs.ch/v1/AUTH_61499a61052f419abad475045aaf88f9/img-svc-bigbrain-input \\\n  --download-dir $SOURCE \\\n  --results-dir $RESULTS \\\n  --stacks \\\n  --filter 'data/raw/original/pm31\\d{2}o.png' \\\n  --parameters \"$PARAMS\" \\\n  --container test_wrapper2 \\\n  --cleanup\n```\n\n\n## `fetch_input`\n\n  Fetches the input data for processing.\n  * Can be a single URL which does not require authentication.\n  * In case it's a stack of images, a CSCS Swift container and an option regex\n  filter for objects in this container is also accepted.\n\n  Examples:\n\n  * Stacks\n\n  ```\n  fetch_input \\\n    --stacks \\\n    --filter 'Pitx3-tTA-horizontal-case6513/6513_PitX3_tiffs/6513_Pitx3_tTA_lacZ_Xgal_s01\\d_0.22.tif' \\\n    --destination ./source \\\n    https://object.cscs.ch/v1/AUTH_6ebec77683fb472f94d352be92b5a577/tTA-atlas\n  ```\n\n  * Single files\n\n  ```\n  fetch_input \\\n    --destination ./source \\\n    https://github.com/HumanBrainProject/neuroglancer-scripts/raw/master/examples/JuBrain/colin27T1_seg.nii.gz\n  ```\n\n\n## `ingest_ngs`\n\nChunk up the input data using the neuroglancer_scripts\n\n## `upload_results`\n\nUploads the results to a SWIFT container.\n\nExpects a JSON file with SWIFT settings.\n\n**IMPORTANT**  When this file is deployed to the HPC\nmake sure it is kept in a service accounts's home folder and the file itself\nis only readable by that specific user (`chmod 0600`)!!!\n\nYOU DO NOT WANT TO SHARE THESE CREDENTIALS WITH OTHERS.\n\nExample:\n\n```\n{\n  \"auth_version\": \"3\",\n  \"os_username\": \"username\",\n  \"os_password\": \"password\",\n  \"os_project_id\": \"08c08f9f119744cbbf77e216988da3eb\",\n  \"os_auth_url\": \"https://pollux.cscs.ch:13000/v3\".\n  \"os_storage_url\": \"https://object.cscs.ch/v1/AUTH_08c08f9f119744cbbf77e216988da3eb\"\n}\n```\n\n# Deployment\n\nManual operation for now.\n\n* Build a source distribution,\n  * `python setup.py sdist`\n* then copy it to the HPC cluster,\n  * ```\n    VERSION=$(grep version setup.py | cut -d= -f2 | cut -d\\' -f2) \\\n    PKG=hbp-image-tasks-$VERSION.tar.gz \\\n    rsync -av \\\n      dist/$PKG \\\n      ich019sa@ela.cscs.ch:~\n    ```\n* finally install it to the existing virtualenv\n  * SSH to ela.cscs.ch\n  * SSH to daint.cscs.ch (no Python on ela)\n  * activate virtualenv and install package.\n    * `/store/hbp/ich019/img-svc-tasks/bin/pip install /users/ich019sa/hbp-image-tasks-0.0.1.tar.gz`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhumanbrainproject%2Fimg-svc-tasks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhumanbrainproject%2Fimg-svc-tasks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhumanbrainproject%2Fimg-svc-tasks/lists"}