{"id":17320953,"url":"https://github.com/tom-tan/shaft","last_synced_at":"2026-01-06T10:46:21.062Z","repository":{"id":37831885,"uuid":"449150027","full_name":"tom-tan/shaft","owner":"tom-tan","description":"A workflow engine for CWL CommandLineTool","archived":false,"fork":false,"pushed_at":"2024-08-30T03:31:45.000Z","size":900,"stargazers_count":5,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-01T09:13:20.142Z","etag":null,"topics":["commonwl","workflow-engine"],"latest_commit_sha":null,"homepage":"","language":"D","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tom-tan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-18T05:16:51.000Z","updated_at":"2024-11-02T07:18:16.000Z","dependencies_parsed_at":"2024-07-13T10:29:31.405Z","dependency_job_id":"d4b81c4d-d49f-426a-822f-68614fb29a5b","html_url":"https://github.com/tom-tan/shaft","commit_stats":null,"previous_names":[],"tags_count":35,"template":false,"template_full_name":"tom-tan/dlang-for-remote-container-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tom-tan%2Fshaft","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tom-tan%2Fshaft/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tom-tan%2Fshaft/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tom-tan%2Fshaft/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tom-tan","download_url":"https://codeload.github.com/tom-tan/shaft/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245779431,"owners_count":20670684,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["commonwl","workflow-engine"],"created_at":"2024-10-15T13:34:26.406Z","updated_at":"2026-01-06T10:46:21.033Z","avatar_url":"https://github.com/tom-tan.png","language":"D","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Shaft: A workflow engine for CommandLineTool in local machine\n[![release](https://badgen.net/github/release/tom-tan/shaft)](https://github.com/tom-tan/shaft/releases/latest)\n[![license](https://badgen.net/github/license/tom-tan/shaft)](https://github.com/tom-tan/shaft/blob/main/LICENSE)\n[![CI](https://github.com/tom-tan/shaft/actions/workflows/ci.yml/badge.svg)](https://github.com/tom-tan/shaft/actions/workflows/ci.yml)\n\nShaft is a workflow engine to be used as a \"shaft\" of other workflow engines for the [Common Workflow Language](https://www.commonwl.org/) (CWL).\n\nThe main purpose of this engine is to be embedded in more enhanced workflow engines such as [ep3](https://github.com/tom-tan/ep3) and therefore it focuses on supporting [CommandLineTool](https://www.commonwl.org/v1.2/CommandLineTool.html) and [ExpressionTool](https://www.commonwl.org/v1.2/Workflow.html#ExpressionTool) documents in local machines.\n\n## Main features\n- Easy to deploy (statically-linked binary)\n- [Enable to take inherited process requirements and hints via input object](#passing-inherited-process-requirements-and-hints-via-input-object)\n    - it is necessary to integrate shaft with other workflow engine that supports `Workflow`\n\n## Conformance tests for CWL v1.0\n[![release](https://badgen.net/github/release/tom-tan/shaft)](https://github.com/tom-tan/shaft/releases/latest) ![commit](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/commit.json)\n\n### Classes\n[![CommandLineTool](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/command_line_tool.json?icon=commonwl)](https://github.com/tom-tan/conformance/blob/master/shaft/cwl_v1.0/shaft_latest/command_line_tool.md) [![ExpressionTool](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/expression_tool.json?icon=commonwl)](https://github.com/tom-tan/conformance/blob/master/shaft/cwl_v1.0/shaft_latest/expression_tool.md)\n\n### Required features\n[![Required](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/required.json?icon=commonwl)](https://github.com/tom-tan/conformance/blob/master/shaft/cwl_v1.0/shaft_latest/required.md)\n\n### Optional features\n[![DockerRequirement](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/docker.json?icon=commonwl)](https://github.com/tom-tan/conformance/blob/master/shaft/cwl_v1.0/shaft_latest/docker.md) [![EnvVarRequirement](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/env_var.json?icon=commonwl)](https://github.com/tom-tan/conformance/blob/master/shaft/cwl_v1.0/shaft_latest/env_var.md) [![InitialWorkDirRequirement](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/initial_work_dir.json?icon=commonwl)](https://github.com/tom-tan/conformance/blob/master/shaft/cwl_v1.0/shaft_latest/initial_work_dir.md) [![InlineJavascriptRequirement](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/inline_javascript.json?icon=commonwl)](https://github.com/tom-tan/conformance/blob/master/shaft/cwl_v1.0/shaft_latest/inline_javascript.md) [![ResourceRequirement](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/resource.json?icon=commonwl)](https://github.com/tom-tan/conformance/blob/master/shaft/cwl_v1.0/shaft_latest/resource.md) [![SchemaDefRequirement](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/schema_def.json?icon=commonwl)](https://github.com/tom-tan/conformance/blob/master/shaft/cwl_v1.0/shaft_latest/schema_def.md) [![ShellCommandRequirement](https://badgen.net/https/raw.githubusercontent.com/tom-tan/conformance/master/shaft/cwl_v1.0/shaft_latest/shell_command.json?icon=commonwl)](https://github.com/tom-tan/conformance/blob/master/shaft/cwl_v1.0/shaft_latest/shell_command.md)\n\n## Shaft extension\n### Taking inherited process requirements and hints via the input object\n\nThe [CWL specification](https://www.commonwl.org/v1.2/CommandLineTool.html#Requirements_and_hints) says that the step processes in a workflow must inherit requirements and hints declared in the parent workflow.\n\nTo integrate shaft with other workflow engine for `Workflow`, shaft can take inherited requirements and hints via `shaft:inherited-requirements` and `shaft:inherited-hints` in the input object. They must have an array of process requirements that are inherited from the parent workflow.\n\n- Examples: Inheriting `EnvVarRequirement` via `shaft:inherited-requirements`\n  - tests/tools/env-tool1-noenv.cwl\n    ```cwl\n    # based on https://github.com/common-workflow-language/common-workflow-language/blob/main/v1.0/v1.0/env-tool1.cwl\n    class: CommandLineTool\n    cwlVersion: v1.0\n    hints:\n      ResourceRequirement:\n        ramMin: 8\n    inputs:\n      in: string\n    outputs:\n      out:\n        type: File\n        outputBinding:\n          glob: out\n\n    # requirements:\n    #   EnvVarRequirement:\n    #     envDef:\n    #       TEST_ENV: $(inputs.in)\n\n    baseCommand: [\"/bin/sh\", \"-c\", \"echo $TEST_ENV\"]\n\n    stdout: out\n    ```\n\n  - test/jobs/senv-job.yml\n    ```yml\n    in: \"hello test env\"\n    shaft:inherited-requirements:\n      - class: EnvVarRequirement\n        envDef:\n          TEST_ENV: override\n    ```\n  - Execution result:\n    ```console\n    $ ./bin/shaft --quiet tests/tools/env-tool1-noenv.cwl tests/jobs/env-job.yml | jq .\n    {\n      \"out\": {\n        \"basename\": \"out\",\n        \"checksum\": \"sha1$cdc1e84968261d6a7575b5305945471f8be199b6\",\n        \"class\": \"File\",\n        \"dirname\": \"/workspaces/shaft\",\n        \"location\": \"file:///workspaces/shaft/out\",\n        \"nameext\": \"\",\n        \"nameroot\": \"out\",\n        \"path\": \"/workspaces/shaft/out\",\n        \"size\": 9\n      }\n    }\n    $ cat out\n    override\n    ```\n\n- Examples: Inheriting `EnvVarRequirement` via `shaft:inherited-requirements` but not overriding requirements in the process\n  - tests/tools/env-tool1.cwl\n    ```cwl\n    class: CommandLineTool\n    cwlVersion: v1.0\n    hints:\n      ResourceRequirement:\n        ramMin: 8\n    inputs:\n      in: string\n    outputs:\n      out:\n        type: File\n        outputBinding:\n          glob: out\n\n    requirements:\n      EnvVarRequirement:\n        envDef:\n          TEST_ENV: $(inputs.in)\n\n    baseCommand: [\"/bin/sh\", \"-c\", \"echo $TEST_ENV\"]\n\n    stdout: out\n    ```\n\n  - test/jobs/senv-job.yml (same as previous example)\n    ```yml\n    in: \"hello test env\"\n    shaft:inherited-requirements:\n      - class: EnvVarRequirement\n        envDef:\n          TEST_ENV: override\n    ```\n  - Execution result:\n    ```console\n    $ ./bin/shaft --quiet tests/tools/env-tool1.cwl tests/jobs/env-job.yml | jq .\n    {\n      \"out\": {\n        \"basename\": \"out\",\n        \"checksum\": \"sha1$b3ec4ed1749c207e52b3a6d08c59f31d83bff519\",\n        \"class\": \"File\",\n        \"dirname\": \"/workspaces/shaft\",\n        \"location\": \"file:///workspaces/shaft/out\",\n        \"nameext\": \"\",\n        \"nameroot\": \"out\",\n        \"path\": \"/workspaces/shaft/out\",\n        \"size\": 15\n      }\n    }\n    $ cat out\n    hello test env\n    ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftom-tan%2Fshaft","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftom-tan%2Fshaft","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftom-tan%2Fshaft/lists"}