{"id":16940234,"url":"https://github.com/greenpau/argo-workflows-aws-plugin","last_synced_at":"2025-07-02T07:35:45.806Z","repository":{"id":201091388,"uuid":"706699271","full_name":"greenpau/argo-workflows-aws-plugin","owner":"greenpau","description":"Argo Workflows Executor Plugin for AWS Services, e.g. SageMaker Pipelines, Glue, etc.","archived":false,"fork":false,"pushed_at":"2023-11-10T20:14:30.000Z","size":71,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-31T18:58:24.682Z","etag":null,"topics":["argo-workflows","argo-workflows-plugin","sagemaker","sagemaker-pipelines"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/greenpau.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null}},"created_at":"2023-10-18T13:03:19.000Z","updated_at":"2024-05-07T03:53:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"75d52562-1f9e-4ace-8256-c28686d29742","html_url":"https://github.com/greenpau/argo-workflows-aws-plugin","commit_stats":{"total_commits":50,"total_committers":2,"mean_commits":25.0,"dds":"0.18000000000000005","last_synced_commit":"bacc800891e3702bba26e5d46c06818d99e775af"},"previous_names":["greenpau/argo-workflows-aws-plugin"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greenpau%2Fargo-workflows-aws-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greenpau%2Fargo-workflows-aws-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greenpau%2Fargo-workflows-aws-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greenpau%2Fargo-workflows-aws-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/greenpau","download_url":"https://codeload.github.com/greenpau/argo-workflows-aws-plugin/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greenpau%2Fargo-workflows-aws-plugin/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259179347,"owners_count":22817533,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["argo-workflows","argo-workflows-plugin","sagemaker","sagemaker-pipelines"],"created_at":"2024-10-13T21:06:37.762Z","updated_at":"2025-07-02T07:35:45.752Z","avatar_url":"https://github.com/greenpau.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# argo-workflows-aws-plugin\n\n\u003ca href=\"https://github.com/greenpau/argo-workflows-aws-plugin/actions/\" target=\"_blank\"\u003e\u003cimg src=\"https://github.com/greenpau/argo-workflows-aws-plugin/workflows/build/badge.svg\"\u003e\u003c/a\u003e\n\nArgo Workflows Executor Plugin for AWS Services, e.g. SageMaker Pipelines, Glue, etc.\n\n\u003c!-- begin-markdown-toc --\u003e\n## Table of Contents\n\n* [Supported AWS Services](#supported-aws-services)\n* [Getting Started](#getting-started)\n  * [Add IAM Role and Policy](#add-iam-role-and-policy)\n  * [Enable Executor Plugins](#enable-executor-plugins)\n  * [Installation](#installation)\n  * [Add Workflow Template](#add-workflow-template)\n  * [Trigger Workflow](#trigger-workflow)\n  * [Uninstall Plugin](#uninstall-plugin)\n* [References](#references)\n\n\u003c!-- end-markdown-toc --\u003e\n\n## Supported AWS Services\n\nThe following tables describe the implementation state for the protocol's RPC\nmethods and database operations.\n\n| **Service Name** | **Implemented?** |\n| --- | --- |\n| Amazon SageMaker Pipelines | :heavy_check_mark: |\n| AWS Glue | :heavy_check_mark: |\n| AWS Step Functions | :heavy_check_mark: |\n| AWS Lambda | :construction: |\n\n## Getting Started\n\n### Add IAM Role and Policy\n\nThe plugin requires IAM role and policy to execute its operations.\n\nThe following CDK code add a role, which is later referenced in `plugin.yaml` manifest.\n\n```ts\n    const audClaim = `${cluster.clusterOpenIdConnectIssuer}:aud`;\n    const subClaim = `${cluster.clusterOpenIdConnectIssuer}:sub`;\n\n    const k8sConditions = new cdk.CfnJson(this, \"KubeOIDCCondition\", {\n      value: {\n        [audClaim]: \"sts.amazonaws.com\",\n        // [subClaim]: \"system:serviceaccount:kube-system:aws-node\",\n        [subClaim]: \"system:serviceaccount:argo:awf-aws-executor-plugin\",\n      },\n    });\n\n    const awfPluginRole = new cdk.aws_iam.Role(this, \"ArgoWorkflowsExecutorPluginRole\", {\n      roleName: `${stack.stackName}-awf-aws-executor-plugin`,\n      assumedBy: new cdk.aws_iam.WebIdentityPrincipal(\n        `arn:aws:iam::${cdk.Aws.ACCOUNT_ID}:oidc-provider/${cluster.clusterOpenIdConnectIssuer}`\n      ).withConditions({\n        StringEquals: k8sConditions,\n      }),\n    });\n\n    awfPluginRole.addToPolicy(new cdk.aws_iam.PolicyStatement({\n      effect: cdk.aws_iam.Effect.ALLOW,\n      resources: [\"arn:aws:sagemaker:*:*:pipeline/*\"],\n      actions: [\n        \"sagemaker:DescribePipeline\",\n        \"sagemaker:StartPipelineExecution\",\n        \"sagemaker:ListPipelineExecutionSteps\",\n        \"sagemaker:DescribePipelineExecution\",\n        \"sagemaker:ListPipelineExecutions\",\n        \"sagemaker:ListPipelines\"\n      ]\n    }));\n```\n\n### Enable Executor Plugins\n\nFirst, enable Executor Plugins:\n\n```bash\nkubectl patch deployment \\\n  workflow-controller \\\n  --namespace argo \\\n  --type='json' \\\n  -p='[{\"op\": \"add\", \"path\": \"/spec/template/spec/containers/0/env/0\", \"value\": {\n    \"name\": \"ARGO_EXECUTOR_PLUGINS\",\n    \"value\": \"true\",\n}}]'\n```\n\nNext, restart:\n\n```bash\nkubectl -n argo set env deployment/workflow-controller ARGO_EXECUTOR_PLUGINS=true\nkubectl rollout restart -n argo deployment workflow-controller\n```\n\n### Installation\n\nDownload the plugin manifest:\n\n```bash\nwget https://raw.githubusercontent.com/greenpau/argo-workflows-aws-plugin/main/assets/plugin.yaml\n```\n\nEdit `metadata.annotations.eks.amazonaws.com/role-arn` in the `ServiceAccount`. (see `DEVELOPMENT.md` for\nmore information about associated IAM role and policy)\n\nNext, install the plugin:\n\n```bash\nkubectl apply -f plugin.yaml\n```\n\nThe output follows:\n\n```\nserviceaccount/awf-aws-plugin-sa unchanged\nclusterrole.rbac.authorization.k8s.io/argo-plugin-addition-role unchanged\nclusterrolebinding.rbac.authorization.k8s.io/awf-aws-plugin-addition-binding unchanged\nclusterrolebinding.rbac.authorization.k8s.io/awf-aws-plugin-binding unchanged\nconfigmap/awf-aws-plugin created\n```\n\nList Argo Workflows Executor Plugins again:\n\n```\n$ kubectl get cm -l workflows.argoproj.io/configmap-type=ExecutorPlugin -n argo\n\nNAME             DATA   AGE\nawf-aws          2      34s\n```\n\nGet details about the plugins:\n\n```bash\nkubectl describe cm -l workflows.argoproj.io/configmap-type=ExecutorPlugin -n argo\n```\n\n### Add Workflow Template\n\nCreate a workflow template:\n\n```bash\nkubectl apply -f https://raw.githubusercontent.com/greenpau/argo-workflows-aws-plugin/main/assets/amz-sagemaker-pipelines-workflow-template.yaml\n```\n\n### Trigger Workflow\n\nStart new workflow:\n\n```bash\nkubectl create -f https://raw.githubusercontent.com/greenpau/argo-workflows-aws-plugin/main/assets/amz-sagemaker-pipelines-workflow.yaml\n```\n\nThe output follows:\n\n```\nworkflow.argoproj.io/sm-pipelines-tswbm created\n```\n\nReview the status of the workflow by the its name, e.g. `sm-pipelines-tswbm`:\n\n```bash\nkubectl describe pod -n argo sm-pipelines-tswbm-1340600742-agent\n```\n\nReview logs of the containers (`main`, `awf-aws`) inside the pod:\n\n```bash\nkubectl logs -n argo -c main sm-pipelines-tswbm-1340600742-agent\nkubectl logs -n argo -c awf-aws sm-pipelines-tswbm-1340600742-agent\n```\n\n### Uninstall Plugin\n\nIf necessary, run the following commands to uninstall the plugin:\n\n```bash\nkubectl delete -f https://raw.githubusercontent.com/greenpau/argo-workflows-aws-plugin/main/assets/plugin.yaml\n```\n\n## References\n\n* [Argo Workflows - Plugin Directory](https://argoproj.github.io/argo-workflows/plugin-directory/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgreenpau%2Fargo-workflows-aws-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgreenpau%2Fargo-workflows-aws-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgreenpau%2Fargo-workflows-aws-plugin/lists"}