{"id":20749294,"url":"https://github.com/agnostiqhq/covalent-aws-plugins","last_synced_at":"2025-10-04T11:49:38.409Z","repository":{"id":59412603,"uuid":"529014387","full_name":"AgnostiqHQ/covalent-aws-plugins","owner":"AgnostiqHQ","description":"Executor plugins interfacing Covalent with various AWS compute platforms","archived":false,"fork":false,"pushed_at":"2023-12-15T16:35:47.000Z","size":511,"stargazers_count":17,"open_issues_count":6,"forks_count":0,"subscribers_count":11,"default_branch":"develop","last_synced_at":"2025-09-09T00:53:58.531Z","etag":null,"topics":["aws","aws-batch","aws-braket","aws-ec2","aws-ecs","aws-lambda","braket","covalent","etl","hybrid-cloud","python","quantum"],"latest_commit_sha":null,"homepage":"https://covalent.xyz","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AgnostiqHQ.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-25T20:51:35.000Z","updated_at":"2024-02-26T08:47:51.000Z","dependencies_parsed_at":"2023-10-14T18:46:45.437Z","dependency_job_id":"3a2df4b9-9602-4e86-826f-63014819c822","html_url":"https://github.com/AgnostiqHQ/covalent-aws-plugins","commit_stats":{"total_commits":50,"total_committers":7,"mean_commits":7.142857142857143,"dds":0.62,"last_synced_commit":"0c92b4aa765217352f6eab57a57b928cbe5b22da"},"previous_names":[],"tags_count":26,"template":false,"template_full_name":null,"purl":"pkg:github/AgnostiqHQ/covalent-aws-plugins","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AgnostiqHQ%2Fcovalent-aws-plugins","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AgnostiqHQ%2Fcovalent-aws-plugins/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AgnostiqHQ%2Fcovalent-aws-plugins/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AgnostiqHQ%2Fcovalent-aws-plugins/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AgnostiqHQ","download_url":"https://codeload.github.com/AgnostiqHQ/covalent-aws-plugins/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AgnostiqHQ%2Fcovalent-aws-plugins/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278308629,"owners_count":25965654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-batch","aws-braket","aws-ec2","aws-ecs","aws-lambda","braket","covalent","etl","hybrid-cloud","python","quantum"],"created_at":"2024-11-17T08:22:06.893Z","updated_at":"2025-10-04T11:49:38.367Z","avatar_url":"https://github.com/AgnostiqHQ.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u0026nbsp;\n\n\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\"./doc/static/AWS_Plugins.png\" width=150%\u003e\n\n[![covalent](https://img.shields.io/badge/covalent-0.177.0-purple)](https://github.com/AgnostiqHQ/covalent)\n[![apache](https://img.shields.io/badge/License-Apache_License_2.0-blue)](https://www.apache.org/licenses/LICENSE-2.0)\n\n\u003c/div\u003e\n\n## Covalent AWS Plugins\n\n\u003cimg src=\"./doc/static/covalent-ec2-code-example.png\" width=\"550px\" align=\"right\"\u003e\n\n[Covalent](https://github.com/AgnostiqHQ/covalent) is a python based workflow orchestration tool used to execute HPC and quantum tasks in heterogenous environments.\n\nBy installing Covalent AWS Plugins users can leverage a broad plugin ecosystem to execute tasks using AWS resources best fit for each task.\n\nCovalent AWS Plugins installs a set of executor plugins that allow tasks to be run in an EC2 instance, AWS Lambda, AWS ECS Cluster, AWS Batch Compute Environment, and as an AWS Braket Job for tasks requiring Quantum devices.\n\nIf you're new to covalent visit our [Getting Started Guide](https://covalent.readthedocs.io/en/latest/getting_started/index.html).\n\n\n\n## Installation\n\nTo use the AWS plugin ecosystem with Covalent, simply install it with `pip`:\n\n```bash\npip install \"covalent-aws-plugins[all]\"\n```\n\n⚠️You must include [all] in order to install all of the AWS plugins\n\nThis will ensure that all the AWS executor plugins listed below are installed.\n\n\u003e You may require [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli) to be installed to use the AWS EC2 plugin.\n\nPlease read our [read the docs (RTD)](https://covalent.readthedocs.io/en/latest/api/executors/awsplugins.html) guide for more detailed information about using AWS plugins.\n\n## Included Plugins\n\nWhile each plugin can be installed separately, `covalent-aws-plugins` pip package will automatically download all of the plugins listed below.\n\n| | Plugin Name | Use Case |\n|---|-------------|-------------|\n|![AWS Batch](./doc/static/Batch.png)| AWS Batch Executor |**Useful for heavy compute workloads (high CPU/memory).** Tasks are queued to execute in the user defined Batch compute environment.|\n|![AWS EC2](./doc/static/EC2.png)|AWS EC2 Executor|**General purpose compute workloads where users can select compute resources.** An EC2 instance is auto-provisioned using terraform with selected compute settings to execute tasks.|\n|![AWS Braket](./doc/static/Braket.png)|AWS Braket Executor|**Suitable for Quantum/Classical hybrid workflows.** Tasks are executed using a combination of classical and quantum devices.|\n|![AWS ECS](./doc/static/ECS.png)|AWS ECS Executor|**Useful for moderate to heavy workloads (low memory requirements).** Tasks are executed in an AWS ECS cluster as containers.|\n|![AWS Lambda](./doc/static/Lambda.png)|AWS Lambda Executor|**Suitable for short lived tasks that can be parallalized (low memory requirements).** Tasks are executed in serverless AWS Lambda functions.|\n\n## Usage\n\nFirstly, **import covalent**.\n\n```python\nimport covalent as ct\n```\nSecondly, **define your executor(s)** (expand any of the below plugins).\n\n\u003cdetails\u003e\n  \u003csummary\u003eAWS Batch Executor\u003c/summary\u003e\n\n   Read more about how to use this executor in our [docs](https://covalent.readthedocs.io/en/latest/api/executors/awsbatch.html#usage-example). Below are the basics of how it can be used.\n\n```python\nexecutor = ct.executor.AWSBatchExecutor(\n    s3_bucket_name = \"covalent-batch-qa-job-resources\",\n    batch_job_definition_name = \"covalent-batch-qa-job-definition\",\n    batch_queue = \"covalent-batch-qa-queue\",\n    batch_execution_role_name = \"ecsTaskExecutionRole\",\n    batch_job_role_name = \"covalent-batch-qa-job-role\",\n    batch_job_log_group_name = \"covalent-batch-qa-log-group\",\n    vcpu = 2, # Number of vCPUs to allocate\n    memory = 3.75, # Memory in GB to allocate\n    time_limit = 300, # Time limit of job in seconds\n)\n\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eAWS EC2 Executor\u003c/summary\u003e\n\n   Read more about how to use this executor in our [docs](https://covalent.readthedocs.io/en/latest/api/executors/awsec2.html#usage-example). Below are the basics of how it can be used.\n\n```python\nexecutor = ct.executor.EC2Executor(\n\tinstance_type=\"t2.micro\",\n\tvolume_size=8, #GiB\n\tssh_key_file=\"~/.ssh/ec2_key\"\n)\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eAWS Braket Executor\u003c/summary\u003e\n\n   Read more about how to use this executor in our [docs](https://covalent.readthedocs.io/en/latest/api/executors/awsbraket.html#usage-example). Below are the basics of how it can be used.\n\n```python\nexecutor = ct.executor.BraketExecutor(\n    s3_bucket_name=\"braket_s3_bucket\",\n    ecr_repo_name=\"braket_ecr_repo\",\n    braket_job_execution_role_name=\"covalent-braket-iam-role\",\n    quantum_device=\"arn:aws:braket:::device/quantum-simulator/amazon/sv1\",\n    classical_device=\"ml.m5.large\",\n    storage=30,\n)\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eAWS ECS Executor\u003c/summary\u003e\n\n   Read more about how to use this executor in our [docs](https://covalent.readthedocs.io/en/latest/api/executors/awsecs.html#usage-example). Below are the basics of how it can be used.\n\n```python\nexecutor = ct.executor.ECSExecutor(\n    s3_bucket_name=\"covalent-fargate-task-resources\",\n    ecr_repo_name=\"covalent-fargate-task-images\",\n    ecs_cluster_name=\"covalent-fargate-cluster\",\n    ecs_task_family_name=\"covalent-fargate-tasks\",\n    ecs_task_execution_role_name=\"ecsTaskExecutionRole\",\n    ecs_task_role_name=\"CovalentFargateTaskRole\",\n    ecs_task_subnet_id=\"subnet-000000e0\",\n    ecs_task_security_group_id=\"sg-0000000a\",\n    ecs_task_log_group_name=\"covalent-fargate-task-logs\",\n    vcpu=1,\n    memory=2\n)\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eAWS Lambda Executor\u003c/summary\u003e\n\n   Read more about how to use this executor in our [docs](https://covalent.readthedocs.io/en/latest/api/executors/awslambda.html#usage-example). Below are the basics of how it can be used.\n\n```python\nexecutor = ct.executor.AWSLambdaExecutor(\n    lambda_role_name=\"CovalentLambdaExecutionRole\",\n    s3_bucket_name=\"covalent-lambda-job-resources\",\n    timeout=60,\n    memory_size=512\n)\n```\n\u003c/details\u003e\n\n\n- Lastly, define a workflow to execute a particular task using one of the above executors\n\n```python\n@ct.electron(\n    executor=executor\n)\ndef compute_pi(n):\n    # Leibniz formula for π\n    return 4 * sum(1.0/(2*i + 1)*(-1)**i for i in range(n))\n\n@ct.lattice\ndef workflow(n):\n    return compute_pi(n)\n\n\ndispatch_id = ct.dispatch(workflow)(100000000)\nresult = ct.get_result(dispatch_id=dispatch_id, wait=True)\nprint(result.result)\n\n```\n\nWhich should output\n\n```\n3.141592643589326\n```\n\n\n## Release Notes\n\nRelease notes are available in the [Changelog](https://github.com/AgnostiqHQ/covalent-aws-plugins/blob/main/CHANGELOG.md).\n\n## Citation\n\nPlease use the following citation in any publications:\n\n\u003e W. J. Cunningham, S. K. Radha, F. Hasan, J. Kanem, S. W. Neagle, and S. Sanand.\n\u003e *Covalent.* Zenodo, 2022. https://doi.org/10.5281/zenodo.5903364\n\n## License\n\nCovalent is licensed under the Apache License 2.0. See the [LICENSE](https://github.com/AgnostiqHQ/covalent-braket-plugin/blob/main/LICENSE) file or contact the [support team](mailto:support@agnostiq.ai) for more details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagnostiqhq%2Fcovalent-aws-plugins","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagnostiqhq%2Fcovalent-aws-plugins","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagnostiqhq%2Fcovalent-aws-plugins/lists"}