{"id":18602135,"url":"https://github.com/build-on-aws/ci-cd-serverless-spark","last_synced_at":"2025-10-29T16:35:39.656Z","repository":{"id":156470204,"uuid":"624163592","full_name":"build-on-aws/ci-cd-serverless-spark","owner":"build-on-aws","description":"Sample CI/CD pipeline for using GitHub Actions with Amazon EMR Serverless Spark.","archived":false,"fork":false,"pushed_at":"2023-04-07T17:43:20.000Z","size":14,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-12-17T17:40:53.560Z","etag":null,"topics":["amazon-emr","apache-spark","aws","github-actions","serverless","spark"],"latest_commit_sha":null,"homepage":"https://buildon.aws/tutorials/deploy-serverless-spark-jobs-to-aws-using-github-actions","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit-0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/build-on-aws.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-05T22:04:23.000Z","updated_at":"2024-03-05T19:36:18.000Z","dependencies_parsed_at":null,"dependency_job_id":"e9109fbc-3fe0-4b25-99b9-456efa88955a","html_url":"https://github.com/build-on-aws/ci-cd-serverless-spark","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/build-on-aws%2Fci-cd-serverless-spark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/build-on-aws%2Fci-cd-serverless-spark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/build-on-aws%2Fci-cd-serverless-spark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/build-on-aws%2Fci-cd-serverless-spark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/build-on-aws","download_url":"https://codeload.github.com/build-on-aws/ci-cd-serverless-spark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231417245,"owners_count":18373624,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon-emr","apache-spark","aws","github-actions","serverless","spark"],"created_at":"2024-11-07T02:10:28.396Z","updated_at":"2025-10-29T16:35:34.627Z","avatar_url":"https://github.com/build-on-aws.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Serverless Spark CI/CD on AWS with GitHub Actions\n\nSample code repo that shows how to run GitHub Actions with Amazon EMR Serverless. Forms the basis of the following tutorial:\n\n- [Deploy Serverless Spark jobs to AWS using GitHub Actions](https://buildon.aws/tutorials/deploy-serverless-spark-jobs-to-aws-using-github-actions)\n\nThe code here represents what your repository should look like after following the tutorial.\n\n## Pre-requisites\n\n- An AWS Account with Admin privileges\n- GitHub OIDC Provider in AWS\n- S3 Bucket(s)\n- EMR Serverless Spark application(s)\n- IAM Roles for GitHub and EMR Serverless\n\nYou can create all of these, including some sample data, using the included [CloudFormation template](template.cfn.yaml).\n\n\u003e **Warning** 💰 The CloudFormation template creates EMR Serverless applications that you will be charged for when integration tests **AND** the scheduled workflow runs.\n\n\u003e **Note** The IAM roles created in the template are _very tightly scoped_ to the relevant S3 Buckets and EMR Serverless applications created by the stack.\n\n## Other resources\n\n- [EMR Serverless User Guide](https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/emr-serverless.html)\n- [EMR Serverless Samples](https://github.com/aws-samples/emr-serverless-samples/)\n- [EMR toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=AmazonEMR.emr-tools)\n- [Amazon EMR CLI](https://github.com/awslabs/amazon-emr-cli)\n\n## Security\n\nSee [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.\n\n## License\n\nThis library is licensed under the MIT-0 License. See the LICENSE file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuild-on-aws%2Fci-cd-serverless-spark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbuild-on-aws%2Fci-cd-serverless-spark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuild-on-aws%2Fci-cd-serverless-spark/lists"}