{"id":13541608,"url":"https://github.com/awslabs/aws-codepipeline-governance","last_synced_at":"2025-04-02T09:32:02.811Z","repository":{"id":44547069,"uuid":"370497417","full_name":"awslabs/aws-codepipeline-governance","owner":"awslabs","description":null,"archived":false,"fork":false,"pushed_at":"2022-02-08T20:35:53.000Z","size":720,"stargazers_count":10,"open_issues_count":0,"forks_count":6,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-13T02:37:43.171Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/awslabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-24T22:12:27.000Z","updated_at":"2024-09-25T12:54:06.000Z","dependencies_parsed_at":"2022-09-10T06:13:21.910Z","dependency_job_id":null,"html_url":"https://github.com/awslabs/aws-codepipeline-governance","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Faws-codepipeline-governance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Faws-codepipeline-governance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Faws-codepipeline-governance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awslabs%2Faws-codepipeline-governance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/awslabs","download_url":"https://codeload.github.com/awslabs/aws-codepipeline-governance/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246788653,"owners_count":20834132,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T10:00:51.862Z","updated_at":"2025-04-02T09:32:02.251Z","avatar_url":"https://github.com/awslabs.png","language":"Python","funding_links":[],"categories":["Public Cloud Governance"],"sub_categories":["AWS Governance"],"readme":"# CodePipeline Governance\nEnsure all AWS CodePipelines have mandatory actions being run in the Orchestration Pipeline.\n\n## Description\nThis solution will be initiated through a pipeline action and will parse through the CodePipeline CloudFormation template to ensure all governance rules are met before updating the CloudFormation Stack.  This \nallows the Security or Governance teams to mandate certain stages and/or actions to being ran in a particular order.  But more importantly, this will allow developers to update their own CI/CD Pipeline.  \n\n### Folder Structure\n| Folder/File | Description |  \n| :-------------------------| :-------------------------------------------------------------------------------------------------------------------|\n| cloudformation/sam-bootstrap.yaml  | AWS Cloudformation template that will create the required AWS Resources for the solution to work properly. It will create an IAM Role, KMS Key/Alias and S3 Bucket. All of these AWS Resources are required for an AWS Serverless Application Model (SAM) deployment to successful.|\n| cloudformation/function.yaml | SAM template that will deploy the AWS Lambda Function along with all dependant infrastructure. | \n| src     | Source code for AWS Lambda Functions. |\n| test     | Test code for AWS Lambda Functions. |\n| scripts   | Directory that has the scripts that will be ran from a CodeBuild BuildSpec file |\n| scripts/main.sh | An orchestration script that will run the all other linting/scanning scripts before building/deploying the SAM Function(s). | \n| scripts/pylint.sh   | Shell script that will run the ```pylint``` command against all python files. |\n| scripts/pyscan.sh   | Runs Bandit (python lib) against all python code within the repository to identify any security vulnerabilities. |\n| scripts/sam.sh   | Runs a number of SAM commands to package / build / deploy the SAM Function to a specified account. | \n| scripts/test.sh   | Shell script that will run the ```tox``` command to build a virtual environment and the ```pytest``` command to run any unit tests found in the repository. |\n| pytest.ini   | ini files are the configuration files of the tox project, and can also be used to hold pytest configuration if they have a [pytest] section. |\n| test_requirements.txt   | Pip requirements file for test environment. |\n| tox.ini | Configured file for Tox. Tox is a command-line driven automated testing tool for Python, based on the use of virtualenv. |\n\n## Pre-requisite Steps:\n- Install the Serverless Application Model CLI (SAM) [Link to AWS Doc](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)\n  - Since this solution builds the SAM function inside Lambda-like container, Docker must be installed and running on your workstation.\n\n## Solution Architecture\nThere are 2 ways of deploying the CodePipeline Governance solution. \n\n### Separate Accounts AWS Resource Diagram (Option 1)\n![alt text](images/separate-account-arch-diag.png) \n\nThe *first deployment* method uses AWS Organizations and Centralized Shared Services account to manage the CodePipeline Governance Lambda Function. \nThis method allows other AWS Accounts within the Organization the ability to call the Lambda Function using a specific IAM Role. This eases the \nmanagement of the CodePipeline Governance Lambda Function and enables CodePipeline's within multiple AWS Accounts to invoke the function. \n\n### Single Account AWS Resource Diagram (Option 2)\n![alt text](images/single-account-arch-diag.png) \n\nThe *second deployment* method installs the CodePipeline Governance Lambda Function in the same account as your deployment of AWS CodePipeline.  This \nmethod is typically chosen when a Centralized Shared Services account or AWS Organization isn't used.  \n\n## How to deploy Option 1 (Function and Codepipeline exist in separate accounts)\n### Deploy to Shared Services Account\n- Run the cloudformation/sam-bootstrap.yaml into the AWS Account. This will deploy an S3 Bucket and KMS Key for the use of the SAM deployment.\n  ```bash\n  aws cloudformation deploy --stack-name SAM-Bootstrap --template-file cloudformation/sam-bootstrap.yaml\n  ```\n\n- Deploy the ScanCodePipeline Serverless Application Model (SAM) Lambda function and dependencies.\n    - Replace \"Orchestration-Service\" if you are using a different CodePipeline Service Role. \"Orchestration-Service\" is \n      what is used in both CodePipeline examples (codepipeline-example.yaml and codepipeline-sep-acc-example.yaml). \n  ```bash\n  # -o =\u003e AWS Organizational ID\n  # -r =\u003e CodePipeline Service Role Name\n  \n  ORG_ID=$(aws organizations describe-organization --query 'Organization.Id' --output text) \n  bash ./scripts/sam.sh -o ${ORG_ID} -r Orchestration-Service\n  ````\n\n- Deploy Initial DynamoDB Item into the Scan_CodePipeline_Rules Table.\n  ```bash\n  aws dynamodb put-item --table-name Scan_CodePipeline_Rules --item file://initial-dynamodb-item-sep-acc.json\n  ``` \n    \n### Deploy to AWS CodePipeline Account \n- This deploys an IAM Role for the Lambda Function to assume back into the Pipeline Account to respond with the CodePipeline Status (Success | Failure)\n  - Replace \"123456789012\" with AWS Account where the ScanCodePipeline Lambda Function was deployed\n  ```bash\n  aws cloudformation deploy --stack-name ScanCodePipelineAssumedRole --template-file cloudformation/assumed-iam-role.yaml --capabilities CAPABILITY_NAMED_IAM --parameter-overrides pScanCodePipelineAccount=123456789012\n  ````\n  \n### Deploy/Run CodePipeline Example\n- Deploy CodePipeline (w/ dependencies) CloudFormation template.\n  - Replace \"123456789012\" with AWS Account where the ScanCodePipeline Lambda Function was deployed\n  ```bash\n  aws cloudformation deploy --stack-name ScanCodePipelineExample --template-file cloudformation/codepipeline-sep-acc-example.yaml --capabilities CAPABILITY_NAMED_IAM --parameter-overrides pScanCodePipelineAccount=123456789012 \n  ``` \n\n- Zip up application directory.\n  ```bash\n  zip -r application.zip * -x \"*.aws-sam*\" -x \"*.DS_Store*\" \n  ```\n\n- Update zip to S3 to initiate the pipeline.\n  ```bash\n  AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)\n  AWS_REGION=$(aws configure get region) || AWS_REGION='us-east-1'\n  aws s3 cp application.zip s3://orchestration-${AWS_ACCOUNT_ID}-${AWS_REGION}/application.zip \n  ```\n \n## How to deploy Option 2 (Function and Codepipeline exist in the same account)\n### Deploy to AWS Account\n- Run the cloudformation/sam-bootstrap.yaml into the AWS Account. This will deploy an S3 Bucket and KMS Key for the use of the SAM deployment.\n  ```bash\n  aws cloudformation deploy --stack-name SAM-Bootstrap --template-file cloudformation/sam-bootstrap.yaml\n  ```\n  \n- Deploy the ScanCodePipeline Serverless Application Model (SAM) Lambda function and dependencies.\n  ```bash\n  bash ./scripts/sam.sh \n  ````\n  \n- Deploy Initial DynamoDB Item into the Scan_CodePipeline_Rules Table.\n  ```bash\n  aws dynamodb put-item --table-name Scan_CodePipeline_Rules --item file://initial-dynamodb-item.json\n  ```\n\n### Deploy/Run CodePipeline Example\n- Deploy CodePipeline (w/ dependencies) CloudFormation template.\n  ```bash\n  aws cloudformation deploy --stack-name ScanCodePipelineExample --template-file cloudformation/codepipeline-example.yaml --capabilities CAPABILITY_NAMED_IAM \n  ```\n\n- Zip up application directory.\n  ```bash\n  zip -r application.zip * -x \"*.aws-sam*\" -x \"*.DS_Store*\" \n  ```\n\n- Update zip to S3 to initiate the pipeline.\n  ```bash\n  AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)\n  AWS_REGION=$(aws configure get region) || AWS_REGION='us-east-1'\n  aws s3 cp application.zip s3://orchestration-${AWS_ACCOUNT_ID}-${AWS_REGION}/application.zip \n  ```\n\n## What the Pipeline should look like\n![alt text](images/codepipeline-example.png) \n\n## Deploy New / Update CodePipeline Governance Rules\n- Create a yaml file with the CodePipeline Stage or Actions you wish to act as the Rule for the CodePipeline CloudFormation Template.\n    - There is an example of a Rule located in *scripts/convert-2-dynamodb-item.yaml*. This example Rule ensures that the Scan \n      Action doesn't get removed from the CodePipeline Template.\n    - Additional Notes \n      - Update the *RuleNumber* value if creating multiple rules.\n      - *PatternType* is a future feature that will let a users associate certain rules to specific pipelines based on the pipeline's tag. \n    ```yaml\n    RuleNumber: \"001\"\n    PatternType: \"All\" \n    Contents:\n      Stages:\n        - Name: BuildAndPackage\n          Actions:\n            - Name: Scan-CodePipeline\n              ActionTypeId:\n                Category: Invoke\n                Owner: AWS\n                Provider: Lambda\n                Version: \"1\"\n              Configuration:\n                FunctionName: ScanUpdateCodePipeline\n                UserParameters: |\n                  {\n                    \"cfn_template\": \"cloudformation/codepipeline-example.yaml\"\n                  }\n              InputArtifacts:\n                - Name: Source\n              RunOrder: 1\n            - Name: Update-CodePipeline\n              ActionTypeId:\n                Category: Deploy\n                Owner: AWS\n                Version: \"1\"\n                Provider: CloudFormation\n              InputArtifacts:\n                - Name: Source\n              Configuration:\n                ActionMode: CREATE_UPDATE\n                Capabilities: CAPABILITY_NAMED_IAM\n                TemplatePath: Source::cloudformation/codepipeline-example.yaml\n              RunOrder: 2\n    ```\n \n- Run the convert-2-dynamodb-item script to build the Rule to fit the DynamoDB format.\n  ```bash\n  python scripts/convert-2-dynamodb-item.py \u003e new_initial-dynamodb-item.json\n  ```\n  \n- Deploy New/Updated DynamoDB Item into the Scan_CodePipeline_Rules Table.\n  ```bash\n  aws dynamodb put-item --table-name Scan_CodePipeline_Rules --item file://new_initial-dynamodb-item.json\n  ``` \n  \nEnd Result - Rule Example (located in DynamoDB):\n\n![alt text](images/rule-example-dynamodb.png) \n\n### What can NOT be added to a Governance Rule\n- [Intrinsic Functions](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html)\n\n## How does ScanCodepipeline Function get used in CodePipeline\nExample code from CodePipeline.yaml:\n```yaml\n...\n    - Name: BuildAndPackage\n      Actions:\n        - Name: Scan-CodePipeline\n          ActionTypeId:\n            Category: Invoke\n            Owner: AWS\n            Provider: Lambda\n            Version: \"1\"\n          Configuration:\n            FunctionName: ScanCodePipeline\n            UserParameters: !Sub |\n              {\n                \"cfn_template\": \"iac/cloudformation/codepipeline-example.yaml\"\n              }\n          InputArtifacts:\n            - Name: Source\n          OutputArtifacts: []\n          RunOrder: 1\n...\n```\n\n## Failure Definitions\nCodePipeline action output of CloudFormation deployment failure\n\n![alt text](images/failure.png) \n\nCodePipeline action output of CloudFormation deployment failure with output being over 500 characters\n\n![alt text](images/failure-over_500_characters.png)\n\nCodePipeline action output of CloudFormation validation failure \n\n![alt text](images/failure-cfn_validation_error.png) \n\nCodePipeline action output of CloudFormation Security Scan\n\n![alt text](images/failure-scan.png) \n\n## License\nThis project is licensed under the Apache-2.0 License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawslabs%2Faws-codepipeline-governance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fawslabs%2Faws-codepipeline-governance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawslabs%2Faws-codepipeline-governance/lists"}