{"id":24503624,"url":"https://github.com/plus3it/wrangler-watchmaker","last_synced_at":"2026-01-02T03:05:09.902Z","repository":{"id":40167351,"uuid":"132001070","full_name":"plus3it/wrangler-watchmaker","owner":"plus3it","description":"Manages buckets and files needed for the public/default watchmaker configuration","archived":false,"fork":false,"pushed_at":"2025-02-24T00:10:06.000Z","size":396,"stargazers_count":2,"open_issues_count":0,"forks_count":7,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-02-24T01:22:08.721Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/plus3it.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-03T13:42:52.000Z","updated_at":"2025-02-24T00:10:08.000Z","dependencies_parsed_at":"2023-02-09T09:00:44.016Z","dependency_job_id":"b1a391ed-f7b0-455b-9fab-9c7ef68aefd5","html_url":"https://github.com/plus3it/wrangler-watchmaker","commit_stats":null,"previous_names":[],"tags_count":61,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plus3it%2Fwrangler-watchmaker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plus3it%2Fwrangler-watchmaker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plus3it%2Fwrangler-watchmaker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plus3it%2Fwrangler-watchmaker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/plus3it","download_url":"https://codeload.github.com/plus3it/wrangler-watchmaker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243703817,"owners_count":20334048,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-21T23:17:52.889Z","updated_at":"2026-01-02T03:05:09.852Z","avatar_url":"https://github.com/plus3it.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# wrangler-watchmaker\nManages public buckets and files needed for the default `watchmaker`\nconfiguration.\n\nThis project uses `dev` and `release` pipelines to manage the public files and\nother content needed for `watchmaker`, when using its default/public\nconfiguration. All resources in the pipelines are defined in terraform\nconfigurations and managed using terragrunt.\n\nThe `dev` pipeline is currently executed manually, and the `release` pipeline\nexecutes automatically whenever the project version is bumped.\n\nThis project uses `terragrunt` to manage the backend for terraform state,\nand to reduce boilerplate code duplication in terraform root modules.\n\nTerragrunt is a simple wrapper around terraform... On the command line,\noptions passed to terragrunt are passed through to terraform. Commands you\nwould call as `terraform ...` you can instead just call as `terragrunt ...`.\n\nTerragrunt will create a cache folder in its working directory, copy the root\nmodule there, execute terragrunt hooks, and use `terraform init` to pull in\nexternal modules and plugins. In some ways, Terragrunt is a bit like a Python\nvirtualenv, in that it isolates the working environment from the rest of the\nsystem.\n\n## Dependencies\n\nThis project uses modules with external terraform resources that execute\npython scripts. The terragrunt configuration will ensure the python package\nrequirements are installed. However, python packages often require root\nprivileges to install. To work around that, and improve cross-platform\ncompatibility, this project installs python requirements into a virtualenv\nusing `pipenv`. This means you must ensure `pipenv` is installed and in your\nPATH.\n\n* *nix:\n\n    ```\n    sudo pip install pipenv\n    ```\n\n* MacOS:\n\n    ```\n    brew install pipenv\n    ```\n\n* Windows:\n\n    ```\n    pip install pipenv\n    ```\n\n* Pip's \"user\" environment (not always well-supported...):\n\n    ```\n    pip install --user pipenv\n    ```\n\n## Dev pipeline\n\nThe `dev` pipeline consists of three configurations: bucket, files-repo, and\nsalt-repo.\n\n### Dev configurations\n\nThe bucket configuration creates an S3 bucket and applies a bucket policy.\n\nThe files-repo configuration retrieves files from various http/s, s3, or local\nsources and places them in the S3 bucket.\n\nThe salt-repo configuration uses `rsync` to mirror the SaltStack yum repo\nlocally, and the `aws` cli to sync that repo to the S3 bucket.\n\n### Dev workflows\n\nThere are three typical `dev` workflows:\n\n* add/modify the file repo\n* add/modify salt versions\n* execute terraform/terragrunt to apply the updated configurations\n\nAfter updating the file repo or salt version, commit the change and open a pull\nrequest. The change will be reviewed and merged. Once merged to `master`, the\nCI build system will automatically execute the third workflow to update the bucket\ncontents.\n\n#### Updating the dev file repo\n\nTo modify the file repo, update the `uri_map` in the\n[dev files-repo configuration](dev/files-repo/wrangler.auto.tfvars). The left\nside of the map (the key) represents the URI that will be retrieved. The right\nside of the map (the value) is the path in the S3 Bucket where the file will be\nstored.\n\n#### Updating the dev salt repo\n\nTo modify the salt versions, update `salt_version` and/or `extra_salt_versions`\nin the [dev salt-repo module](dev/salt-repo/wrangler.auto.tfvars).\n`salt_version` specifies the version of salt that will be used in the yum repo\ndefinition file hosted at an \"unversioned\" URI. `extra_salt_versions` is a list\nof additional salt versions to retrieve. These versions will get yum repo\ndefinition files with versioned URIs.\n\n#### Manually apply the dev configurations\n\nApplying the dev configurations *should* be handled by the build system\nautomatically. However, you *may* find occasion where you need to execute it\nmanually.\n\nThis workflow should be executed only _after_ updating the file repo or salt\nrepo configurations, and after the change has been reviewed and merged to the\nmaster branch.\n\nFirst, ensure you've installed `pipenv`, per the [Dependencies](#dependencies).\n\nTo execute terraform/terragrunt to update the bucket contents, checkout the\nmaster branch, update it from upstream, export the environment variables used\nby terragrunt for the backend state, and use the `deploy/dev` make target:\n\n```\ngit checkout master\ngit pull upstream master\nexport WRANGLER_BUCKET=\u003cwrangler-state-bucket\u003e\nexport WRANGLER_DDB_TABLE=\u003cwrangler-state-ddb\u003e\nexport AWS_DEFAULT_REGION=\u003cregion\u003e\nmake deploy/dev TF_VAR_bucket_name=\u003cdev-bucket\u003e TF_VAR_s3_objects_map='{}'\n```\n\n## Release pipeline\n\nThe `release` pipeline also consists of three configurations: bucket,\ncopy-bucket, and salt-yum-defs.\n\nExecution of the `release` pipeline is handled automatically by the build\nsystem. It is initiated when the version is bumped. This project uses\n[bumpversion](https://github.com/c4urself/bump2version) to manage project\nversioning.\n\nThe execution of the pipeline then is simple: use `bumpversion` to increment\nthe version, open a pull request, and if/when merged the build system will\nhandle the rest.\n\n### Release configurations\n\nThe bucket configuration creates an S3 bucket and applies a bucket policy.\n\nThe copy-bucket configuration copies files from the `dev` bucket to the\n`release` bucket. This is structured to include the file repo and the salt\nrepo, but _not_ the `dev` salt yum definitions (since they point at the `dev`\nbucket).\n\nThe salt-yum-defs configuration creates yum definitions for the salt repo that\npoint to the `release` bucket.\n\n### Release workflows\n\nThere is really only one `release` workflow: managing the salt version.\n\n#### Update the release salt version\n\nThis is very similar to the corresponding `dev` workflow, but only creates the\nyum repo definition files, since the repo packages are copied from the `dev`\nbucket.\n\nTo update the `release` salt version, update `salt_version` and/or\n`extra_salt_versions` in the\n[release salt-yum-defs configuration](release/salt-yum-defs/wrangler.auto.tfvars).\n\nSome care should be taken when deciding to modify `salt_version` in the\n`release` pipeline... `watchmaker` configs may be using the resulting\n\"unversioned\" URI repo definition. When the `salt_version` is modified,\n`watchmaker` clients will get the updated salt version.\n\n`extra_salt_versions` works just as it does in the `dev` pipeline...\n`watchmaker` clients may choose to use the versioned repo definitions in\ncustom configurations to pin/control the salt version they use.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplus3it%2Fwrangler-watchmaker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplus3it%2Fwrangler-watchmaker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplus3it%2Fwrangler-watchmaker/lists"}