https://github.com/awsdocs/aws-doc-sdk-examples-tools
https://github.com/awsdocs/aws-doc-sdk-examples-tools
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/awsdocs/aws-doc-sdk-examples-tools
- Owner: awsdocs
- License: apache-2.0
- Created: 2024-01-26T21:02:02.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-09-08T14:50:23.000Z (4 months ago)
- Last Synced: 2025-09-08T16:39:58.518Z (4 months ago)
- Language: Python
- Size: 31.3 MB
- Stars: 12
- Watchers: 3
- Forks: 16
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Notice: NOTICE
Awesome Lists containing this project
README
## AWS Doc SDK Examples Tools
This python library is a set of tools to manage [AWS Doc SDK Example metadata](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/.doc_gen).
It is used by the AWS Doc SDK Examples team, as well as tributary sources of
example snippets.
-tools:
- Validates example metadata.
- Provides an API to program against example metadata.
- Hydrates additional derived data not explicitly written by engineers into example metadata.
## Check-in tests
### Purpose
The check-in tests are run whenever a pull request is submitted or changed. They
can be included in a Github Action with a job like this:
```
jobs:
validate:
runs-on: ubuntu-latest
steps:
- name: checkout repo content
uses: actions/checkout@v4
- name: validate metadata
uses: awsdocs/aws-doc-sdk-examples-tools@main
```
The check-in tests walk the full repository and scan code files to look for
the following issues.
- Disallow a list of specified words.
- Disallow any 20- or 40- character strings that fit a specified regex profile
that indicates they might be secret access keys. Allow strings that fit the
regex profile if they are in the allow list.
- Disallow file names that contain 20- or 40- character strings that fit the same
regex profile, unless the filename is in the allow list.
- Verify that snippet-start and snippet-end tags are in matched pairs. You are
not required to include these tags, but if you do they must be in pairs.
- Ensures any snippet_file in metadata excerpts are present in the repo.
A count of errors found is returned. When CI receives a non-zero return code,
it treats the checks as failed and displays a message in the pull request.
### Updating validations
The above configuration tracks the `main` branch directly. To follow more stable releases, use the most recent release tag in the github action.
```
uses: awsdocs/aws-doc-sdk-examples-tools@2024-08-26-A
```
### Running during development
```
python3.8 -m venv .venv
# With a python 3.8 venv in .venv
source .venv/bin/activate # Adjust for windows as necessary
python -m pip install -r requirements.txt
python -m pip install -e .
python -m mypy aws_doc_sdk_examples_tools
python -m pytest -vv
python -m black --check aws_doc_sdk_examples_tools
```
## Validation Extensions
Some validation options can be extended by creating `.doc_gen/validation.yaml`.
- `allow_list`: The 40-character check is _very_ sensitive. To allow certain patterns, add them as a string to the `allow_list` key, which will be loaded as a set of strings to allow.
- `sample_files`: Sample files are only allowed with certain names. To allow additional sample files, add their file name (with extension, but not path) to this list.
## New Releases
There are two stages, testing and deployment.
### 1. Testing
1. **Merge your changes into** [aws-doc-sdk-examples-tools/commits/main](https://github.com/awsdocs/aws-doc-sdk-examples-tools/commits/main).
2. **Create a testing branch** from [aws-doc-sdk-examples@main](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main).
3. **Find the commit SHA that matches the latest change from [aws-doc-sdk-examples-tools/commits/main](https://github.com/awsdocs/aws-doc-sdk-examples-tools/commits/main)**.
4. **Update the following files in your testing branch** with the commit SHA (format: `org/repo@hash`, e.g. `awsdocs/aws-doc-sdk-examples-tools@e7c283e916e8efc9113277e2f38c8fa855a79d0a`):
- In [.github/workflows/validate-doc-metadata.yml](https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/.github/workflows/validate-doc-metadata.yml), replace the current tag with the SHA.
- Add only the commit SHA to the `allow_list` field in [.doc_gen/validation.yaml](https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/.doc_gen/validation.yaml).
5. **Open a Draft PR to main branch**: Do not publish for review. Wait for checks/tests to pass on the PR.
### 2. Deployment
1. **Run `stamp.sh --release` from the `main` branch to automatically perform the following actions**:
- Update the `setup.py` version.
- Create a tag in the -tools repository at the same SHA you identified earlier.
- stamp.sh will create the next [stamp](https://blog.aspect.build/versioning-releases-from-a-monorepo) (which is valid [semver](https://packaging.python.org/en/latest/specifications/version-specifiers/#version-specifiers)) number as appropriate for the changes in this release. e.g. `2024.40.2`.
- Push the new tag to `main`
1. **Update your testing PR branch**
- Remove SHA and add tag to [validate-doc-metadata.yml](https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/.github/workflows/validate-doc-metadata.yml)
- Remove the SHA from [.doc_gen/validation.yaml](https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/.doc_gen/validation.yaml)
- This is easily accomplished in the Github UI.
1. **Create a release**: Use the automated ["Create release from tag" button](https://github.com/awsdocs/aws-doc-sdk-examples-tools/releases/new) to create a new release with the new tag.
1. **Perform internal update process**.
- See `update.sh` script in internal package.
## Security
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
## License
This project is licensed under the Apache-2.0 License.