{"id":28413988,"url":"https://github.com/sigstore/sigstore-conformance","last_synced_at":"2026-02-12T10:11:40.516Z","repository":{"id":61151694,"uuid":"541893186","full_name":"sigstore/sigstore-conformance","owner":"sigstore","description":"Conformance testing for Sigstore clients","archived":false,"fork":false,"pushed_at":"2025-06-19T07:54:03.000Z","size":340,"stargazers_count":9,"open_issues_count":27,"forks_count":13,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-06-19T08:42:07.652Z","etag":null,"topics":["conformance","sigstore","tests"],"latest_commit_sha":null,"homepage":"https://sigstore.dev","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sigstore.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-09-27T03:54:22.000Z","updated_at":"2025-06-19T07:54:06.000Z","dependencies_parsed_at":"2023-10-13T17:12:33.014Z","dependency_job_id":"d9bdbb75-b678-453a-8a1d-aad8bf88ef41","html_url":"https://github.com/sigstore/sigstore-conformance","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"purl":"pkg:github/sigstore/sigstore-conformance","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sigstore%2Fsigstore-conformance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sigstore%2Fsigstore-conformance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sigstore%2Fsigstore-conformance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sigstore%2Fsigstore-conformance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sigstore","download_url":"https://codeload.github.com/sigstore/sigstore-conformance/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sigstore%2Fsigstore-conformance/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261378532,"owners_count":23149740,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conformance","sigstore","tests"],"created_at":"2025-06-03T06:10:56.747Z","updated_at":"2026-02-12T10:11:40.511Z","avatar_url":"https://github.com/sigstore.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"sigstore-conformance\n====================\n\n\u003c!--- @begin-badges@ ---\u003e\n[![CI](https://github.com/sigstore/sigstore-conformance/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/sigstore/sigstore-conformance/actions/workflows/ci.yml)\n[![Self-test](https://github.com/sigstore/sigstore-conformance/actions/workflows/conformance.yml/badge.svg?branch=main)](https://github.com/sigstore/sigstore-conformance/actions/workflows/conformance.yml)\n\u003c!--- @end-badges@ ---\u003e\n\n`sigstore-conformance` is a conformance testing suite for Sigstore clients.\nTest results for known clients are published daily in the\n[Sigstore Client Conformance Report](https://sigstore.github.io/sigstore-conformance).\n\nThis suite provides a high-level view of client behaviour as a whole and sets\nout to answer questions such as:\n- Does the client fail when given a signing certificate that isn't signed by\n  the Fulcio root CA during the signing workflow?\n- Does the client fail when given an invalid inclusion proof from Rekor during\n  the verification workflow?\n- Does the client fail when given an invalid signed certificate timestamp as\n  part of the Fulcio response in the signing workflow?\n- etc\n\nAn official Sigstore client specification is being worked on at the moment as\npart of the [Sigstore Architecture Documentation](https://github.com/sigstore/architecture-docs).\nOnce it's complete, `sigstore-conformance` aims to be able to test a client's\nadherence to the specification.\n\nSome general testing principles for this suite are:\n- *Tests should be \"workflow\" focused.* This testing suite is not about fuzzing\n  every possible input to the client CLI or achieving code coverage.\n- *Tests should exercise the entire client end-to-end rather than individual\n  subsystems in isolation.* Tests should include all network interactions with\n  Sigstore infrastructure such as Rekor, Fulcio, etc. These tests should run\n  against Sigstore staging and production infrastructure as well as custom built\n  mock services to test atypical scenarios.\n- *Most verification tests can be parametrized without new code.* See\n  [test/assets/bundle-verify/README](test/assets/bundle-verify/README.md)\n\n## Usage\n\n1. Include an executable in your project that implements the\nclient-under-test [CLI protocol](docs/cli_protocol.md).\n2. Use the `sigstore/sigstore-conformance` action in your test workflow:\n    ```yaml\n    jobs:\n      conformance:\n        runs-on: ubuntu-latest\n        steps:\n          - uses: actions/checkout@v4\n\n          # insert your client installation steps here\n\n          # Run tests against production Sigstore environment\n          - uses: sigstore/sigstore-conformance@v0.0.25\n            with:\n              entrypoint: my-conformance-client\n\n          # Run tests against staging Sigstore environment\n          - uses: sigstore/sigstore-conformance@v0.0.25\n            with:\n              entrypoint: my-conformance-client\n              environment: staging\n    ```\n\nSee [selftest workflow](https://github.com/sigstore/sigstore-python/blob/main/.github/workflows/conformance.yml)\nfor a complete example.\n\n### `sigstore/sigstore-conformance` action inputs\n\nThe important action inputs are\n* `entrypoint`: required string. A command that implements the client-under-test\n  [CLI protocol](docs/cli_protocol.md)\n* `environment`: 'production' (default) or 'staging'. This selects the Sigstore environment to\n  run against\n* `xfail`: optional string. Whitespace separated test names that are expected to fail. Shell style\n  wild-cards can be used (e.g. `test_verify*intoto*`). Note that \"[\" used in some test names is\n  a wild card character that can be matched with e.g. \"[[]\".\n\nSee [action.yml](action.yml) for full list of inputs.\n\n### Configuring `xfail` for optional tests\n\n[[example]](https://github.com/sigstore/sigstore-conformance/blob/eae6eb1f59e25c6d3d602c5dad3dc55767c2f1cb/.github/workflows/conformance.yml#L44)\n\nDepending on a client's optional feature support, the following test patterns can be added to the `xfail` config\n- `test_verify*intoto*`: Older, deprecated intoto types that can be ignored for new clients.\n- `test_verify*managed-key-happy-path]`, `test_verify*managed-key-and-trusted-root]`: Tests that require clients to support managed keys\n\n\n## Development\n\nEasiest way to run the conformance suite locally is with the provided virtual environment:\n```sh\n$ make dev\n$ source env/bin/activate\n(env) $\n```\n\nThe test suite can be configured with\n* `--entrypoint=$SIGSTORE_CLIENT` where SIGSTORE_CLIENT is path to a script that implements the\n  [CLI specification](https://github.com/sigstore/sigstore-conformance/blob/main/docs/cli_protocol.md)\n* optional `--staging`: This instructs the test suite to run against Sigstore staging infrastructure\n* optional `--skip-signing`: Runs verification tests only\n* The environment variable `GHA_SIGSTORE_CONFORMANCE_XFAIL` can be used to\n  set expected failures\n\n```sh\n(env) $ # run all tests\n(env) $ pytest -v --entrypoint=$SIGSTORE_CLIENT\n(env) $ # run verification tests only\n(env) $ pytest -v --entrypoint=$SIGSTORE_CLIENT --skip-signing\n```\n\nFollowing example runs the test suite with the included selftest client script:\n```sh\n(env) $ # run all tests\n(env) $ GHA_SIGSTORE_CONFORMANCE_XFAIL=\"test_verify*-intoto-with-custom-trust-root] test_verify*managed-key-happy-path] test_verify*managed-key-and-trusted-root]\" \\\n    pytest -v --entrypoint=selftest-client\n...\n(env) $ # run single test\n(env) $ pytest -v --entrypoint=selftest-client -k test_verify[DIGEST-happy-path]\n...\n```\n\n## Licensing\n\n`sigstore-conformance` is licensed under the Apache 2.0 License.\n\n## Code of Conduct\n\nEveryone interacting with this project is expected to follow the\n[sigstore Code of Conduct](https://github.com/sigstore/.github/blob/main/CODE_OF_CONDUCT.md)\n\n## Security\n\nShould you discover any security issues, please refer to sigstore's [security\nprocess](https://github.com/sigstore/.github/blob/main/SECURITY.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsigstore%2Fsigstore-conformance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsigstore%2Fsigstore-conformance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsigstore%2Fsigstore-conformance/lists"}