https://github.com/auraoneai/eval-conformance-suite
Executable rubric-spec v1 conformance checks and embeddable SVG badges.
https://github.com/auraoneai/eval-conformance-suite
ai-evaluation conformance evals rubric
Last synced: 17 days ago
JSON representation
Executable rubric-spec v1 conformance checks and embeddable SVG badges.
- Host: GitHub
- URL: https://github.com/auraoneai/eval-conformance-suite
- Owner: auraoneai
- License: mit
- Created: 2026-05-12T01:33:58.000Z (about 1 month ago)
- Default Branch: main
- Last Pushed: 2026-05-12T02:43:57.000Z (about 1 month ago)
- Last Synced: 2026-05-12T03:27:58.003Z (about 1 month ago)
- Topics: ai-evaluation, conformance, evals, rubric
- Language: Python
- Homepage: https://auraone.ai/open
- Size: 7.81 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
Awesome Lists containing this project
README
# eval-conformance-suite
Executable rubric-spec v1 conformance checks and reproducible SVG badges. Inspect AI is documented as the first external adapter target. Reference badges can be hosted at `auraone.ai/conformance/.svg`.
## Quickstart
```bash
pip install eval-conformance-suite rubric-spec
eval-conformance-suite run --against rubric_spec.adapters.inspect_ai
eval-conformance-suite badge --against inspect_ai
```
The CI suite runs all bundled `rubric-spec` adapters: Inspect AI, PromptFoo, DeepEval, LangSmith, and EvalKit.
## What This Is Not
Not a benchmark. It only checks schema and adapter compliance using synthetic fixtures.