https://github.com/auraoneai/contamination-audit
Local contamination checks for eval data overlap, hashes, and n-gram leakage.
https://github.com/auraoneai/contamination-audit
ai-evaluation data-contamination evals leakage
Last synced: 6 days ago
JSON representation
Local contamination checks for eval data overlap, hashes, and n-gram leakage.
- Host: GitHub
- URL: https://github.com/auraoneai/contamination-audit
- Owner: auraoneai
- License: mit
- Created: 2026-05-12T01:33:08.000Z (23 days ago)
- Default Branch: main
- Last Pushed: 2026-05-12T06:21:46.000Z (22 days ago)
- Last Synced: 2026-05-12T08:28:01.307Z (22 days ago)
- Topics: ai-evaluation, data-contamination, evals, leakage
- Language: Python
- Homepage: https://auraone.ai/open
- Size: 14.6 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
Awesome Lists containing this project
README
# contamination-audit
`contamination-audit` combines n-gram overlap, optional embedding similarity, canary matching, answer-pattern checks, and public-corpus hash matching.
## Quickstart
```bash
pip install contamination-audit
contamination-audit run --eval-data examples/eval.jsonl --corpora pile,c4,hf-mmlu
```
By default, embedding checks use a no-dependency lexical cosine fallback. To run semantic embedding checks locally:
```bash
pip install 'contamination-audit[embedding]'
contamination-audit run --eval-data examples/eval.jsonl --embedding-backend sentence-transformers --embedding-model all-MiniLM-L6-v2
```
## What This Is Not
Not proof of uncontaminated data; it is a code-only diagnostic. Examples are synthetic.