https://github.com/auraoneai/iaa-kit
Modern inter-annotator agreement metrics with bootstrap intervals, ordinal support, and missing-data handling.
https://github.com/auraoneai/iaa-kit
ai-evaluation evals inter-annotator-agreement statistics
Last synced: 7 days ago
JSON representation
Modern inter-annotator agreement metrics with bootstrap intervals, ordinal support, and missing-data handling.
- Host: GitHub
- URL: https://github.com/auraoneai/iaa-kit
- Owner: auraoneai
- License: mit
- Created: 2026-05-12T01:32:27.000Z (23 days ago)
- Default Branch: main
- Last Pushed: 2026-05-12T05:08:48.000Z (23 days ago)
- Last Synced: 2026-05-12T07:05:19.607Z (23 days ago)
- Topics: ai-evaluation, evals, inter-annotator-agreement, statistics
- Language: Python
- Homepage: https://auraone.ai/open
- Size: 14.6 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
Awesome Lists containing this project
README
# iaa-kit
`iaa-kit` is a pure-NumPy inter-annotator agreement library with Krippendorff alpha, Cohen kappa, weighted kappa, Fleiss kappa, Gwet AC1/AC2, bootstrap confidence intervals, missing-data handling, and synthetic-report generation.
## Quickstart
```bash
python -m venv .venv
. .venv/bin/activate
pip install iaa-kit
python -c "from iaa_kit import krippendorff_alpha; print(krippendorff_alpha([[1,1,2],[2,2,2]], scale='ordinal'))"
```
## What This Is Not
This is not a human-label vendor, benchmark, or source of real annotations. Examples are synthetic.