https://github.com/nilsreiter/iaa

command line stuff to calculate inter annotator agreement
https://github.com/nilsreiter/iaa

fleiss inter-annotator-agreement kappa

Last synced: 5 months ago
JSON representation

command line stuff to calculate inter annotator agreement

• Host: GitHub
• URL: https://github.com/nilsreiter/iaa
• Owner: nilsreiter
• Created: 2016-03-21T19:57:08.000Z (about 10 years ago)
• Default Branch: master
• Last Pushed: 2016-11-11T14:00:53.000Z (over 9 years ago)
• Last Synced: 2025-04-05T18:51:50.588Z (about 1 year ago)
• Topics: fleiss, inter-annotator-agreement, kappa
• Language: Awk
• Size: 5.86 KB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Inter-Annotator-Agreement

This repository contains code to calculate inter-annotator-agreement (Fleiss' Kappa at the moment) on the command line using awk.

## Fleiss' Kappa

Calculates [Fleiss' kappa](https://en.wikipedia.org/wiki/Fleiss%27_kappa)

```

$ awk -f kappa.awk FILENAME

```

Testing

```

$ awk -f kappa.awk test.csv

0.209931

```

`test.csv` contains the same data as on the Wikipedia page as an example.

## Accuracy

```

$ awk -f accuracy.awk FILENAME

```

Testing 

```

$ awk -f accuracy.awk test-accuracy.tsv

0.528571

```