https://github.com/deeppavlov/qa_metrics

Last synced: 4 months ago
JSON representation

README

## Count ODQA metrics

# Available metrics

* ranker em recall
* squad f1 v1
* squad em v1

# Usage

```bash
pip install -r requirements.txt
python count_metrics.py -n -ak
```

# Available datasets

* sddata_pull
* ru_squad

# Example
```bash
python count_metrics.py dataset.csv sddata_pull -n 1 5 -ak 'Answer Toloker 1'
```

The evaluation result is written to `metrics.log`

The model answers are written into `answer_top_n.csv` files, where `n` is the number of documents retrieved.