https://github.com/kharvd/wnut17-llm-bench

https://github.com/kharvd/wnut17-llm-bench

Last synced: 16 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/kharvd/wnut17-llm-bench
• Owner: kharvd
• Created: 2024-05-20T03:45:20.000Z (7 months ago)
• Default Branch: main
• Last Pushed: 2024-05-20T03:51:57.000Z (7 months ago)
• Last Synced: 2024-11-09T04:07:43.423Z (about 2 months ago)
• Language: Python
• Size: 5.86 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
## Preprocessing

Dataset from https://noisy-text.github.io/2017/emerging-rare-entities.html

```bash

cat emerging.dev.conll | python ./process_dataset.py >| emerging.dev.jsonl

```

## Extracting entities

(Small subset for testing)

```bash

head -n100 ./emerging.dev.jsonl | COHERE_API_KEY= python run_cohere.py --num_threads 3 --out_file emerging.dev.predicted.command-r-plus.jsonl

```

## Evaluation

```bash

python ./eval.py emerging.dev.jsonl emerging.dev.predicted.command-r-plus.jsonl

```