https://github.com/gigio1023/lm-evaluation-harness-to-csv
Parsing json to csv for results of GPT-NeoX
https://github.com/gigio1023/lm-evaluation-harness-to-csv
Last synced: 2 months ago
JSON representation
Parsing json to csv for results of GPT-NeoX
- Host: GitHub
- URL: https://github.com/gigio1023/lm-evaluation-harness-to-csv
- Owner: gigio1023
- License: apache-2.0
- Created: 2022-10-04T08:04:00.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2022-10-06T03:21:29.000Z (over 2 years ago)
- Last Synced: 2025-03-12T17:43:43.234Z (3 months ago)
- Language: Python
- Homepage:
- Size: 63.5 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# lm-evaluation-harness-to-csv
Convert JSON files to CSV files to record notion table for using [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).
## Dependency
- pandas
## How to use
### Run script
The script will generate one csv file for specific num_few_shot, step.
```sh
python make_csv.py --model {model} --base {base_dir} --output {output_dir} --step {training_step} --few_shot {number_of_few_shot}
```
### Merge with CSV in Notion
Click the button at the table, and import csv file in table.
## Data hierarchy
origin json output files
```
- results
- metrics1
- acc or f1 or etc...
- metrics2
- metrics3
- metrics4
- versions
- config
```
csv file(round(4), macro-f1)
```
Task 0-shot 5-shot 10-shot 50-shot 1-shot
metric1 ## ## ## ## ##
metric2 ## ## ## ## ##
metric3 ## ## ## ## ##
metric4 ## ## ## ## ##
```