https://github.com/zphang/lm_perplexity
https://github.com/zphang/lm_perplexity
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/zphang/lm_perplexity
- Owner: zphang
- Created: 2020-11-13T02:43:05.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2020-12-21T10:06:15.000Z (over 4 years ago)
- Last Synced: 2025-01-29T13:43:36.961Z (4 months ago)
- Language: Python
- Size: 23.4 KB
- Stars: 0
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# lm_perplexity
## Usage
Evaluating on GPT-2 (uses GPU):
```bash
python lm_perplexity/run_lm_perplexity.py \
--model_config_path preset_configs/gpt2_medium.json \
--data_path /path/to/mydata.jsonl.zst
```
Evaluating on GPT-3 (requires OpenAI API key):
```bash
export OPENAI_API_SECRET_KEY=YOUR_KEY_HERE
python lm_perplexity/run_lm_perplexity.py \
--model_config_path preset_configs/gpt3_curie.json \
--data_path /path/to/mydata.jsonl.zst
```
Additional Options:
* `--max_docs n`: Only evaluate the first `n` docs. Use this for testing.
* `--output_path /path/to/results.json`: Write results to JSON file.
## Requirements
* torch
* transformers
* openai
* lm_dataformat
* tqdm