https://github.com/tiger-ai-lab/tablecot
The code and data for paper "Large Language Models are few(1)-shot Table Reasoners" [EACL2023]
https://github.com/tiger-ai-lab/tablecot
Last synced: 11 months ago
JSON representation
The code and data for paper "Large Language Models are few(1)-shot Table Reasoners" [EACL2023]
- Host: GitHub
- URL: https://github.com/tiger-ai-lab/tablecot
- Owner: TIGER-AI-Lab
- Created: 2022-10-22T18:44:27.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-04-30T15:30:52.000Z (about 2 years ago)
- Last Synced: 2025-07-08T20:45:43.642Z (12 months ago)
- Language: Python
- Homepage:
- Size: 4.08 MB
- Stars: 47
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# TableCoT
The code and data used for EACL-2023 Paper [Large Language Models are few(1)-shot Table Reasoners](https://arxiv.org/abs/2210.06710)
## Preliminary
First, you need to specify your OPENAI_API_KEY, please find it in your account in https://openai.com/api/.
```
export OPENAI_KEY=[YOUR_KEY]
```
## For WikiTableQuestions
```
python prompt.py --start 0 --end 500
```
This will call Chain of Thoughts prompting to solve the 0-500 example in the test set of WikiTableQA. The output will be saved to output/response_..._s0_e500.json.
You can further call this following to extract the answers from the predictions.
```
cd outputs/
python postprocess_answer.py --inputs response_..._s0_e500.json
```
Finally, call this following to compute the final EM score.
```
python compute_scores.py --inputs response_..._s0_e500.json.processed
```
## For TabFact
```
python prompt.py --start 0 --end 500
```
This will call Chain of Thoughts prompting to solve the 0-500 example in the test set of WikiTableQA. The output will be saved to output/response_..._s0_e500.json. This will directly output the accuracy after it finishes.