https://github.com/THU-BPM/chatgpt-sql

The prediction results of ChatGPT on various datasets of Text-to-SQL.
https://github.com/THU-BPM/chatgpt-sql

Last synced: 2 months ago
JSON representation

The prediction results of ChatGPT on various datasets of Text-to-SQL.

• Host: GitHub
• URL: https://github.com/THU-BPM/chatgpt-sql
• Owner: THU-BPM
• Created: 2023-03-12T02:02:08.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2023-06-14T01:07:03.000Z (over 1 year ago)
• Last Synced: 2024-08-18T11:13:38.364Z (6 months ago)
• Language: Python
• Homepage: https://arxiv.org/pdf/2303.13547.pdf
• Size: 90.7 MB
• Stars: 96
• Watchers: 1
• Forks: 7
• Open Issues: 3
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• Awesome-Text2SQL - [code

README

        
# chatgpt-sql

The prediction results of ChatGPT on various datasets of Text-to-SQL.

For the specific evaluation methods and results, please refer to this paper "A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability" (https://arxiv.org/pdf/2303.13547.pdf). 

## Download dataset

You can find the datasets in the following links: 

* Spider dataset: https://drive.google.com/uc?export=download&id=1TqleXec_OykOYFREKKtschzY29dUcVAQ

* Spider-SYN dataset: https://github.com/ygan/Spider-Syn

* Spider-DK dataset: https://github.com/ygan/Spider-DK

* Spider-Realistic dataset: https://zenodo.org/record/5205322

* ADVETA: https://github.com/microsoft/ContextualSP

* Spider-CG: https://github.com/ygan/SpiderSS-SpiderCG

* Sparc: https://yale-lily.github.io/sparc

* CoSQL: https://yale-lily.github.io/cosql

* CSPIDER: https://taolusi.github.io/CSpider-explorer/

* DuSQL: https://github.com/luge-ai/luge-ai/tree/master/semantic-parsing

## Test example on spider dataset

### generate data 

```python

python generate_txt.py

```

### generate evaluation result (unzip the database.zip first)

```python

python evaluation.py --gold gold_example.txt --pred pred_example.txt --etype exec --db spider/database --table spider/tables.json

```

The result should be following:

![image](figure/result.png)