https://stanfordnlp.github.io/coqa/

CoQA -- A Conversational Question Answering Challenge
https://stanfordnlp.github.io/coqa/

Last synced: about 1 month ago
JSON representation

CoQA -- A Conversational Question Answering Challenge

• Host: GitHub
• URL: https://stanfordnlp.github.io/coqa/
• Owner: stanfordnlp
• Created: 2018-08-08T23:31:33.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2020-05-10T20:46:30.000Z (over 4 years ago)
• Last Synced: 2024-08-02T16:30:50.102Z (4 months ago)
• Language: Shell
• Homepage:
• Size: 47 MB
• Stars: 1
• Watchers: 7
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md

Awesome Lists containing this project

• Reading-Comprehension-Question-Answering-Papers - paper
• awesome-llm-eval - CoQA
• StarryDivineSky - stanfordnlp.github.io/coqa

README

        
# CoQA -- A Conversational Question Answering Challenge

The goal of [CoQA challenge](https://stanfordnlp.github.io/coqa) is to measure the ability of machines to read a passage and answer a series of interconnected questions that appear in  a conversation.

This repository hosts the code for generating the website and the associated data files.

The resulting web files are stored on the [gh-pages branch](https://github.com/stanfordnlp/coqa/tree/gh-pages).

# License

CoQA contains data from seven domains. Of these, only five are shared for public and the remaining two are private (for out-of-domain testing).

* Literature and Wikipedia domains are shared under [CC BY-SA 4.0](http://creativecommons.org/licenses/by-sa/4.0/legalcode) license. 

* Children's stories are collected from [MCTest](https://web.archive.org/web/20161026175200/http://research.microsoft.com/en-us/um/redmond/projects/mctest).

* Mid/High school exam passages are collected from [RACE](http://www.cs.cmu.edu/~glai1/data/race/)

* News articles are collected from [CNN dataset](https://cs.nyu.edu/~kcho/DMQA/).

# Commands

To automate sanity checks, run the following commands

```

# First create submissions/FILE.txt indicating the worksheet, cl run command, submission name and date, e.g. see submissions/BERT-Augmented-AoA.txt

python print-coqa-eval-commands.py < submissions/FILE.txt > submissions/FILE.sh

sh submissions/FILE.sh 

```

To see if the submission succeeded, run the following command

```

python populate_results.py < submissions/FILE.txt

```

To populate the leaderaboard, run the following command

```

cp out-v1.0.new.json out-v1.0.json

gulp

gulp connect

gulp deploy

```