{"id":13595372,"url":"https://github.com/allenai/bi-att-flow","last_synced_at":"2025-05-15T17:01:15.861Z","repository":{"id":48099338,"uuid":"61758076","full_name":"allenai/bi-att-flow","owner":"allenai","description":"Bi-directional Attention Flow (BiDAF) network is a multi-stage hierarchical process that represents context at different levels of granularity and uses a bi-directional attention flow mechanism to achieve a query-aware context representation without early summarization.","archived":false,"fork":false,"pushed_at":"2023-05-31T01:34:05.000Z","size":97182,"stargazers_count":1536,"open_issues_count":74,"forks_count":681,"subscribers_count":103,"default_branch":"master","last_synced_at":"2025-04-07T21:15:19.033Z","etag":null,"topics":["bidaf","nlp","question-answering","squad","tensorflow"],"latest_commit_sha":null,"homepage":"http://allenai.github.io/bi-att-flow","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allenai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-06-22T23:21:38.000Z","updated_at":"2025-03-21T04:44:20.000Z","dependencies_parsed_at":"2023-10-20T16:42:40.815Z","dependency_job_id":null,"html_url":"https://github.com/allenai/bi-att-flow","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenai%2Fbi-att-flow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenai%2Fbi-att-flow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenai%2Fbi-att-flow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allenai%2Fbi-att-flow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allenai","download_url":"https://codeload.github.com/allenai/bi-att-flow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254384901,"owners_count":22062419,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bidaf","nlp","question-answering","squad","tensorflow"],"created_at":"2024-08-01T16:01:48.800Z","updated_at":"2025-05-15T17:01:15.489Z","avatar_url":"https://github.com/allenai.png","language":"Python","funding_links":[],"categories":["Python","Uncategorized","Codes"],"sub_categories":["Uncategorized","Most QA systems have roughly 3 parts"],"readme":"# Bi-directional Attention Flow for Machine Comprehension\n \n- This the original implementation of [Bi-directional Attention Flow for Machine Comprehension][paper].\n- The CodaLab worksheet for the [SQuAD Leaderboard][squad] submission is available [here][worksheet].\n- For TensorFlow v1.2 compatible version, see the [dev][dev] branch.\n- Please contact [Minjoon Seo][minjoon] ([@seominjoon][minjoon-github]) for questions and suggestions.\n\n## 0. Requirements\n#### General\n- Python (verified on 3.5.2. Issues have been reported with Python 2!)\n- unzip, wget (for running `download.sh` only)\n\n#### Python Packages\n- tensorflow (deep learning library, only works on r0.11)\n- nltk (NLP tools, verified on 3.2.1)\n- tqdm (progress bar, verified on 4.7.4)\n- jinja2 (for visaulization; if you only train and test, not needed)\n\n## 1. Pre-processing\nFirst, prepare data. Donwload SQuAD data and GloVe and nltk corpus\n(~850 MB, this will download files to `$HOME/data`):\n```\nchmod +x download.sh; ./download.sh\n```\n\nSecond, Preprocess Stanford QA dataset (along with GloVe vectors) and save them in `$PWD/data/squad` (~5 minutes):\n```\npython -m squad.prepro\n```\n\n## 2. Training\nThe model has ~2.5M parameters.\nThe model was trained with NVidia Titan X (Pascal Architecture, 2016).\nThe model requires at least 12GB of GPU RAM.\nIf your GPU RAM is smaller than 12GB, you can either decrease batch size (performance might degrade),\nor you can use multi GPU (see below).\nThe training converges at ~18k steps, and it took ~4s per step (i.e. ~20 hours).\n\nBefore training, it is recommended to first try the following code to verify everything is okay and memory is sufficient:\n```\npython -m basic.cli --mode train --noload --debug\n```\n\nThen to fully train, run:\n```\npython -m basic.cli --mode train --noload\n```\n\nYou can speed up the training process with optimization flags:\n```\npython -m basic.cli --mode train --noload --len_opt --cluster\n```\nYou can still omit them, but training will be much slower.\n\nNote that during the training, the EM and F1 scores from the occasional evaluation are not the same with the score from official squad evaluation script. \nThe printed scores are not official (our scoring scheme is a bit harsher).\nTo obtain the official number, use the official evaluator (copied in `squad` folder, `squad/evaluate-v1.1.py`). For more information See 3.Test.\n\n\n## 3. Test\nTo test, run:\n```\npython -m basic.cli\n```\n\nSimilarly to training, you can give the optimization flags to speed up test (5 minutes on dev data):\n```\npython -m basic.cli --len_opt --cluster\n```\n\nThis command loads the most recently saved model during training and begins testing on the test data.\nAfter the process ends, it prints F1 and EM scores, and also outputs a json file (`$PWD/out/basic/00/answer/test-####.json`,\nwhere `####` is the step # that the model was saved).\nNote that the printed scores are not official (our scoring scheme is a bit harsher).\nTo obtain the official number, use the official evaluator (copied in `squad` folder) and the output json file:\n\n```\npython squad/evaluate-v1.1.py $HOME/data/squad/dev-v1.1.json out/basic/00/answer/test-####.json\n```\n\n### 3.1 Loading from pre-trained weights\nInstead of training the model yourself, you can choose to use pre-trained weights that were used for [SQuAD Leaderboard][squad] submission.\nRefer to [this worksheet][worksheet] in CodaLab to reproduce the results.\nIf you are unfamiliar with CodaLab, follow these simple steps (given that you met all prereqs above):\n\n1. Download `save.zip` from the [worksheet][worksheet] and unzip it in the current directory.\n2. Copy `glove.6B.100d.txt` from your glove data folder (`$HOME/data/glove/`) to the current directory.\n3. To reproduce single model:\n  \n  ```\n  basic/run_single.sh $HOME/data/squad/dev-v1.1.json single.json\n  ```\n  \n  This writes the answers to `single.json` in the current directory. You can then use the official evaluator to obtain EM and F1 scores. If you want to run on GPU (~5 mins), change the value of batch_size flag in the shell file to a higher number (60 for 12GB GPU RAM). \n4. Similarly, to reproduce ensemble method:\n  \n  ```\n  basic/run_ensemble.sh $HOME/data/squad/dev-v1.1.json ensemble.json \n  ```\n  If you want to run on GPU, you should run the script sequentially by removing '\u0026' in the forloop, or you will need to specify different GPUs for each run of the for loop.\n\n## Results\n\n### Dev Data\n\nNote these scores are from the official evaluator (copied in `squad` folder, `squad/evaluate-v1.1.py`). For more information See 3.Test.\nThe scores appeared during the training could be lower than the scores from the official evaluator. \n\n|          | EM (%) | F1 (%) |\n| -------- |:------:|:------:|\n| single   | 67.7   | 77.3   |\n| ensemble | 72.6   | 80.7   |\n\n### Test Data\n\n|          | EM (%) | F1 (%) |\n| -------- |:------:|:------:|\n| single   | 68.0   | 77.3   |\n| ensemble | 73.3   | 81.1   |\n\nRefer to [our paper][paper] for more details.\nSee [SQuAD Leaderboard][squad] to compare with other models.\n\n\n\u003c!--\n## Using Pre-trained Model\n\nIf you would like to use pre-trained model, it's very easy! \nYou can download the model weights [here][save] (make sure that its commit id matches the source code's).\nExtract them and put them in `$PWD/out/basic/00/save` directory, with names unchanged.\nThen do the testing again, but you need to specify the step # that you are loading from:\n```\npython -m basic.cli --mode test --batch_size 8 --eval_num_batches 0 --load_step ####\n```\n--\u003e\n\n\n## Multi-GPU Training \u0026 Testing\nOur model supports multi-GPU training.\nWe follow the parallelization paradigm described in [TensorFlow Tutorial][multi-gpu].\nIn short, if you want to use batch size of 60 (default) but if you have 3 GPUs with 4GB of RAM,\nthen you initialize each GPU with batch size of 20, and combine the gradients on CPU.\nThis can be easily done by running:\n```\npython -m basic.cli --mode train --noload --num_gpus 3 --batch_size 20\n```\n\nSimilarly, you can speed up your testing by:\n```\npython -m basic.cli --num_gpus 3 --batch_size 20 \n```\n\n## Demo\nFor now, please refer to the `demo` branch of this repository.\n \n\n[multi-gpu]: https://www.tensorflow.org/versions/r0.11/tutorials/deep_cnn/index.html#training-a-model-using-multiple-gpu-cards\n[squad]: http://stanford-qa.com\n[paper]: https://arxiv.org/abs/1611.01603\n[worksheet]: https://worksheets.codalab.org/worksheets/0x37a9b8c44f6845c28866267ef941c89d/\n[minjoon]: https://seominjoon.github.io\n[minjoon-github]: https://github.com/seominjoon\n[dev]: https://github.com/allenai/bi-att-flow/tree/dev\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallenai%2Fbi-att-flow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallenai%2Fbi-att-flow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallenai%2Fbi-att-flow/lists"}