{"id":29040078,"url":"https://github.com/StonyBrookNLP/musique","last_synced_at":"2025-06-26T14:05:20.905Z","repository":{"id":76842411,"uuid":"391796616","full_name":"StonyBrookNLP/musique","owner":"StonyBrookNLP","description":"Repository for MuSiQue: Multi-hop Questions via Single-hop Question Composition, TACL 2022","archived":false,"fork":false,"pushed_at":"2024-06-12T19:40:56.000Z","size":1331,"stargazers_count":67,"open_issues_count":2,"forks_count":7,"subscribers_count":11,"default_branch":"main","last_synced_at":"2024-06-14T01:59:30.979Z","etag":null,"topics":["dataset","multihop-reasoning","nlu","question-answering","reading-comprehension"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc-by-4.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/StonyBrookNLP.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-02T02:44:28.000Z","updated_at":"2024-06-13T07:22:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"25a2a964-4770-4825-9aa5-b28362a9162b","html_url":"https://github.com/StonyBrookNLP/musique","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/StonyBrookNLP/musique","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StonyBrookNLP%2Fmusique","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StonyBrookNLP%2Fmusique/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StonyBrookNLP%2Fmusique/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StonyBrookNLP%2Fmusique/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/StonyBrookNLP","download_url":"https://codeload.github.com/StonyBrookNLP/musique/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StonyBrookNLP%2Fmusique/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262081083,"owners_count":23255661,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataset","multihop-reasoning","nlu","question-answering","reading-comprehension"],"created_at":"2025-06-26T14:03:06.235Z","updated_at":"2025-06-26T14:05:20.884Z","avatar_url":"https://github.com/StonyBrookNLP.png","language":"Python","funding_links":[],"categories":["Datasets","Python"],"sub_categories":["Training-Free Approaches"],"readme":"# \u003ch2 align=\"center\"\u003e :musical_note: MuSiQue: Multi-hop Questions via Single-hop Question Composition \u003c/h2\u003e\n\nRepository for our TACL 2022 paper \"[MuSiQue: Multi-hop Questions via Single-hop Question Composition](https://arxiv.org/pdf/2108.00573.pdf)\"\n\n# Data\n\nMuSiQue is distributed under a [CC BY 4.0 License](https://creativecommons.org/licenses/by/4.0/).\n\n**Usage Caution:** If you're using any of our seed single-hop datasets ([SQuAD](https://arxiv.org/abs/1606.05250), [T-REx](https://hadyelsahar.github.io/t-rex/paper.pdf), [Natural Questions](https://storage.googleapis.com/pub-tools-public-publication-data/pdf/1f7b46b5378d757553d3e92ead36bda2e4254244.pdf), [MLQA](https://arxiv.org/pdf/1910.07475.pdf), [Zero Shot RE](https://arxiv.org/pdf/1706.04115.pdf)) in any way (e.g., pretraining on them), please note that MuSiQue was created by composing questions from these seed datasets. Therefore, single-hop questions used in MuSiQue's dev/test sets may occur in the training sets of these seed datasets. To help avoid information leakage, we are releasing the IDs of single-hop questions that are used in MuSiQue dev/test sets. Once you download the data below, these IDs and corresponding questions will be in `data/dev_test_singlehop_questions_v1.0.json`. If you use our seed single-hop datasets in any way in your model, please be sure to **avoid using any single-hop question IDs present in this file**\n\nTo download MuSiQue, either run the following script or download it manually from [here](https://drive.google.com/file/d/1tGdADlNjWFaHLeZZGShh2IRcpO6Lv24h/view?usp=sharing).\n\n```\nbash download_data.sh\n```\n\nThe result will be stored in `data/` directory. It contains (i) train, dev and test sets of `MuSiQue-Ans` and `MuSiQue-Full`, (ii) single-hop questions and ids from source datasets (squad, natural questions, trex, mlqa, zerore) that are part of dev or test of MuSiQue.\n\n\n# Predictions\n\nWe're releasing the model predictions (in official format) for 4 models on dev sets of `MuSiQue-Ans` and `MuSiQue-Full`. To get it, you can run the following script or download it manually from [here](https://drive.google.com/file/d/1XZocqLOTAu4y_1EeAj1JM4Xc1JxGJtx6/view?usp=sharing).\n\n```\nbash download_predictions.sh\n```\n\n\n# Evaluation\n\nYou can use `evaluate_v1.0.py` to evaluate your predictions against ground-truths. For eg.:\n\n```\npython evaluate_v1.0.py predictions/musique_ans_v1.0_dev_end2end_model_predictions.jsonl data/musique_ans_v1.0_dev.jsonl\n```\n\nThese are the results you would get for MuSiQue-Answerable and MuSiQue-Full validation sets and for each of the four models (End2End Model, Select+Answer Model, Execution by End2End Model, Execution by Select+Answer Model).\n\n```bash\n# MuSiQue-Answerable\npython evaluate_v1.0.py predictions/musique_ans_v1.0_dev_end2end_model_predictions.jsonl data/musique_ans_v1.0_dev.jsonl\n# =\u003e {\"answer_f1\": 0.423, \"support_f1\": 0.676}\n\npython evaluate_v1.0.py predictions/musique_ans_v1.0_dev_select_answer_model_predictions.jsonl data/musique_ans_v1.0_dev.jsonl\n# =\u003e {\"answer_f1\": 0.473, \"support_f1\": 0.723}\n\npython evaluate_v1.0.py predictions/musique_ans_v1.0_dev_step_execution_by_end2end_model_predictions.jsonl data/musique_ans_v1.0_dev.jsonl\n# =\u003e {\"answer_f1\": 0.456, \"support_f1\": 0.778}\n\npython evaluate_v1.0.py predictions/musique_ans_v1.0_dev_step_execution_by_select_answer_model_predictions.jsonl data/musique_ans_v1.0_dev.jsonl\n# =\u003e {\"answer_f1\": 0.497, \"support_f1\": 0.792}\n\n# MuSiQue-Full\npython evaluate_v1.0.py predictions/musique_full_v1.0_dev_end2end_model_predictions.jsonl data/musique_full_v1.0_dev.jsonl\n# =\u003e {\"answer_f1\": 0.406, \"support_f1\": 0.325, \"group_answer_sufficiency_f1\": 0.22, \"group_support_sufficiency_f1\": 0.252}\n\npython evaluate_v1.0.py predictions/musique_full_v1.0_dev_select_answer_model_predictions.jsonl data/musique_full_v1.0_dev.jsonl\n# =\u003e {\"answer_f1\": 0.486, \"support_f1\": 0.522, \"group_answer_sufficiency_f1\": 0.344, \"group_support_sufficiency_f1\": 0.42}\n\npython evaluate_v1.0.py predictions/musique_full_v1.0_dev_step_execution_by_end2end_model_predictions.jsonl data/musique_full_v1.0_dev.jsonl\n# =\u003e {\"answer_f1\": 0.463, \"support_f1\": 0.75, \"group_answer_sufficiency_f1\": 0.321, \"group_support_sufficiency_f1\": 0.447}\n\npython evaluate_v1.0.py predictions/musique_full_v1.0_dev_step_execution_by_select_answer_model_predictions.jsonl data/musique_full_v1.0_dev.jsonl\n# =\u003e {\"answer_f1\": 0.498, \"support_f1\": 0.777, \"group_answer_sufficiency_f1\": 0.328, \"group_support_sufficiency_f1\": 0.431}\n```\n\n# Leaderboard\n\nWe've two leaderboards for MuSiQue: [MuSiQue-Answerable](https://leaderboard.allenai.org/musique_ans) and [MuSiQue-Full](https://leaderboard.allenai.org/musique_full).\n\nOnce you've the test set predictions in the official format, it's just about uploading the files in the above leadboards! Feel free to contact me (Harsh) in case you've any questions.\n\n\n# Models and Experiments\n\nWe've relased the code that we used for experiments in the paper. If you're interested in trying our trained models, training them from sratch, viewing their predictions or generating their predictions from your trained model, follow the steps below. \n\n## Installations\n\n```bash\n\n# Set env.\nconda create -n musique python=3.8 -y \u0026\u0026 conda activate musique\n\n# Set allennlp in root directory\ngit clone https://github.com/allenai/allennlp\ncd allennlp\ngit checkout v2.1.0\ngit apply ../allennlp.diff # small diff to get longformer global attention to work correctly.\ncd ..\n\npip install allennlp==2.1.0 # we only need dependencies of allennlp\npip uninstall -y allennlp\n\npip install gdown==v4.5.1\npython -m nltk.downloader stopwords\n\npip uninstall -y transformers\npip install transformers==4.7.0 # we used this version of transformers\n```\n\n## Download Raw Data\n\nOur models were developed using a different (non-official) format of the dataset files. So to run our code, you'll first need to download the dataset files in the raw format. \n\n```bash\npython download_raw_data.py\n```\n\nNote that officially released data and what we've used here are only different in the format (e.g. uses different names for json fields), and are not qualitatively different. Take a look at `raw_data_to_official_format.py` if you're interested.\n\n-----\n\nWe've done experiments on 4 datasets (MuSiQue-Ans, MuSiQue-Full, HotpotQA-20K, 2WikiMultihopQA-20K) with 4 multihop models (End2End Model, Select+Answer Model, Execution by End2End Model, Execution by Select+Answer Model) where possible. See Table 1. You can explore each combination using the instruction toggle below.\n\nFor each combination, you'll see instructions on how (i) download trained model (ii) train a model from scratch\n(iii) download model prediction/s (iv) generate predictions with a trained or a downloaded model.\n\nOur models are implemented in `allennlp`. If you're familiar with it, using the code should be pretty straightforward. The only difference is that instead of using `allennlp` command, we're using `run.py` as an entrypoint, which mainly loads `allennlp_lib` to load our allennlp code (readers, models, predictors, etc).\n\n\n\n\n## MuSiQue-Answerable\n\n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eEnd2End Model [EE]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\n----\n\n### Experiment Name\n\n```bash\nend2end_model_for_musique_ans_dataset\n```\n\n### Download model\n\n```bash\npython download_models.py end2end_model_for_musique_ans_dataset\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/end2end_model_for_musique_ans_dataset.jsonnet \\\n                    --serialization-dir serialization_dir/end2end_model_for_musique_ans_dataset\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py end2end_model_for_musique_ans_dataset\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/end2end_model_for_musique_ans_dataset/model.tar.gz \\\n                      raw_data/musique_ans_dev.jsonl \\\n                      --output-file serialization_dir/end2end_model_for_musique_ans_dataset/predictions/musique_ans_dev.jsonl \\\n                      --predictor transformer_rc --batch-size 16 --cuda-device 0 --silent\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/end2end_model_for_musique_ans_dataset/predictions/musique_ans_dev.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eSelect+Answer Model [SA]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\u003cbr\u003eThe system has 2 parts given below: (i) Selector Model (ii) Answerer Model\n\n\n----\n\n### Experiment Name\n\n```bash\n# Selector Model\nselect_and_answer_model_selector_for_musique_ans\n```\n\n### Download model\n\n```bash\npython download_models.py select_and_answer_model_selector_for_musique_ans\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/select_and_answer_model_selector_for_musique_ans.jsonnet \\\n                    --serialization-dir serialization_dir/select_and_answer_model_selector_for_musique_ans\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py select_and_answer_model_selector_for_musique_ans\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/select_and_answer_model_selector_for_musique_ans/model.tar.gz \\\n                      raw_data/musique_ans_train.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_selector_for_musique_ans/predictions/musique_ans_train.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\npython run.py predict serialization_dir/select_and_answer_model_selector_for_musique_ans/model.tar.gz \\\n                      raw_data/musique_ans_dev.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_selector_for_musique_ans/predictions/musique_ans_dev.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Answerer Model\nselect_and_answer_model_answerer_for_musique_ans\n```\n\n### Download model\n\n```bash\npython download_models.py select_and_answer_model_answerer_for_musique_ans\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/select_and_answer_model_answerer_for_musique_ans.jsonnet \\\n                    --serialization-dir serialization_dir/select_and_answer_model_answerer_for_musique_ans\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py select_and_answer_model_answerer_for_musique_ans\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/select_and_answer_model_answerer_for_musique_ans/model.tar.gz \\\n                      serialization_dir/select_and_answer_model_selector_for_musique_ans/predictions/musique_ans_dev.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_answerer_for_musique_ans/predictions/serialization_dir__select_and_answer_model_selector_for_musique_ans__predictions__musique_ans_dev.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/select_and_answer_model_answerer_for_musique_ans/predictions/serialization_dir__select_and_answer_model_selector_for_musique_ans__predictions__musique_ans_dev.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eExecution by End2End Model [EX(EE)]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\u003cbr\u003eThe system has 2 parts given below: (i) Decomposer Model (ii) Executor Model.\n\n\n----\n\n### Experiment Name\n\n```bash\n# Decomposer Model\nexecution_model_decomposer_for_musique_ans_and_full\n```\n\n### Download model\n\n```bash\npython download_models.py execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_model_decomposer_for_musique_ans_and_full.jsonnet \\\n                    --serialization-dir serialization_dir/execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_model_decomposer_for_musique_ans_and_full/model.tar.gz \\\n                      raw_data/musique_ans_dev.jsonl \\\n                      --output-file serialization_dir/execution_model_decomposer_for_musique_ans_and_full/predictions/musique_ans_dev.jsonl \\\n                      --predictor question_translator --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Executor Model\nexecution_by_end2end_model_for_musique_ans\n```\n\n### Download model\n\n```bash\npython download_models.py execution_by_end2end_model_for_musique_ans\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_by_end2end_model_for_musique_ans.jsonnet \\\n                    --serialization-dir serialization_dir/execution_by_end2end_model_for_musique_ans\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_by_end2end_model_for_musique_ans\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_by_end2end_model_for_musique_ans/model.tar.gz \\\n                      serialization_dir/execution_model_decomposer_for_musique_ans_and_full/predictions/musique_ans_dev.jsonl \\\n                      --output-file serialization_dir/execution_by_end2end_model_for_musique_ans/predictions/serialization_dir__execution_model_decomposer_for_musique_ans_and_full__predictions__musique_ans_dev.jsonl \\\n                      --predictor multi_step_end2end_transformer_rc --batch-size 16 --cuda-device 0 --silent \\\n                      --predictor-args '{\"predict_answerability\":false,\"skip_distractor_paragraphs\":false,\"use_predicted_decomposition\":true}'\n\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/execution_by_end2end_model_for_musique_ans/predictions/serialization_dir__execution_model_decomposer_for_musique_ans_and_full__predictions__musique_ans_dev.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eExecution by Select+Answer Model [EX(SA)]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\u003cbr\u003eThe system has 3 parts given below: (i) Decomposer Model (ii) Selector of Executor Model (iii) Answerer of Executor Model.\n\n\n----\n\n### Experiment Name\n\n```bash\n# Decomposer Model\nexecution_model_decomposer_for_musique_ans_and_full\n```\n\n### Download model\n\n```bash\npython download_models.py execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_model_decomposer_for_musique_ans_and_full.jsonnet \\\n                    --serialization-dir serialization_dir/execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_model_decomposer_for_musique_ans_and_full/model.tar.gz \\\n                      raw_data/musique_ans_dev.jsonl \\\n                      --output-file serialization_dir/execution_model_decomposer_for_musique_ans_and_full/predictions/musique_ans_dev.jsonl \\\n                      --predictor question_translator --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Selector of Executor Model\nexecution_by_select_and_answer_model_selector_for_musique_ans\n```\n\n### Download model\n\n```bash\npython download_models.py execution_by_select_and_answer_model_selector_for_musique_ans\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_by_select_and_answer_model_selector_for_musique_ans.jsonnet \\\n                    --serialization-dir serialization_dir/execution_by_select_and_answer_model_selector_for_musique_ans\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_by_select_and_answer_model_selector_for_musique_ans\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_by_select_and_answer_model_selector_for_musique_ans/model.tar.gz \\\n                      raw_data/musique_ans_single_hop_version_train.jsonl \\\n                      --output-file serialization_dir/execution_by_select_and_answer_model_selector_for_musique_ans/predictions/musique_ans_single_hop_version_train.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\npython run.py predict serialization_dir/execution_by_select_and_answer_model_selector_for_musique_ans/model.tar.gz \\\n                      raw_data/musique_ans_single_hop_version_dev.jsonl \\\n                      --output-file serialization_dir/execution_by_select_and_answer_model_selector_for_musique_ans/predictions/musique_ans_single_hop_version_dev.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Answerer of Executor Model\nexecution_by_select_and_answer_model_answerer_for_musique_ans\n```\n\n### Download model\n\n```bash\npython download_models.py execution_by_select_and_answer_model_answerer_for_musique_ans\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_by_select_and_answer_model_answerer_for_musique_ans.jsonnet \\\n                    --serialization-dir serialization_dir/execution_by_select_and_answer_model_answerer_for_musique_ans\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_by_select_and_answer_model_answerer_for_musique_ans\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_by_select_and_answer_model_answerer_for_musique_ans/model.tar.gz \\\n                      serialization_dir/execution_model_decomposer_for_musique_ans_and_full/predictions/musique_ans_dev.jsonl \\\n                      --output-file serialization_dir/execution_by_select_and_answer_model_answerer_for_musique_ans/predictions/serialization_dir__execution_model_decomposer_for_musique_ans_and_full__predictions__musique_ans_dev.jsonl \\\n                      --predictor multi_step_select_and_answer_transformer_rc --batch-size 16 --cuda-device 0 --silent \\\n                      --predictor-args '{\"predict_answerability\":false,\"skip_distractor_paragraphs\":false,\"use_predicted_decomposition\":true,\"selector_model_path\":\"serialization_dir/execution_by_select_and_answer_model_selector_for_musique_ans/model.tar.gz\",\"num_select\":3}'\n\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/execution_by_select_and_answer_model_answerer_for_musique_ans/predictions/serialization_dir__execution_model_decomposer_for_musique_ans_and_full__predictions__musique_ans_dev.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\n## MuSiQue-Full\n\n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eEnd2End Model [EE]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\n----\n\n### Experiment Name\n\n```bash\nend2end_model_for_musique_full_dataset\n```\n\n### Download model\n\n```bash\npython download_models.py end2end_model_for_musique_full_dataset\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/end2end_model_for_musique_full_dataset.jsonnet \\\n                    --serialization-dir serialization_dir/end2end_model_for_musique_full_dataset\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py end2end_model_for_musique_full_dataset\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/end2end_model_for_musique_full_dataset/model.tar.gz \\\n                      raw_data/musique_full_dev.jsonl \\\n                      --output-file serialization_dir/end2end_model_for_musique_full_dataset/predictions/musique_full_dev.jsonl \\\n                      --predictor transformer_rc --batch-size 16 --cuda-device 0 --silent\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/end2end_model_for_musique_full_dataset/predictions/musique_full_dev.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eSelect+Answer Model [SA]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\u003cbr\u003eThe system has 2 parts given below: (i) Selector Model (ii) Answerer Model.\n\n\n----\n\n### Experiment Name\n\n```bash\n# Selector Model\nselect_and_answer_model_selector_for_musique_full\n```\n\n### Download model\n\n```bash\npython download_models.py select_and_answer_model_selector_for_musique_full\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/select_and_answer_model_selector_for_musique_full.jsonnet \\\n                    --serialization-dir serialization_dir/select_and_answer_model_selector_for_musique_full\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py select_and_answer_model_selector_for_musique_full\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/select_and_answer_model_selector_for_musique_full/model.tar.gz \\\n                      raw_data/musique_full_train.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_selector_for_musique_full/predictions/musique_full_train.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\npython run.py predict serialization_dir/select_and_answer_model_selector_for_musique_full/model.tar.gz \\\n                      raw_data/musique_full_dev.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_selector_for_musique_full/predictions/musique_full_dev.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Answerer Model\nselect_and_answer_model_answerer_for_musique_full\n```\n\n### Download model\n\n```bash\npython download_models.py select_and_answer_model_answerer_for_musique_full\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/select_and_answer_model_answerer_for_musique_full.jsonnet \\\n                    --serialization-dir serialization_dir/select_and_answer_model_answerer_for_musique_full\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py select_and_answer_model_answerer_for_musique_full\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/select_and_answer_model_answerer_for_musique_full/model.tar.gz \\\n                      serialization_dir/select_and_answer_model_selector_for_musique_full/predictions/musique_full_dev.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_answerer_for_musique_full/predictions/serialization_dir__select_and_answer_model_selector_for_musique_full__predictions__musique_full_dev.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/select_and_answer_model_answerer_for_musique_full/predictions/serialization_dir__select_and_answer_model_selector_for_musique_full__predictions__musique_full_dev.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eExecution by End2End Model [EX(EE)]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\u003cbr\u003eThe system has 2 parts given below: (i) Decomposer Model (ii) Executor Model.\n\n\n----\n\n### Experiment Name\n\n```bash\n# Decomposer Model\nexecution_model_decomposer_for_musique_ans_and_full\n```\n\n### Download model\n\n```bash\npython download_models.py execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_model_decomposer_for_musique_ans_and_full.jsonnet \\\n                    --serialization-dir serialization_dir/execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_model_decomposer_for_musique_ans_and_full/model.tar.gz \\\n                      raw_data/musique_full_dev.jsonl \\\n                      --output-file serialization_dir/execution_model_decomposer_for_musique_ans_and_full/predictions/musique_full_dev.jsonl \\\n                      --predictor question_translator --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Executor Model\nexecution_by_end2end_model_for_musique_full\n```\n\n### Download model\n\n```bash\npython download_models.py execution_by_end2end_model_for_musique_full\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_by_end2end_model_for_musique_full.jsonnet \\\n                    --serialization-dir serialization_dir/execution_by_end2end_model_for_musique_full\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_by_end2end_model_for_musique_full\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_by_end2end_model_for_musique_full/model.tar.gz \\\n                      serialization_dir/execution_model_decomposer_for_musique_ans_and_full/predictions/musique_full_dev.jsonl \\\n                      --output-file serialization_dir/execution_by_end2end_model_for_musique_full/predictions/serialization_dir__execution_model_decomposer_for_musique_ans_and_full__predictions__musique_full_dev.jsonl \\\n                      --predictor multi_step_end2end_transformer_rc --batch-size 16 --cuda-device 0 --silent \\\n                      --predictor-args '{\"predict_answerability\":true,\"skip_distractor_paragraphs\":false,\"use_predicted_decomposition\":true}'\n\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/execution_by_end2end_model_for_musique_full/predictions/serialization_dir__execution_model_decomposer_for_musique_ans_and_full__predictions__musique_full_dev.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eExecution by Select+Answer Model [EX(SA)]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\u003cbr\u003eThe system has 3 parts given below: (i) Decomposer Model (ii) Selector of Executor Model (iii) Answerer of Executor Model.\n\n\n----\n\n### Experiment Name\n\n```bash\n# Decomposer Model\nexecution_model_decomposer_for_musique_ans_and_full\n```\n\n### Download model\n\n```bash\npython download_models.py execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_model_decomposer_for_musique_ans_and_full.jsonnet \\\n                    --serialization-dir serialization_dir/execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_model_decomposer_for_musique_ans_and_full\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_model_decomposer_for_musique_ans_and_full/model.tar.gz \\\n                      raw_data/musique_full_dev.jsonl \\\n                      --output-file serialization_dir/execution_model_decomposer_for_musique_ans_and_full/predictions/musique_full_dev.jsonl \\\n                      --predictor question_translator --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Selector of Executor Model\nexecution_by_select_and_answer_model_selector_for_musique_full\n```\n\n### Download model\n\n```bash\npython download_models.py execution_by_select_and_answer_model_selector_for_musique_full\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_by_select_and_answer_model_selector_for_musique_full.jsonnet \\\n                    --serialization-dir serialization_dir/execution_by_select_and_answer_model_selector_for_musique_full\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_by_select_and_answer_model_selector_for_musique_full\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_by_select_and_answer_model_selector_for_musique_full/model.tar.gz \\\n                      raw_data/musique_full_single_hop_version_train.jsonl \\\n                      --output-file serialization_dir/execution_by_select_and_answer_model_selector_for_musique_full/predictions/musique_full_single_hop_version_train.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\npython run.py predict serialization_dir/execution_by_select_and_answer_model_selector_for_musique_full/model.tar.gz \\\n                      raw_data/musique_full_single_hop_version_dev.jsonl \\\n                      --output-file serialization_dir/execution_by_select_and_answer_model_selector_for_musique_full/predictions/musique_full_single_hop_version_dev.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Answerer of Executor Model\nexecution_by_select_and_answer_model_answerer_for_musique_full\n```\n\n### Download model\n\n```bash\npython download_models.py execution_by_select_and_answer_model_answerer_for_musique_full\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_by_select_and_answer_model_answerer_for_musique_full.jsonnet \\\n                    --serialization-dir serialization_dir/execution_by_select_and_answer_model_answerer_for_musique_full\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_by_select_and_answer_model_answerer_for_musique_full\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_by_select_and_answer_model_answerer_for_musique_full/model.tar.gz \\\n                      serialization_dir/execution_model_decomposer_for_musique_ans_and_full/predictions/musique_full_dev.jsonl \\\n                      --output-file serialization_dir/execution_by_select_and_answer_model_answerer_for_musique_full/predictions/serialization_dir__execution_model_decomposer_for_musique_ans_and_full__predictions__musique_full_dev.jsonl \\\n                      --predictor multi_step_select_and_answer_transformer_rc --batch-size 16 --cuda-device 0 --silent \\\n                      --predictor-args '{\"predict_answerability\":true,\"skip_distractor_paragraphs\":false,\"use_predicted_decomposition\":true,\"selector_model_path\":\"serialization_dir/execution_by_select_and_answer_model_selector_for_musique_full/model.tar.gz\",\"num_select\":3}'\n\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/execution_by_select_and_answer_model_answerer_for_musique_full/predictions/serialization_dir__execution_model_decomposer_for_musique_ans_and_full__predictions__musique_full_dev.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\n## HotpotQA\n\n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eEnd2End Model [EE]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\n----\n\n### Experiment Name\n\n```bash\nend2end_model_for_hotpotqa_20k_dataset\n```\n\n### Download model\n\n```bash\npython download_models.py end2end_model_for_hotpotqa_20k_dataset\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/end2end_model_for_hotpotqa_20k_dataset.jsonnet \\\n                    --serialization-dir serialization_dir/end2end_model_for_hotpotqa_20k_dataset\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py end2end_model_for_hotpotqa_20k_dataset\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/end2end_model_for_hotpotqa_20k_dataset/model.tar.gz \\\n                      raw_data/hotpotqa_dev_20k.jsonl \\\n                      --output-file serialization_dir/end2end_model_for_hotpotqa_20k_dataset/predictions/hotpotqa_dev_20k.jsonl \\\n                      --predictor transformer_rc --batch-size 16 --cuda-device 0 --silent\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/end2end_model_for_hotpotqa_20k_dataset/predictions/hotpotqa_dev_20k.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eSelect+Answer Model [SA]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\u003cbr\u003eThe system has 2 parts given below: (i) Selector Model (ii) Answerer Model.\n\n\n----\n\n### Experiment Name\n\n```bash\n# Selector Model\nselect_and_answer_model_selector_for_hotpotqa_20k\n```\n\n### Download model\n\n```bash\npython download_models.py select_and_answer_model_selector_for_hotpotqa_20k\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/select_and_answer_model_selector_for_hotpotqa_20k.jsonnet \\\n                    --serialization-dir serialization_dir/select_and_answer_model_selector_for_hotpotqa_20k\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py select_and_answer_model_selector_for_hotpotqa_20k\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/select_and_answer_model_selector_for_hotpotqa_20k/model.tar.gz \\\n                      raw_data/hotpotqa_train_20k.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_selector_for_hotpotqa_20k/predictions/hotpotqa_train_20k.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\npython run.py predict serialization_dir/select_and_answer_model_selector_for_hotpotqa_20k/model.tar.gz \\\n                      raw_data/hotpotqa_dev_20k.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_selector_for_hotpotqa_20k/predictions/hotpotqa_dev_20k.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Answerer Model\nselect_and_answer_model_answerer_for_hotpotqa_20k\n```\n\n### Download model\n\n```bash\npython download_models.py select_and_answer_model_answerer_for_hotpotqa_20k\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/select_and_answer_model_answerer_for_hotpotqa_20k.jsonnet \\\n                    --serialization-dir serialization_dir/select_and_answer_model_answerer_for_hotpotqa_20k\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py select_and_answer_model_answerer_for_hotpotqa_20k\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/select_and_answer_model_answerer_for_hotpotqa_20k/model.tar.gz \\\n                      serialization_dir/select_and_answer_model_selector_for_hotpotqa_20k/predictions/hotpotqa_dev_20k.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_answerer_for_hotpotqa_20k/predictions/serialization_dir__select_and_answer_model_selector_for_hotpotqa_20k__predictions__hotpotqa_dev_20k.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/select_and_answer_model_answerer_for_hotpotqa_20k/predictions/serialization_dir__select_and_answer_model_selector_for_hotpotqa_20k__predictions__hotpotqa_dev_20k.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\n## 2WikiMultihopQA\n\n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eEnd2End Model [EE]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\n----\n\n### Experiment Name\n\n```bash\nend2end_model_for_2wikimultihopqa_20k_dataset\n```\n\n### Download model\n\n```bash\npython download_models.py end2end_model_for_2wikimultihopqa_20k_dataset\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/end2end_model_for_2wikimultihopqa_20k_dataset.jsonnet \\\n                    --serialization-dir serialization_dir/end2end_model_for_2wikimultihopqa_20k_dataset\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py end2end_model_for_2wikimultihopqa_20k_dataset\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/end2end_model_for_2wikimultihopqa_20k_dataset/model.tar.gz \\\n                      raw_data/2wikimultihopqa_dev_20k.jsonl \\\n                      --output-file serialization_dir/end2end_model_for_2wikimultihopqa_20k_dataset/predictions/2wikimultihopqa_dev_20k.jsonl \\\n                      --predictor transformer_rc --batch-size 16 --cuda-device 0 --silent\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/end2end_model_for_2wikimultihopqa_20k_dataset/predictions/2wikimultihopqa_dev_20k.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eSelect+Answer Model [SA]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\u003cbr\u003eThe system has 2 parts given below: (i) Selector Model (ii) Answerer Model.\n\n\n----\n\n### Experiment Name\n\n```bash\n# Selector Model\nselect_and_answer_model_selector_for_2wikimultihopqa_20k_dataset\n```\n\n### Download model\n\n```bash\npython download_models.py select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset.jsonnet \\\n                    --serialization-dir serialization_dir/select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset/model.tar.gz \\\n                      raw_data/2wikimultihopqa_train_20k.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset/predictions/2wikimultihopqa_train_20k.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\npython run.py predict serialization_dir/select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset/model.tar.gz \\\n                      raw_data/2wikimultihopqa_dev_20k.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset/predictions/2wikimultihopqa_dev_20k.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Answerer Model\nselect_and_answer_model_answerer_for_2wikimultihopqa_20k_dataset\n```\n\n### Download model\n\n```bash\npython download_models.py select_and_answer_model_answerer_for_2wikimultihopqa_20k_dataset\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/select_and_answer_model_answerer_for_2wikimultihopqa_20k_dataset.jsonnet \\\n                    --serialization-dir serialization_dir/select_and_answer_model_answerer_for_2wikimultihopqa_20k_dataset\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py select_and_answer_model_answerer_for_2wikimultihopqa_20k_dataset\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/select_and_answer_model_answerer_for_2wikimultihopqa_20k_dataset/model.tar.gz \\\n                      serialization_dir/select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset/predictions/2wikimultihopqa_dev_20k.jsonl \\\n                      --output-file serialization_dir/select_and_answer_model_answerer_for_2wikimultihopqa_20k_dataset/predictions/serialization_dir__select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset__predictions__2wikimultihopqa_dev_20k.jsonl \\\n                      --predictor transformer_rc --batch-size 16 --cuda-device 0 --silent\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/select_and_answer_model_answerer_for_2wikimultihopqa_20k_dataset/predictions/serialization_dir__select_and_answer_model_selector_for_2wikimultihopqa_20k_dataset__predictions__2wikimultihopqa_dev_20k.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eExecution by End2End Model [EX(EE)]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\u003cbr\u003eThe system has 2 parts given below: (i) Decomposer Model (ii) Executor Model.\n\n\n----\n\n### Experiment Name\n\n```bash\n# Decomposer Model\nexecution_model_decomposer_for_2wikimultihopqa\n```\n\n### Download model\n\n```bash\npython download_models.py execution_model_decomposer_for_2wikimultihopqa\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_model_decomposer_for_2wikimultihopqa.jsonnet \\\n                    --serialization-dir serialization_dir/execution_model_decomposer_for_2wikimultihopqa\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_model_decomposer_for_2wikimultihopqa\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_model_decomposer_for_2wikimultihopqa/model.tar.gz \\\n                      raw_data/2wikimultihopqa_dev_20k.jsonl \\\n                      --output-file serialization_dir/execution_model_decomposer_for_2wikimultihopqa/predictions/2wikimultihopqa_dev_20k.jsonl \\\n                      --predictor question_translator --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Executor Model\nexecution_by_end2end_model_for_2wikimultihopqa\n```\n\n### Download model\n\n```bash\npython download_models.py execution_by_end2end_model_for_2wikimultihopqa\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_by_end2end_model_for_2wikimultihopqa.jsonnet \\\n                    --serialization-dir serialization_dir/execution_by_end2end_model_for_2wikimultihopqa\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_by_end2end_model_for_2wikimultihopqa\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_by_end2end_model_for_2wikimultihopqa/model.tar.gz \\\n                      serialization_dir/execution_model_decomposer_for_2wikimultihopqa/predictions/2wikimultihopqa_dev_20k.jsonl \\\n                      --output-file serialization_dir/execution_by_end2end_model_for_2wikimultihopqa/predictions/serialization_dir__execution_model_decomposer_for_2wikimultihopqa__predictions__2wikimultihopqa_dev_20k.jsonl \\\n                      --predictor multi_step_end2end_transformer_rc --batch-size 16 --cuda-device 0 --silent \\\n                      --predictor-args '{\"predict_answerability\":false,\"skip_distractor_paragraphs\":false,\"use_predicted_decomposition\":true}'\n\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/execution_by_end2end_model_for_2wikimultihopqa/predictions/serialization_dir__execution_model_decomposer_for_2wikimultihopqa__predictions__2wikimultihopqa_dev_20k.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\u003cdetails\u003e \u003csummary\u003e\n\u003cstrong\u003eExecution by Select+Answer Model [EX(SA)]\u003c/strong\u003e\n\u003c/summary\u003e\n\n\u003cbr\u003eThe system has 3 parts given below: (i) Decomposer Model (ii) Selector of Executor Model (iii) Answerer of Executor Model.\n\n\n----\n\n### Experiment Name\n\n```bash\n# Decomposer Model\nexecution_model_decomposer_for_2wikimultihopqa\n```\n\n### Download model\n\n```bash\npython download_models.py execution_model_decomposer_for_2wikimultihopqa\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_model_decomposer_for_2wikimultihopqa.jsonnet \\\n                    --serialization-dir serialization_dir/execution_model_decomposer_for_2wikimultihopqa\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_model_decomposer_for_2wikimultihopqa\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_model_decomposer_for_2wikimultihopqa/model.tar.gz \\\n                      raw_data/2wikimultihopqa_dev_20k.jsonl \\\n                      --output-file serialization_dir/execution_model_decomposer_for_2wikimultihopqa/predictions/2wikimultihopqa_dev_20k.jsonl \\\n                      --predictor question_translator --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Selector of Executor Model\nexecution_by_select_and_answer_model_selector_for_2wikimultihopqa\n```\n\n### Download model\n\n```bash\npython download_models.py execution_by_select_and_answer_model_selector_for_2wikimultihopqa\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_by_select_and_answer_model_selector_for_2wikimultihopqa.jsonnet \\\n                    --serialization-dir serialization_dir/execution_by_select_and_answer_model_selector_for_2wikimultihopqa\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_by_select_and_answer_model_selector_for_2wikimultihopqa\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_by_select_and_answer_model_selector_for_2wikimultihopqa/model.tar.gz \\\n                      raw_data/2wikimultihopqa_single_hop_version_train_20k.jsonl \\\n                      --output-file serialization_dir/execution_by_select_and_answer_model_selector_for_2wikimultihopqa/predictions/2wikimultihopqa_single_hop_version_train_20k.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\npython run.py predict serialization_dir/execution_by_select_and_answer_model_selector_for_2wikimultihopqa/model.tar.gz \\\n                      raw_data/2wikimultihopqa_single_hop_version_dev.jsonl \\\n                      --output-file serialization_dir/execution_by_select_and_answer_model_selector_for_2wikimultihopqa/predictions/2wikimultihopqa_single_hop_version_dev.jsonl \\\n                      --predictor inplace_text_ranker --batch-size 16 --cuda-device 0 --silent\n\n```\n\n\n----\n\n### Experiment Name\n\n```bash\n# Answerer of Executor Model\nexecution_by_select_and_answer_model_answerer_for_2wikimultihopqa\n```\n\n### Download model\n\n```bash\npython download_models.py execution_by_select_and_answer_model_answerer_for_2wikimultihopqa\n```\n\n### Train from scratch\n\n```bash\npython run.py train experiment_configs/execution_by_select_and_answer_model_answerer_for_2wikimultihopqa.jsonnet \\\n                    --serialization-dir serialization_dir/execution_by_select_and_answer_model_answerer_for_2wikimultihopqa\n```\n\n### Download prediction/s\n\n```bash\npython download_raw_predictions.py execution_by_select_and_answer_model_answerer_for_2wikimultihopqa\n```\n\n### Predict with a trained or a downloaded model\n\n\n```bash\npython run.py predict serialization_dir/execution_by_select_and_answer_model_answerer_for_2wikimultihopqa/model.tar.gz \\\n                      serialization_dir/execution_model_decomposer_for_2wikimultihopqa/predictions/2wikimultihopqa_dev_20k.jsonl \\\n                      --output-file serialization_dir/execution_by_select_and_answer_model_answerer_for_2wikimultihopqa/predictions/serialization_dir__execution_model_decomposer_for_2wikimultihopqa__predictions__2wikimultihopqa_dev_20k.jsonl \\\n                      --predictor multi_step_select_and_answer_transformer_rc --batch-size 16 --cuda-device 0 --silent \\\n                      --predictor-args '{\"predict_answerability\":false,\"skip_distractor_paragraphs\":false,\"use_predicted_decomposition\":true,\"selector_model_path\":\"serialization_dir/execution_by_select_and_answer_model_selector_for_2wikimultihopqa/model.tar.gz\",\"num_select\":3}'\n\n\n# If you want to convert predictions to the official format, run:\npython raw_predictions_to_official_format.py serialization_dir/execution_by_select_and_answer_model_answerer_for_2wikimultihopqa/predictions/serialization_dir__execution_model_decomposer_for_2wikimultihopqa__predictions__2wikimultihopqa_dev_20k.jsonl\n\n```\n\n\u003c/details\u003e\n    \n\n\n\n# Citation\n\nIf you use this in your work, please cite use:\n\n```\n@article{trivedi2021musique,\n  title={{M}u{S}i{Q}ue: Multihop Questions via Single-hop Question Composition},\n  author={Trivedi, Harsh and Balasubramanian, Niranjan and Khot, Tushar and Sabharwal, Ashish},\n  journal={Transactions of the Association for Computational Linguistics},\n  year={2022}\n  publisher={MIT Press}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FStonyBrookNLP%2Fmusique","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FStonyBrookNLP%2Fmusique","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FStonyBrookNLP%2Fmusique/lists"}