{"id":19932051,"url":"https://github.com/amazon-science/wqa_tanda","last_synced_at":"2026-01-27T23:48:18.471Z","repository":{"id":49845548,"uuid":"222008871","full_name":"amazon-science/wqa_tanda","owner":"amazon-science","description":"This repo provides code and data used in our TANDA paper.","archived":false,"fork":false,"pushed_at":"2024-09-13T17:08:46.000Z","size":33,"stargazers_count":108,"open_issues_count":1,"forks_count":26,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-09-09T05:11:51.652Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/amazon-science.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-15T21:50:01.000Z","updated_at":"2024-09-13T17:08:50.000Z","dependencies_parsed_at":"2025-03-01T11:37:51.176Z","dependency_job_id":null,"html_url":"https://github.com/amazon-science/wqa_tanda","commit_stats":null,"previous_names":["alexa/wqa_tanda"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/amazon-science/wqa_tanda","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fwqa_tanda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fwqa_tanda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fwqa_tanda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fwqa_tanda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/amazon-science","download_url":"https://codeload.github.com/amazon-science/wqa_tanda/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amazon-science%2Fwqa_tanda/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28827902,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-27T23:29:49.665Z","status":"ssl_error","status_checked_at":"2026-01-27T23:25:58.379Z","response_time":168,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T23:08:53.702Z","updated_at":"2026-01-27T23:48:18.456Z","avatar_url":"https://github.com/amazon-science.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection\n\nWe put together a script, data, and trained models used in our [paper](https://arxiv.org/abs/1911.04118). In a nutshell, TANDA is a technique for fine-tuning pre-trained Transformer models sequentially in two steps:\n* first, transfer a pre-trained model to a model for a general task by fine-tuning it on a large and high-quality dataset;\n* then, perform a second fine-tuning step to adapt the transferred model to the target domain.\n\n## Script\n\nWe base our implementation on the [transformers](https://github.com/huggingface/transformers) package. We use the following script to enable `sequential fine-tuning` option for the package.\n\n```\ngit clone https://github.com/huggingface/transformers.git\ncd transformers\ngit checkout f3386 -b tanda-sequential-finetuning\ngit apply tanda-sequential-finetuning-with-asnq.diff\n```\n\n* `f3386` is the latest commit as of `Sun Nov 17 18:08:51 2019 +0900`, and `tanda-sequential-finetuning-with-asnq.diff` is the diff to enable the option.\n\nFor example, to transfer with ASNQ and adapt with a target dataset:\n* download [the ASNQ dataset](#answer-sentence-natural-questions-asnq) and the target dataset (e.g. Wiki-QA, formatted similar as ASNQ), and\n* run the following script\n \n\n```\npython run_glue.py \\\n    --model_type bert \\\n    --model_name_or_path bert-base-uncased \\\n    --task_name ASNQ \\\n    --do_train \\\n    --do_eval \\\n    --do_lower_case \\\n    --data_dir [PATH-TO-ASNQ] \\\n    --per_gpu_train_batch_size 150 \\\n    --learning_rate 2e-5 \\\n    --num_train_epochs 2.0 \\\n    --output_dir [PATH-TO-TRANSFER-FOLDER]\n\npython run_glue.py \\\n    --model_type bert \\\n    --model_name_or_path [PATH-TO-TRANSFER-FOLDER] \\\n    --task_name ASNQ \\\n    --do_train \\\n    --do_eval \\\n    --sequential \\\n    --do_lower_case \\\n    --data_dir [PATH-TO-WIKI-QA] \\\n    --per_gpu_train_batch_size 150 \\\n    --learning_rate 1e-6 \\\n    --num_train_epochs 2.0 \\\n    --output_dir [PATH-TO-OUTPUT-FOLDER]\n```\n\n## Data\n\nWe use the following datasets in the paper:\n\n### Answer-Sentence Natural Questions (ASNQ)\n* ASNQ is a dataset for answer sentence selection derived from Google Natural Questions (NQ) dataset (Kwiatkowski et al. 2019). The dataset details can be found in our paper.\n* ASNQ is used to transfer the pre-trained models in the paper, and can be downloaded [here](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/data/asnq.tar).\n* ASNQ-Dev++ can be downloaded [here](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/data/asnq.dev%2B%2B.tar).\n\n### Domain Datasets\n* **Wiki-QA**: we used the Wiki-QA dataset from [here](http://aka.ms/WikiQA) and removed all the questions that have no correct answers.\n* **TREC-QA**: we used the `*-filtered.jsonl` version of this dataset from [here](https://github.com/mcrisc/lexdecomp/tree/master/trec-qa).\n\n\n## Models\n\n### Models Transferred on ASNQ\n\n - [BERT-Base ASNQ](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/models/tanda_bert_base_asnq.tar)\n - [BERT-Large ASNQ](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/models/tanda_bert_large_asnq.tar)\n - [RoBERTa-Base ASNQ](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/models/tanda_roberta_base_asnq.tar)\n - [RoBERTa-Large ASNQ](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/models/tanda_roberta_large_asnq.tar)\n\n### TANDA: Models Transferred on ASNQ, then Fine-Tuned with Wiki-QA\n\n - [TANDA: BERT-Base ASNQ \u0026rarr; Wiki-QA](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/models/tanda_bert_base_asnq_wikiqa.tar)\n - [TANDA: BERT-Large ASNQ \u0026rarr; Wiki-QA](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/models/tanda_bert_large_asnq_wikiqa.tar)\n - [TANDA: RoBERTa-Large ASNQ \u0026rarr; Wiki-QA](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/models/tanda_roberta_large_asnq_wikiqa.tar)\n\n### TANDA: Models Transferred on ASNQ, then Fine-Tuned with TREC-QA\n\n - [TANDA: BERT-Base ASNQ \u0026rarr; TREC-QA](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/models/tanda_bert_base_asnq_trec.tar)\n - [TANDA: BERT-Large ASNQ \u0026rarr; TREC-QA](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/models/tanda_bert_large_asnq_trec.tar)\n - [TANDA: RoBERTa-Large ASNQ \u0026rarr; TREC-QA](https://d3t7erp6ge410c.cloudfront.net/tanda-aaai-2020/models/tanda_roberta_large_asnq_trec.tar)\n\n## How To Cite TANDA\nThe paper appeared in the AAAI 2020 proceedings. Please cite our work if you find our paper, dataset, pretrained models or code useful:\n\n```\n@article{Garg_2020,\n   title={TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection},\n   volume={34},\n   ISSN={2159-5399},\n   url={http://dx.doi.org/10.1609/AAAI.V34I05.6282},\n   DOI={10.1609/aaai.v34i05.6282},\n   number={05},\n   journal={Proceedings of the AAAI Conference on Artificial Intelligence},\n   publisher={Association for the Advancement of Artificial Intelligence (AAAI)},\n   author={Garg, Siddhant and Vu, Thuy and Moschitti, Alessandro},\n   year={2020},\n   month={Apr},\n   pages={7780–7788}\n}\n```\n\n## License Summary\n\nThe documentation, including the shared [data](#data) and [models](#models), is made available under the Creative Commons Attribution-ShareAlike 3.0 Unported License. See the LICENSE file.\n\nThe sample [script](#script) within this documentation is made available under the MIT-0 license. See the LICENSE-SAMPLECODE file.\n\n\n## Contact\nFor help or issues, please submit a GitHub issue.\n\nFor direct communication, please contact Siddhant Garg (https://github.com/sid7954), Thuy Vu (thuyvu is at amazon dot com), or Alessandro Moschitti (amosch is at amazon dot com).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famazon-science%2Fwqa_tanda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famazon-science%2Fwqa_tanda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famazon-science%2Fwqa_tanda/lists"}