{"id":19520156,"url":"https://github.com/osu-nlp-group/qa4re","last_synced_at":"2025-07-23T21:38:15.033Z","repository":{"id":167716269,"uuid":"639269285","full_name":"OSU-NLP-Group/QA4RE","owner":"OSU-NLP-Group","description":"[ACL'23 Findings] \"Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors\"","archived":false,"fork":false,"pushed_at":"2023-12-22T04:36:07.000Z","size":52,"stargazers_count":39,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-26T07:35:33.407Z","etag":null,"topics":["instruction-tuning","large-language-models","relation-extraction","zero-shot-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OSU-NLP-Group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-11T05:58:02.000Z","updated_at":"2025-02-19T06:40:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"51b678e9-a4ce-4799-ad88-680ddea279d4","html_url":"https://github.com/OSU-NLP-Group/QA4RE","commit_stats":null,"previous_names":["osu-nlp-group/qa4re"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/OSU-NLP-Group/QA4RE","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OSU-NLP-Group%2FQA4RE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OSU-NLP-Group%2FQA4RE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OSU-NLP-Group%2FQA4RE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OSU-NLP-Group%2FQA4RE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OSU-NLP-Group","download_url":"https://codeload.github.com/OSU-NLP-Group/QA4RE/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OSU-NLP-Group%2FQA4RE/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266753961,"owners_count":23979144,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["instruction-tuning","large-language-models","relation-extraction","zero-shot-learning"],"created_at":"2024-11-11T00:24:06.186Z","updated_at":"2025-07-23T21:38:15.011Z","avatar_url":"https://github.com/OSU-NLP-Group.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLM-QA4RE\n\nData and code for ACL 2023 Findings: [Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors](https://arxiv.org/pdf/2305.11159.pdf).\n\nWe present LLM-QA4RE, which aligns underrepresented tasks in the instruction-tuning dataset (relation extraction) to a common task (question answering) to unlock instruction-tuned LLMs' abilities on relation extraction. \n\nQA4RE achieves significant and consistent performance gains over 6 LLMs across 4 datasets. In addition, it shows strong transferability to model sizes from 175B (GPT-3.5 series) to even 80M (FLAN-T5 Small).\n\n\u003c!-- ![QA4RE-main figure.jpeg](https://s2.loli.net/2023/05/15/Lk1saYNjni3yqWP.jpg) --\u003e\n\n\u003ca href=\"https://sm.ms/image/Lk1saYNjni3yqWP\" target=\"_blank\"\u003e\u003cimg src=\"https://s2.loli.net/2023/05/15/Lk1saYNjni3yqWP.jpg\" width=\"75%\" height=\"75%\" \u003e\u003c/a\u003e\n\n\n\u003c!-- #### TODO\n- [x] Organize and release Code for GPT-3.5 Series LLMs\n- [x] Release output results of GPT-3.5 Series LLMs\n- [x] Organize and release CODE for FLAN T5 Series LLMs\n- [ ] Refactor code to save results as json/jsonl --\u003e\n\n## Installation\n\nRun the following commands to create a conda environment with the required packages.\n\n```shell\nconda create -n QA4RE python=3.9.13 pip\nconda activate QA4RE\npip install -r requirements.txt\n# same env with few-shot-bioIE\n```\n\n\n## Data and Launch\nDownload data and subsets via [Google Drive](https://drive.google.com/file/d/1tAB7V4_bV76FiPGMsoOWWJnZPtpePtwe/view?usp=sharing)\n\nResults and prompts are saved in [Google Drive](https://drive.google.com/file/d/1hsbwd6Qf5nSH9w5uWgSj9snnpJuLTpkH/view?usp=sharing)\n\n\u003c!-- are prepared in `./data` dir --\u003e\nUnzip directly in `./` and then the root folder should organize like this:\n\n```\n.\n├───  data\n│    ├───  RETACRED\n│    ├───  TACRED\n│    ├───  TACREV\n│    ├───  semeval\n├───  outputs\n│    ├───  RETACRED\n│    ├───  TACRED\n│    ├───  TACREV\n│    ├───  semeval\n├───  projs\n│    ├───  QA4RE\n│    ├───  vanillaRE\n│    ├───  README.md\n│    ├───  re_templates.py\n│    └───  re_utils.py\n├───  utils\n│   ...\n```\n\nFor running, please refer to the [README](./projs/README.md) in `./projs` dir.\n\n## Results\n#### QA4RE works on GPT-3.5 Series and FLAN-T5 Series, 6 LLMs in total\n\u003c!-- ![QA4RE-table 1.jpeg](https://s2.loli.net/2023/05/15/is8XGo71lODm3Bq.jpg) --\u003e\n\u003ca href=\"https://sm.ms/image/is8XGo71lODm3Bq\" target=\"_blank\"\u003e\u003cimg src=\"https://s2.loli.net/2023/05/15/is8XGo71lODm3Bq.jpg\" width=\"75%\" height=\"75%\" \u003e\u003c/a\u003e\n\n\n#### QA4RE works on smaller instruction-tuned models.\n\u003c!-- ![QA4RE-table 8.jpeg](https://s2.loli.net/2023/05/15/IEUrGuBWn9FNmb8.jpg) --\u003e\n\u003ca href=\"https://sm.ms/image/IEUrGuBWn9FNmb8\" target=\"_blank\"\u003e\u003cimg src=\"https://s2.loli.net/2023/05/15/IEUrGuBWn9FNmb8.jpg\" width=\"75%\" height=\"75%\"\u003e\u003c/a\u003e\n\n## Cite\n\nIf you find our paper, code, or data helpful, please consider citing the paper:\n\n```\n@inproceedings{Zhang2023LLM-QA4RE,\n  title={Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors},\n  author={Kai Zhang, Bernal Jiménez Gutiérrez, Yu Su},\n  booktitle={Findings of ACL},\n  year={2023}\n}\n```\n\nThis work is based on our prior work:\n\n```\n@inproceedings{Gutierrez2022Thinking,\n  title={Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again},\n  author={Bernal Jiménez Gutiérrez, Nikolas McNeal, Clay Washington, You Chen, Lang Li, Huan Sun, Yu Su},\n  booktitle={Findings of EMNLP},\n  year={2022}\n}\n```\n\n## Question\n\nIf you have any questions, please feel free to contact `drogozhang[AT]gmail[DOT]com` or open an issue so we can help you better and quicker :)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fosu-nlp-group%2Fqa4re","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fosu-nlp-group%2Fqa4re","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fosu-nlp-group%2Fqa4re/lists"}