{"id":53228,"url":"https://github.com/RenzeLou/awesome-instruction-learning","name":"awesome-instruction-learning","description":"Papers and Datasets on Instruction Tuning and Following. ✨✨✨","projects_count":177,"last_synced_at":"2026-08-31T16:00:31.599Z","repository":{"id":143230770,"uuid":"604404340","full_name":"RenzeLou/awesome-instruction-learning","owner":"RenzeLou","description":"Papers and Datasets on Instruction Tuning and Following. ✨✨✨","archived":false,"fork":false,"pushed_at":"2024-04-04T19:48:35.000Z","size":6553,"stargazers_count":512,"open_issues_count":0,"forks_count":20,"subscribers_count":7,"default_branch":"main","last_synced_at":"2026-08-12T01:48:31.555Z","etag":null,"topics":["awesome-list","datasets","in-context-learning","instruction","instruction-learning","instruction-tuning","large-language-models","paper-list","pretrained-language-model","prompt","survey"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2303.10475","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RenzeLou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-02-21T01:43:05.000Z","updated_at":"2026-07-18T06:52:45.000Z","dependencies_parsed_at":"2023-06-09T01:00:23.690Z","dependency_job_id":"94386838-4796-499c-a5cc-f9084673edb0","html_url":"https://github.com/RenzeLou/awesome-instruction-learning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/RenzeLou/awesome-instruction-learning","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RenzeLou%2Fawesome-instruction-learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RenzeLou%2Fawesome-instruction-learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RenzeLou%2Fawesome-instruction-learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RenzeLou%2Fawesome-instruction-learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RenzeLou","download_url":"https://codeload.github.com/RenzeLou/awesome-instruction-learning/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RenzeLou%2Fawesome-instruction-learning/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":37004163,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-08-22T15:14:58.755Z","status":"online","status_checked_at":"2026-08-31T02:00:07.497Z","response_time":119,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"created_at":"2024-01-15T10:27:58.748Z","updated_at":"2026-08-31T16:00:31.600Z","primary_language":null,"list_of_lists":false,"displayable":true,"categories":["4. 🗂️ Taxonomy","5. 📊 Analyses","6. 🤖 Applications","7. 📖 Extended Reading","3. 📚 Corpora","❤️ Contribution","2. 🎓 Surveys and Tutorials","⭐ Star History"],"sub_categories":["4.3 Human-oriented Instruction","5.6 Complexity","6.3 General-purpose Language Models","5.1 Scale","5.3 Robustness and Safety","7.3 Human Feedback vs. Model Feedback","6.4 Other Papers","7.2 ChatGPT-related Papers","4.2 PLM-oriented Instruction","7.5 Other Papers","6.2 Data and Feature Augmentation","4.1 Entailment-oriented Instruction","5.2 Explanability","5.4 Evaluation","5.5 Negation","5.7 Other Papers","6.1 Human-Computer Interaction","7.1 Instruction Induction","7.4 Scalable Oversight and Alignment"],"readme":"\u003c!-- omit in toc --\u003e\n\u003ch1 align=\"center\"\u003e Awesome Instruction Learning \u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/RenzeLou/awesome-instruction-learning\"\u003e\u003cimg src=\"https://awesome.re/badge.svg\" alt=\"Awesome\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/RenzeLou/awesome-instruction-learning#-star-history\"\u003e\u003cimg src=\"https://img.shields.io/github/stars/RenzeLou/awesome-instruction-learning?style=social\" alt=\"Stars\" /\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/RenzeLou/awesome-instruction-learning/commits/main\"\u003e\u003cimg src=\"https://img.shields.io/github/last-commit/RenzeLou/awesome-instruction-learning?color=#00FA9A\" alt=\"Commit\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/RenzeLou/awesome-instruction-learning/blob/main/count_number.py\"\u003e\u003cimg src=\"https://img.shields.io/badge/PaperNumber-199-blue\" alt=\"PaperNumber\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/RenzeLou/awesome-instruction-learning/pulls\"\u003e\u003cimg src=\"https://img.shields.io/badge/PRs-Welcome-red\" alt=\"PullRequests\" /\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003c!-- [![Awesome](https://awesome.re/badge.svg)](https://github.com/RenzeLou/awesome-instruction-learning) [![Stars](https://img.shields.io/github/stars/RenzeLou/awesome-instruction-learning?style=social)](https://github.com/RenzeLou/awesome-instruction-learning#-star-history)\n\n[![Commit](https://img.shields.io/github/last-commit/RenzeLou/awesome-instruction-learning?color=#00FA9A)](https://github.com/RenzeLou/awesome-instruction-learning/commits/main) [![PaperNumber](https://img.shields.io/badge/PaperNumber-161-blue)](https://github.com/RenzeLou/awesome-instruction-learning/blob/main/count_number.py) [![PullRequests](https://img.shields.io/badge/PRs-Welcome-red)](https://github.com/RenzeLou/awesome-instruction-learning/pulls)  --\u003e\n\n\u003cp align=\"center\"\u003e\n🔥🔥🔥 An awesome reading list of \u003cb\u003eInstruction Tuning and Following\u003c/b\u003e, including \u003cem\u003epapers\u003c/em\u003e and \u003cem\u003edatasets\u003c/em\u003e. \n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\u003ci\u003e 👉 Explore our latest survey update! Feel free to dive in and discover the improvements we've made 👀 🤗 : \u003ca href=\"https://arxiv.org/abs/2303.10475\"\u003e \u003cb\u003eLatest Survey\u003c/b\u003e \u003c/a\u003e \u003c/i\u003e\n\u003c/p\u003e\n\u003c!-- https://drive.google.com/file/d/1vrx3BSkHlkNO6_nP9G9l9Ape7vEoTOdf/view?usp=sharing --\u003e\n\n---\n\n\u003c!-- What is instruction learning?\nWhy instruction learning?\n--\u003e\n\n\u003c!-- TODO\nadd \"must read\" section to select a core subset of instruction tuning papers\n--\u003e\n\n\n\u003c!-- omit in toc --\u003e\n## ❤️ Contribution\n\nThis repository is currently maintained by \u003cins\u003e[Renze Lou](https://renzelou.github.io/) @ PennState\u003c/ins\u003e and \u003cins\u003e[Kai Zhang](https://drogozhang.github.io/) @ OhioState\u003c/ins\u003e. **We appreciate any contributions** ❤️.\n\n\n\u003c!-- **\u003cfont color='red'\u003eWork still in progress\u003c/font\u003e**  🚀, **we appreciate any suggestions and contributions** ❤️. --\u003e\n\nIf you have any suggestions or find any missed papers, feel free to [reach out](https://outlook.office.com/mail/deeplink/compose?mailtouri=mailto%3Amarionojump0722%40gmail.com) or submit a [pull request](https://github.com/RenzeLou/awesome-instruction-learning/pulls):\n\n1. Use following markdown format.\n\n```markdown\n**Paper Title.** *Author 1, Author 2, and Author 3.* \u003cins\u003eConference/Journal/Preprint\u003c/ins\u003e Year. [[pdf](link)]; [[other resources](link)].\n```\n\u003c!-- \u003e1. **Paper Title.** *Author 1, Author 2, and Author 3.* Conference/Journal/Preprint Year. [[pdf](link)]. --\u003e\n\n\n2. If one preprint paper has multiple versions, please use **the earliest submitted year**.\n   \n3. Display the papers in **a year descending order** (the latest, the first).\n\n\n\u003c!-- omit in toc --\u003e\n## 🥳 Citation\n\nFind this repository helpful? 😊😊😊  \n\nPlease consider citing our paper. 👇👇👇\n\n\n\u003c!-- *(**Note that the current version of our survey is only a draft, and we are still working on it.** The first readable version is arriving soon.)* 🚀 --\u003e\n\n```\n@article{lou2023instruction,\n  title={A Comprehensive Survey on Instruction Following},\n  author={Lou, Renze and Zhang, Kai and Yin, Wenpeng},\n  journal={arXiv preprint arXiv:2303.10475},\n  year={2023}\n}\n```\n\n\n\n---\n\n\u003c!-- omit in toc --\u003e\n## 🔍 Table of Contents \n\n- [1. 💁🏽‍♀️ Introduction](#1-️-introduction)\n- [2. 🎓 Surveys and Tutorials](#2--surveys-and-tutorials)\n- [3. 📚 Corpora](#3--corpora)\n- [4. 🗂️ Taxonomy](#4-️-taxonomy)\n  - [4.1 Entailment-oriented Instruction](#41-entailment-oriented-instruction)\n  - [4.2 PLM-oriented Instruction](#42-plm-oriented-instruction)\n  - [4.3 Human-oriented Instruction](#43-human-oriented-instruction)\n- [5. 📊 Analyses](#5--analyses)\n  - [5.1 Scale](#51-scale)\n  - [5.2 Explanability](#52-explanability)\n  - [5.3 Robustness and Safety](#53-robustness-and-safety)\n  - [5.4 Evaluation](#54-evaluation)\n  - [5.5 Negation](#55-negation)\n  - [5.6 Complexity](#56-complexity)\n  - [5.7 Other Papers](#57-other-papers)\n- [6. 🤖 Applications](#6--applications)\n  - [6.1 Human-Computer Interaction](#61-human-computer-interaction)\n  - [6.2 Data and Feature Augmentation](#62-data-and-feature-augmentation)\n  - [6.3 General-purpose Language Models](#63-general-purpose-language-models)\n  - [6.4 Other Papers](#64-other-papers)\n- [7. 📖 Extended Reading](#7--extended-reading)\n  - [7.1 Instruction Induction](#71-instruction-induction)\n  - [7.2 ChatGPT-related Papers](#72-chatgpt-related-papers)\n  - [7.3 Human Feedback vs. Model Feedback](#73-human-feedback-vs-model-feedback)\n  - [7.4 Scalable Oversight and Alignment](#74-scalable-oversight-and-alignment)\n  - [7.5 Other Papers](#75-other-papers)\n\n---\n\n\n## 1. 💁🏽‍♀️ Introduction\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=./resources/introduction.png width=85% title=\"Instruction Learning vs. Full Supervised Learning\" /\u003e\n\u003c/div\u003e\n\n\u003c!-- \u003ccenter\u003e\n    \u003cimg style=\"border-radius: 0.3125em;\n    box-shadow: 0 2px 4px 0 rgba(34,36,38,.12),0 2px 10px 0 rgba(34,36,38,.08);\" \n    src=\"./resources/introduction.png\"\u003e\n    \u003cbr\u003e\n    \u003cdiv style=\"color:orange; border-bottom: 1px solid #d9d9d9;\n    display: inline-block;\n    color: #999;\n    padding: 2px;\"\u003eFull Supervised Learning vs. Instruction Learning\u003c/div\u003e\n\u003c/center\u003e --\u003e\n\n\n\nWhy *instruction-driven* learning instead of *example-driven* learning?\n\n\n- 👉 **Affordable.**  For the conventional example-driven supervised learning, each \u003cins\u003e*downstream*\u003c/ins\u003e task usually requires extensive labeled examples 💰. While for instruction learning, each \u003cins\u003e*downstream*\u003c/ins\u003e task may require only one instruction and just a few examples 🤩.\n- 👉 **One model, all tasks.** An ideal AI system should be able to quickly understand and handle various new tasks 💫.\n- 👉 **A promising research direction.** Traditional example-driven supervised learning uses labeled instances to represent the task semantics, i.e., training models by observing numerous examples to recover the original task meaning. Therefore, **why not directly use the task instruction**, **which has already occupied the essential task semantics**?\n\n\u003c!-- We all know collecting extensive labeled examples are usually expensive 💰. --\u003e\n\n## 2. 🎓 Surveys and Tutorials\n\n\u003c!-- There are several awesome surveys and tutorials on textual instruction learning. --\u003e\n\u003c!-- To our knowledge, our survey is the first one to provide a comprehensive and broader overview of the field of instruction learning. --\u003e\n\u003c!-- Since each survey focuses on specific in-context instruction, we attach a label to each of them to distinguish these topics.\n, including `prompt`, `demonstrations`, `reasoning`, and `overview` (which means a broader perspective). --\u003e\n\n\nWe use the label ![comprehensive](https://img.shields.io/badge/comprehensive-FFA07A) to denote the papers with a more comprehensive perspective. While some other papers are more specific to a certain in-context instruction, including ![prompt](https://img.shields.io/badge/prompt-90EE90), few-shot ![in-context demonstrations](https://img.shields.io/badge/demonstrations-FFB6C1), and CoT ![reasoning](https://img.shields.io/badge/reasoning-9cf).\n\n1. **A Comprehensive Survey on Instruction Following.** *Renze Lou, Kai Zhang, and Wenpeng Yin.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/abs/2303.10475)]; [[paper list](https://github.com/RenzeLou/awesome-instruction-learning)]. ![comprehensive](https://img.shields.io/badge/comprehensive-FFA07A)\n   \n2. **Learning from Task Instructions.** *Wenpeng Yin, Qinyuan Ye, Pengfei Liu, Xiang Ren, and Hinrich Schütze.* \u003cins\u003eEMNLP Tutorial\u003c/ins\u003e 2023. [[pdf](https://aclanthology.org/2023.emnlp-tutorial.4.pdf)]. ![comprehensive](https://img.shields.io/badge/comprehensive-FFA07A)\n   \n3. **Nature Language Reasoning, A Survey.** *Fei Yu, Hongbo Zhang, and Benyou Wang.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2303.14725.pdf)]; [[paper list](https://github.com/FreedomIntelligence/ReasoningNLP)]. ![reasoning](https://img.shields.io/badge/reasoning-9cf)\n\n4. **Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.** *Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig.* \u003cins\u003eACM Computing Surveys\u003c/ins\u003e 2023. [[pdf](https://dl.acm.org/doi/pdf/10.1145/3560815)]; [[website](http://pretrain.nlpedia.ai/)]. ![prompt](https://img.shields.io/badge/prompt-90EE90)\n   \n5. **A Survey on In-context Learning**. *Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui*. \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2301.00234.pdf)]. ![in-context demonstrations](https://img.shields.io/badge/demonstrations-FFB6C1)\n   \n6. **Towards Reasoning in Large Language Models: A Survey.** *Jie Huang, and Kevin Chen-Chuan Chang.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.10403.pdf)]; [[paper list](https://github.com/jeffhj/LM-reasoning)]. ![reasoning](https://img.shields.io/badge/reasoning-9cf)\n\n7. **Reasoning with Language Model Prompting: A Survey.** *Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.09597.pdf)]; [[paper list](https://github.com/zjunlp/Prompt4ReasoningPapers)]. ![reasoning](https://img.shields.io/badge/reasoning-9cf)\n\n\n\n## 3. 📚 Corpora\n\n**The high-quality dataset is the key factor for successful instruction tuning**. Therefore, we put the \"corpora\" section here to emphasize its importance.\n\nWe carefully design the following table, make it easy to be referred to, and keep it up-to-date. Hope it can contribute to future research of instruction tuning. 🤗\n\n *(Some rows come from [Longpre et al.](https://arxiv.org/pdf/2301.13688.pdf), thanks for their great work ❤️.)* \n\n\u003ctable id=\"copora-table\" style=\"height: 353px; width: 690px;\" width=\"629\"\u003e\n\u003ctbody\u003e\n\u003ctr style=\"height: 37px;\"\u003e\n\u003ctd style=\"height: 47px; width: 124.992px; text-align: left;\" rowspan=\"2\"\u003e\u003cstrong\u003eName\u0026nbsp;\u003c/strong\u003e\u003c/td\u003e\n\u003ctd style=\"height: 47px; width: 61.2891px; text-align: right;\" rowspan=\"2\"\u003e\u003cstrong\u003eRelease\u003c/strong\u003e\u003c/td\u003e\n\u003ctd style=\"height: 47px; width: 85.1875px; text-align: center;\" rowspan=\"2\"\u003e\u003cstrong\u003eData/Code\u003c/strong\u003e\u003c/td\u003e\n\u003ctd style=\"height: 37px; width: 144.289px; text-align: center;\" colspan=\"2\"\u003e\u003cstrong\u003eScale\u003c/strong\u003e\u003c/td\u003e\n\u003ctd style=\"height: 47px; width: 109.258px; text-align: center;\" rowspan=\"2\"\u003e\u003cstrong\u003eLanguage\u003c/strong\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 47px;\" rowspan=\"2\"\u003e\u003cstrong\u003eAnnotator\u003c/strong\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 10px;\"\u003e\n\u003ctd style=\"height: 10px; width: 60.7969px; text-align: right;\"\u003e\u003cstrong\u003e#Tasks\u003c/strong\u003e\u003c/td\u003e\n\u003ctd style=\"height: 10px; width: 77.4922px; text-align: right;\"\u003e\u003cstrong\u003e#Ins. (K)\u003c/strong\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2005.00700.pdf\"\u003eUnifiedQA\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e05/2020\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/allenai/unifiedqa\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e46\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e750\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2104.08835.pdf\"\u003eCrossFit\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2021\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/INK-USC/CrossFit\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e159\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e71,000\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2104.08773.pdf\"\u003eNatural Inst. v1\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2021\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://instructions.apps.allenai.org/\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e61\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e620\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2109.01652.pdf\"\u003eFlan 2021\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e09/2021\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/google-research/FLAN/tree/main#flan-2021\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e62\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e4,400\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2202.01279.pdf\"\u003eP3\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e10/2021\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://huggingface.co/datasets/bigscience/P3\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e62\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e12,000\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2110.15943.pdf\"\u003eMetaICL\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e10/2021\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/facebookresearch/MetaICL\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e142\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e3,500\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://openreview.net/pdf?id=Vzh1BFUCiIX\"\u003eExMix\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e11/2021\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/google-research/text-to-text-transfer-transformer\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e107\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e500\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\n\u003cp\u003e\u003ca href=\"https://arxiv.org/pdf/2204.07705.pdf\"\u003eSuperNI\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://arxiv.org/pdf/2204.07705.pdf\"\u003e(Natural Inst. v2)\u003c/a\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2022\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://instructions.apps.allenai.org/\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e1,613\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e5,000\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/multilingual-red\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2210.02414.pdf\"\u003eGLM\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e10/2022\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/THUDM/GLM-130B\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e77\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e12,000\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/bilingual-yellow\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2301.13688.pdf\"\u003eFlan 2022\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e10/2022\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/google-research/FLAN/tree/main/flan/v2\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e1,836\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e15,000\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/multilingual-red\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2211.01786.pdf\"\u003exP3\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e11/2022\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://huggingface.co/datasets/bigscience/xP3\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e71\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e81,000\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/multilingual-red\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2212.09689.pdf\"\u003eUnnatural Inst.\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e12/2022\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/orhonovich/unnatural-instructions\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e117\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e64\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e\n\u003cp\u003e🤖 InstructGPT\u003csub\u003e002\u003c/sub\u003e\u003c/p\u003e\n\u003cp\u003e\u003csub\u003e\u003ccode\u003etext-davinci-002\u003c/code\u003e\u003c/sub\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2212.10560.pdf\"\u003eSelf-Instruct\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e12/2022\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/yizhongw/self-instruct\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e82\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e\n\u003cp\u003e🤖 GPT-3\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003ccode\u003e\u003csub\u003edavinci\u003c/sub\u003e\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2212.12017.pdf\"\u003eOPT-IML\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e12/2022\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e2,207\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e18,000\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/multilingual-red\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://crfm.stanford.edu/2023/03/13/alpaca.html\"\u003eAlpaca\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e03/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/tatsu-lab/stanford_alpaca\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e52\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e\n\u003cp\u003e🤖 InstructGPT\u003csub\u003e003\u003c/sub\u003e\u003c/p\u003e\n\u003cp\u003e\u003csub\u003e\u003ccode\u003etext-davinci-003\u003c/code\u003e\u003c/sub\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2304.01196.pdf\"\u003eBaize\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/project-baize/baize-chatbot/tree/main/data\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e100\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/dialogue-%E2%9C%94-lightgreen\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://bair.berkeley.edu/blog/2023/04/03/koala/\"\u003eKoala\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/dialogue-%E2%9C%94-lightgreen\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e✍\u0026nbsp;Human\u003c/p\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://s3.amazonaws.com/static.nomic.ai/gpt4all/2023_GPT4All_Technical_Report.pdf\"\u003eGPT4All\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://huggingface.co/datasets/nomic-ai/gpt4all-j-prompt-generations\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e808\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/dialogue-%E2%9C%94-lightgreen\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e✍\u0026nbsp;Human\u003c/p\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2304.03277.pdf\"\u003eAlpaca-gpt4\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e113\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/bilingual-yellow\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e\n\u003cp\u003e🤖 GPT-4\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003csub\u003e\u003ccode\u003egpt-4\u003c/code\u003e\u003c/sub\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://vicuna.lmsys.org/\"\u003eVicuna\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e76\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/dialogue-%E2%9C%94-lightgreen\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e✍\u0026nbsp;Human\u003c/p\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm\"\u003eDolly\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/databrickslabs/dolly/tree/master/data\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e15\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://drive.google.com/file/d/10iR5hKwFqAKhL3umx8muOWSRm7hs5FqX/view\"\u003eOasst\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://huggingface.co/datasets/OpenAssistant/oasst1\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e84\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/multilingual-red\" alt=\"\" /\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/dialogue-%E2%9C%94-lightgreen\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e✍\u0026nbsp;Human\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2304.08460.pdf\"\u003eLongForm\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/akoksal/LongForm\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e27\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e✍\u0026nbsp;Human\u003c/p\u003e\n\u003cp\u003e🤖 InstructGPT\u003csub\u003e003\u003c/sub\u003e\u003c/p\u003e\n\u003cp\u003e\u003csub\u003e\u003ccode\u003etext-davinci-003\u003c/code\u003e\u003c/sub\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr style=\"height: 18px;\"\u003e\n\u003ctd style=\"height: 18px; width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2304.07995.pdf\"\u003eSymbolic-Instruct\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://huggingface.co/datasets/sail/symbolic-instruction-tuning\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 77.4922px; text-align: right;\"\u003e796\u003c/td\u003e\n\u003ctd style=\"height: 18px; width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center; height: 18px;\"\u003e\n\u003cp\u003e✍\u0026nbsp;Human\u003c/p\u003e\n\u003cp\u003eSynthetic Examples\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2304.14402.pdf\"\u003eLaMini\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://huggingface.co/datasets/MBZUAI/LaMini-instruction\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e2,580\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2304.12244.pdf\"\u003eWizardLM\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/nlpxucan/WizardLM\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e196\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2305.09857.pdf\"\u003eCOEDIT\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e05/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/vipulraheja/coedit\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e82\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e✍\u0026nbsp;Human\u003c/p\u003e\n\u003c!-- \u003cp\u003ecollecting from existing text-editing datasets\u003c/p\u003e --\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2305.14233.pdf\"\u003eUltraChat\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e05/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://huggingface.co/datasets/stingning/ultrachat\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e1,500\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/dialogue-%E2%9C%94-lightgreen\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2305.14045.pdf\"\u003eCoT Collection\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e05/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/kaistAI/CoT-Collection\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e1,060\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e1,880\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e🤖 Codex\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2305.14327.pdf\"\u003eDynosaur\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e05/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://dynosaur-it.github.io/\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e5,740\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e801\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://renzelou.github.io/Muffin/\"\u003eMUFFIN\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e10/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://huggingface.co/datasets/Reza8848/MUFFIN_68k\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e68\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003cp\u003e🤖 GPT-4\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e✍\u0026nbsp;Human\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2310.19651.pdf\"\u003eDynamics-of-Instruction\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e10/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://huggingface.co/datasets/ChiyuSONG/dynamics-of-instruction-tuning\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e40\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e✍\u0026nbsp;Human\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2311.13246.pdf\"\u003eCoachLM\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e11/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/lunyiliu/CoachLM\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e2\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e✍\u0026nbsp;Human\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2312.15685.pdf\"\u003eDEITA\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e12/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://github.com/hkust-nlp/deita\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e10\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/pdf/2312.14187.pdf\"\u003eWaveCoder\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e12/2023\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e4 code-related tasks\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e20\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e🤖 ChatGPT\u003c/p\u003e\n\u003cp\u003e🤖 GPT-4\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd style=\"width: 124.992px; text-align: left;\"\u003e\u003ca href=\"https://arxiv.org/abs/2404.02823\"\u003eConifer\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 61.2891px; text-align: right;\"\u003e\u003cspan style=\"text-decoration: underline;\"\u003e04/2024\u003c/span\u003e\u003c/td\u003e\n\u003ctd style=\"width: 85.1875px; text-align: center;\"\u003e\u003ca href=\"https://huggingface.co/datasets/ConiferLM/Conifer\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n\u003ctd style=\"width: 60.7969px; text-align: right;\"\u003e/\u003c/td\u003e\n\u003ctd style=\"width: 77.4922px; text-align: right;\"\u003e13\u003c/td\u003e\n\u003ctd style=\"width: 109.258px; text-align: center;\"\u003e\u003cp\u003e\u003cimg src=\"https://img.shields.io/badge/monolingual-informational\" alt=\"\" /\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd style=\"width: 124.984px; text-align: center;\"\u003e\n\u003cp\u003e🤖 GPT-4\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003c!-- Some Notes for the alpaca:\n1. The stanford-alpaca paper is not yet published. It is mainly based on the data generation pipline of Self-Instruct, where the main difference is the author uses InstructGPT-3.5 (text-davinci-003) to replace the GPT-3 (davinci). Besides, they also change the prompt, decoding strategy, and remove the cls tasks discrimination.\n2. Alpaca-gpt4 is based on alpaca, the 52k english instructions (w/ optional inputs) are directly collected from alpaca. The main differences are: (a) using ChatGPT to translate 52k English instructions to parallel Chinese instructions (w/ optional inputs); (b) using GPT-4 to replace GPT-3.5 to annotate the outputs of these bilingual instructions; (c) additionally adopting the data generation pipline of Unnatural Instructions with GPT-4 as the annotation model.\n --\u003e\n\n\n\u003c!-- Since I have already displayed the following data-related papers in the table above, I will not list them explicitly here.\n\n1. **Self-Instruct: Aligning Language Model with Self Generated Instructions.** *Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.10560.pdf)]; [[corpus](https://github.com/yizhongw/self-instruct)].\n   \n2. **Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor.** *Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.09689.pdf)]; [[corpus](https://github.com/orhonovich/unnatural-instructions)]. \n   \n3. **Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks.** *Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, and et al.* \u003cins\u003eEMNLP\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2204.07705.pdf)]; [[corpus](https://instructions.apps.allenai.org/)]. \n   \n4. **Cross-Task Generalization via Natural Language Crowdsourcing Instructions.** *Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi.* \u003cins\u003eACL\u003c/ins\u003e 2022. [[pdf](https://aclanthology.org/2022.acl-long.244.pdf)]; [[corpus](https://instructions.apps.allenai.org/)]. \n   \n5. **PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts.** *Stephen Bach, Victor Sanh, Zheng Xin Yong, and et al.* \u003cins\u003eACL\u003c/ins\u003e 2022. [[pdf](https://aclanthology.org/2022.acl-demo.9.pdf)]; [[toolkit](https://github.com/bigscience-workshop/promptsource)]; [[corpus](https://huggingface.co/datasets/bigscience/P3)]. --\u003e\n\n\n\n## 4. 🗂️ Taxonomy\n\nIn our paper, we divide the textual instructions into three categories.\n\n### 4.1 Entailment-oriented Instruction\n\n\u003c!-- Entailment-oriented instruction constructs the task output into the hypothesis and regards the origin task input as the premise. For example, the origin task is to classify `I love this movie` to a `positive` label. While the entailment-oriented paradigm aims to classify whether `Premise: I love this movie` and `Hypothesis: Is it sentiment-positive?` are entailed.  --\u003e\n\n\u003c!-- For example, `Premise: I love this movie` and `Hypothesis: Is it sentiment-positive?` --\u003e\n\n![entailment_oriented](./resources/entailment_oriented.png)\n\nEntailment-oriented instruction regards the task **input** as the **premise**, and constructs the task **output** into the **hypothesis**. It unifies the conventional classification problems into a textual entailment paradigm.\n\n1. **A Universal Discriminator for Zero-Shot Generalization.** *Haike Xu, Zongyu Lin, Jing Zhou, Yanan Zheng, and Zhilin Yang.* \u003cins\u003eACL\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2211.08099.pdf)]; [[code](https://github.com/Rafa-zy/UD)].\n   \n2. **ConEntail: An Entailment-based Framework for Universal Zero and Few Shot Classification with Supervised Contrastive Pretraining.** *Ranran Haoran Zhang, Aysa Xuemo Fan, and Rui Zhang.* \u003cins\u003eEACL\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2210.07587.pdf)]; [[code](https://github.com/psunlpgroup/ConEntail)].\n   \n3. **OpenStance: Real-world Zero-shot Stance Detection.** *Hanzi Xu, Slobodan Vucetic, and Wenpeng Yin.* \u003cins\u003eCoNLL\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2210.14299.pdf)]; [[code](https://github.com/xhz0809/OpenStance)].\n   \n4. **Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference.** *Bangzheng Li, Wenpeng Yin, and Muhao Chen.* \u003cins\u003eTACL\u003c/ins\u003e 2022. [[pdf](https://aclanthology.org/2022.tacl-1.35.pdf)]; [[code](https://github.com/luka-group/lite)]. \n   \n5. **Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learning.** *Oscar Sainz, Itziar Gonzalez-Dios, Oier Lopez de Lacalle, Bonan Min, and Eneko Agirre.* \u003cins\u003eFindings of NAACL\u003c/ins\u003e 2022. [[pdf](https://aclanthology.org/2022.findings-naacl.187.pdf)]; [[code](https://github.com/luka-group/lite)].\n\n6. **Label Verbalization and Entailment for Effective Zero and Few-Shot Relation Extraction.** *Oscar Sainz, Oier Lopez de Lacalle, Gorka Labaka, Ander Barrena, and Eneko Agirre.* \u003cins\u003eEMNLP\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.emnlp-main.92.pdf)]; [[code](https://github.com/osainz59/Ask2Transformers)].\n\n7. **Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections.** *Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein.* \u003cins\u003eFindings of EMNLP\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.findings-emnlp.244.pdf)]; [[code](https://github.com/ruiqi-zhong/Meta-tuning)]. \n   \n8. **Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System.** *Congying Xia, Wenpeng Yin, Yihao Feng, and Philip Yu.* \u003cins\u003eNAACL\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.naacl-main.106.pdf)]; [[code](https://github.com/congyingxia/IncrementalFSTC)].\n   \n9.  **ExpBERT: Representation Engineering with Natural Language Explanations.** *Shikhar Murty, Pang Wei Koh, and Percy Liang.* \u003cins\u003eACL\u003c/ins\u003e 2020. [[pdf](https://aclanthology.org/2020.acl-main.190.pdf)]; [[code](https://github.com/MurtyShikhar/ExpBERT)].\n   \n10. **Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach.** *Wenpeng Yin, Jamaal Hay, Dan Roth* *.* \u003cins\u003eEMNLP\u003c/ins\u003e 2019. [[pdf](https://arxiv.org/pdf/1909.00161.pdf)]; [[website](https://cogcomp.seas.upenn.edu/page/publication_view/883)].\n\n\n### 4.2 PLM-oriented Instruction\n\n![plm_oriented](./resources/PLM_oriented.png)\n\nPLM-oriented instruction (i.e., prompt) aims to construct a cloze-style input to steer pre-trained language models (PLM) for responses. Here, we diaplay several representative works of PLM-oriented instruction learning. For more works, please refer to [this repository](https://github.com/thunlp/PromptPapers) and [this survey](https://dl.acm.org/doi/pdf/10.1145/3560815).\n\n\n1. **How Does In-Context Learning Help Prompt Tuning?** *Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu, and Mohit Iyyer.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.11521.pdf)]. \n   \n2. **Demystifying Prompts in Language Models via Perplexity Estimation.** *Hila Gonen, Srini Iyer, Terra Blevins, Noah A. Smith, and Luke Zettlemoyer.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.04037.pdf)]. \n   \n3. **RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning.** *Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, and et al.* \u003cins\u003eEMNLP\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2205.12548.pdf)]; [[code](https://github.com/mingkaid/rl-prompt)]. \n   \n4. **PPT: Pre-trained Prompt Tuning for Few-shot Learning.** *Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang.* \u003cins\u003eACL\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2109.04332.pdf)]; [[code](https://github.com/thu-coai/PPT)]. \n   \n5. **P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks.** *Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang.* \u003cins\u003eACL\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2110.07602.pdf)]; [[code](https://github.com/THUDM/P-tuning-v2)].\n   \n6. **KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction.** *Xiang Chen, Ningyu Zhang, Xin Xie, and et al.* \u003cins\u003eWWW\u003c/ins\u003e 2022. [[pdf](http://128.84.21.203/pdf/2104.07650)]; [[code](https://github.com/zjunlp/KnowPrompt)].\n   \n7. **GPT Understands, Too.** *Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang.* \u003cins\u003ePreprint\u003c/ins\u003e 2021. [[pdf](https://arxiv.org/pdf/2103.10385.pdf)]; [[code](https://github.com/THUDM/P-tuning)].\n   \n8.  **Few-Shot Text Generation with Natural Language Instructions.** *Timo Schick and Hinrich Schütze.* \u003cins\u003eEMNLP\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.emnlp-main.32.pdf)]; [[code](https://github.com/timoschick/pet)]. \n   \n9.  **It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners.** *Timo Schick and Hinrich Schütze.* \u003cins\u003eNAACL\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.naacl-main.185.pdf)]; [[code](https://github.com/timoschick/pet)]. \n   \n10. **Learning How to Ask: Querying LMs with Mixtures of Soft Prompts.** *Guanghui Qin and Jason Eisner.* \u003cins\u003eNAACL\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.naacl-main.410.pdf)]; [[code](https://github.com/hiaoxui/soft-prompts)]. \n   \n11. **Prefix-Tuning: Optimizing Continuous Prompts for Generation.** *Xiang Lisa Li and Percy Liang.* \u003cins\u003eACL\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.acl-long.353.pdf)]; [[code](https://github.com/XiangLi1999/PrefixTuning)]. \n   \n12. **Making Pre-trained Language Models Better Few-shot Learners.** *Tianyu Gao, Adam Fisch, and Danqi Chen.* \u003cins\u003eACL\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.acl-long.295.pdf)]; [[code](https://github.com/princeton-nlp/LM-BFF)]. \n   \n13. **Template-Based Named Entity Recognition Using BART.** *Leyang Cui, Yu Wu, Jian Liu, Sen Yang, and Yue Zhang.* \u003cins\u003eFindings of ACL\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.findings-acl.161.pdf)]; [[code](https://github.com/Nealcly/templateNER)]. \n   \n14. **Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference.** *Timo Schick and Hinrich Schütze.* \u003cins\u003eEACL\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.eacl-main.20.pdf)]; [[code](https://github.com/timoschick/pet)].\n   \n15. **Language Models are Unsupervised Multitask Learners.** *Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever.* \u003cins\u003ePreprint\u003c/ins\u003e 2019. [[pdf](https://life-extension.github.io/2020/05/27/GPT%E6%8A%80%E6%9C%AF%E5%88%9D%E6%8E%A2/language-models.pdf)]. \n\n\n### 4.3 Human-oriented Instruction\n\n\u003c!-- **Paper Title.** *Author 1, Author 2, and Author 3.* \u003cins\u003eConference/Journal/Preprint\u003c/ins\u003e Year. [[pdf](link)]; [[other resources](link)]. --\u003e\n\n![Human-oriented Instruction](./resources/human_oriented.png)\n\nHuman-oriented instruction is initially designed for human to understand the task and annotate the data, such as the [Amazon MTurk](https://www.mturk.com/) Instructions, which provides sufficient information about the task (e.g., detailed definition).\n   \n1. **Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors.** *Kai Zhang, Bernal Jiménez Gutiérrez, and Yu Su.* \u003cins\u003eFindings of ACL\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2305.11159.pdf)]; [[code](https://github.com/OSU-NLP-Group/QA4RE)].\n   \n2. **Symbol tuning improves in-context learning in language models.** *Jerry Wei, Le Hou, Andrew Lampinen, Xiangning Chen, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2305.08298.pdf)].\n   \n3. **Small Models are Valuable Plug-ins for Large Language Models.** *Canwen Xu, Yichong Xu, Shuohang Wang, Yang Liu, Chenguang Zhu, and Julian McAuley.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2305.08848.pdf)]; [[code](https://github.com/JetRunner/SuperICL)].\n   \n4. **How Many Data Samples is an Additional Instruction Worth?** *Ravsehaj Singh Puri, Swaroop Mishra, Mihir Parmar, and Chitta Baral.* \u003cins\u003eFindings of EACL\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2203.09161.pdf)]; [[code](https://github.com/Ravsehajsinghpuri/Multi-Variant-Instructions)].\n   \n5. **In-Context Instruction Learning.** *Seonghyeon Ye, Hyeonbin Hwang, Sohee Yang, Hyeongu Yun, Yireun Kim, and Minjoon Seo.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.14691.pdf)]; [[code](https://github.com/seonghyeonye/ICIL)]. \n   \n6. **InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis.** *Kevin Scaria, Himanshu Gupta, Saurabh Arjun Sawant, Swaroop Mishra, and Chitta Baral.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.08624.pdf)]; [[code](https://github.com/kevinscaria/InstructABSA)].\n   \n7. **HINT: Hypernetwork Instruction Tuning for Efficient Zero-Shot Generalisation.** *Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, and Matthew Peters.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.10315.pdf)].\n\n8. **Boosting Natural Language Generation from Instructions with Meta-Learning.** *Budhaditya Deb, Guoqing Zheng, and Ahmed Hassan Awadallah.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2210.11617.pdf)]. \n   \n9.  **GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models.** *Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2203.07281.pdf)]; [[code](https://github.com/archiki/GrIPS)].\n   \n10. **ConTinTin: Continual Learning from Task Instructions.** *Wenpeng Yin, Jia Li, and Caiming Xiong.* \u003cins\u003eACL\u003c/ins\u003e 2022. [[pdf](https://aclanthology.org/2022.acl-long.218.pdf)]. \n   \n11. **InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning.** *Prakhar Gupta, Cathy Jiao, Yi-Ting Yeh, Shikib Mehri, Maxine Eskenazi, and Jeffrey P. Bigham.* \u003cins\u003eEMNLP\u003c/ins\u003e 2022. [[pdf]([link](http://128.84.21.203/pdf/2205.12673))]; [[code](https://github.com/prakharguptaz/Instructdial)]. \n   \n12. **Learning to Generate Task-Specific Adapters from Task Description.** *Qinyuan Ye and Xiang Ren.* \u003cins\u003eACL\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.acl-short.82.pdf)]; [[code](https://github.com/INK-USC/hypter)]. \u003c!-- TODO --\u003e\n   \n13. **The Turking Test: Can Language Models Understand Instructions?** *Avia Efrat and Omer Levy.* \u003cins\u003ePreprint\u003c/ins\u003e 2020. [[pdf](https://arxiv.org/pdf/2010.11982.pdf)]. \n\n\n## 5. 📊 Analyses\n\n### 5.1 Scale\nThe model and task scale are found to be important for instruction-based fine-tuning. Basically, the larger model scale brings more benefits to the generalization, and so does the task scale. However, some works raised objections (e.g., [Jang et al.](https://arxiv.org/pdf/2302.03202.pdf) and [Wang et al.](https://arxiv.org/pdf/2210.00185.pdf)).\n\n\u003c!-- **Paper Title.** *Author 1, Author 2, and Author 3.* \u003cins\u003eConference/Journal/Preprint\u003c/ins\u003e Year. [[pdf](link)]; [[other resources](link)].  --\u003e\n   \n1. **Exploring the Benefits of Training Expert Language Models over Instruction Tuning.** *Joel Jang, Seungone Kim, Seonghyeon Ye, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.03202.pdf)]; [[code](https://github.com/joeljang/ELM)]. \n   \n2. **The Flan Collection: Designing Data and Methods for Effective Instruction Tuning.** *Shayne Longpre, Le Hou, Tu Vu, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2301.13688.pdf)]; [[code](https://github.com/google-research/FLAN/tree/main/flan/v2)]; [[corpus](https://huggingface.co/datasets/SirNeural/flan_v2)].\n   \n3. **UL2: Unifying Language Learning Paradigms.** *Yi Tay, Mostafa Dehghani, Vinh Q. Tran, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2205.05131.pdf)]; [[checkpoint](https://huggingface.co/google/flan-ul2)].\n   \n4. **OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization.** *Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.12017.pdf)].   \n   \n5. **Scaling Instruction-Finetuned Language Models.** *Hyung Won Chung, Le Hou, Shayne Longpre, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2210.11416.pdf)]; [[checkpoint](https://huggingface.co/docs/transformers/model_doc/flan-t5)]. \n   \n6. **Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization.** *Yuxian Gu, Pei Ke, Xiaoyan Zhu, and Minlie Huang.* \u003cins\u003eEMNLP\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2210.09175.pdf)]; [[code](https://github.com/thu-coai/UDIT)]. \n   \n7. **Emergent Abilities of Large Language Models.** *Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, and et al.* \u003cins\u003eTMLR\u003c/ins\u003e 2022. [[pdf](https://openreview.net/pdf?id=yzkSU5zdwD)].\n   \n8.  **Multitask Prompted Training Enables Zero-Shot Task Generalization.** *Victor Sanh, Albert Webson, Colin Raffel, and et al.* \u003cins\u003eICLR\u003c/ins\u003e 2022. [[pdf](https://openreview.net/pdf?id=9Vrb9D0WI4)]; [[checkpoint](https://github.com/bigscience-workshop/t-zero)]; [[corpus](https://github.com/bigscience-workshop/promptsource)]. \n   \n9.  **Finetuned Language Models are Zero-Shot Learners.** *Jason Wei, Maarten Bosma, Vincent Zhao, and et al.* \u003cins\u003eICLR\u003c/ins\u003e 2022. [[pdf](https://openreview.net/pdf?id=gEZrGCozdqR)]; [[code](https://github.com/google-research/flan)].\n    \n10. **Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks.** *Zhenhailong Wang, Xiaoman Pan, Dian Yu, Dong Yu, Jianshu Chen, and Heng Ji.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2210.00185.pdf)]; [[code](https://github.com/MikeWangWZHL/Zemi)].  \n    \n11. **ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization.** *Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang, Haiyu Li, and Zhilin Yang.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2201.06910.pdf)]. \n    \n12. **The Power of Scale for Parameter-Efficient Prompt Tuning.** *Brian Lester, Rami Al-Rfou, and Noah Constant.* \u003cins\u003eEMNLP\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.emnlp-main.243.pdf)]; [[code](https://github.com/google-research/prompt-tuning)]. \n\n### 5.2 Explanability\n\nWe exhibit works that focus on the interpretability and reliability of instruction learning, i.e., explaining *when* and *why* instruction can take effect.\n\n\u003c!-- **Paper Title.** *Author 1, Author 2, and Author 3.* \u003cins\u003eConference/Journal/Preprint\u003c/ins\u003e Year. [[pdf](link)]; [[other resources](link)]. --\u003e\n   \n1. **What In-Context Learning \"Learns\" In-Context: Disentangling Task Recognition and Task Learning.** *Jane Pan, Tianyu Gao, Howard Chen, and Danqi Chen.* \u003cins\u003eFindings of ACL\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2305.09731.pdf)]; [[code](https://github.com/princeton-nlp/WhatICLLearns)].\n   \n2. **REV: Information-Theoretic Evaluation of Free-Text Rationales.** *Hanjie Chen, Faeze Brahman, Xiang Ren, and et al.* \u003cins\u003eACL\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2210.04982.pdf)]; [[code](https://github.com/HanjieChen/REV)].\n   \n3. **Interpretability at Scale: Identifying Causal Mechanisms in Alpaca.** *Zhengxuan Wu, Atticus Geiger, Christopher Potts, and Noah D. Goodman.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2305.08809.pdf)]; [[code](https://github.com/frankaging/align-transformers)].\n   \n4. **Large Language Models Are Implicitly Topic Models: Explaining and Finding Good Demonstrations for In-Context Learning.** *Xinyi Wang, Wanrong Zhu, Michael Saxon, Mark Steyvers, and William Yang Wang.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2301.11916.pdf)]; [[code](https://github.com/WANGXinyiLinda/concept-based-demonstration-selection)].\n   \n5. **The Learnability of In-Context Learning.** *Noam Wies, Yoav Levine, and Amnon Shashua.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2303.07895.pdf)].\n   \n6. **Why think step-by-step? Reasoning emerges from the locality of experience.** *Ben Prystawski, and Noah D. Goodman.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2304.03843.pdf)].\n   \n7. **Larger language models do in-context learning differently.** *Jerry Wei, Jason Wei, Yi Tay, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2303.03846.pdf)].\n   \n8. **​​What learning algorithm is in-context learning? Investigations with linear models.** *Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou.* \u003cins\u003eICLR\u003c/ins\u003e 2023. [[pdf](https://openreview.net/pdf?id=0g0X4H8yN4I)]; [[code](https://github.com/ekinakyurek/google-research/tree/master/incontext)].\n   \n9.  **Can language models learn from explanations in context?** *Andrew K. Lampinen, Ishita Dasgupta, Stephanie C. Y. Chan, and et al.* \u003cins\u003eFindings of EMNLP\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2204.02329.pdf)]. \n   \n10. **Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?** *Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer.* \u003cins\u003eEMNLP\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2202.12837.pdf)]; [[code](https://github.com/Alrope123/rethinking-demonstrations)]. \n   \n11. **Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts.** *Daniel Khashabi, Xinxi Lyu, Sewon Min, and et al.* \u003cins\u003eNAACL\u003c/ins\u003e 2022. [[pdf](https://aclanthology.org/2022.naacl-main.266.pdf)]; [[code](https://github.com/Alrope123/prompt-waywardness)]. \n   \n12. **Do Prompt-Based Models Really Understand the Meaning of Their Prompts?.** *Albert Webson and Ellie Pavlick.* \u003cins\u003eNAACL\u003c/ins\u003e 2022. [[pdf](https://aclanthology.org/2022.naacl-main.167.pdf)]; [[code](https://github.com/awebson/prompt_semantics)].\n   \n13. **Reframing Instructional Prompts to GPTk’s Language.** *Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi.* \u003cins\u003eFindings of ACL\u003c/ins\u003e 2022. [[pdf](https://aclanthology.org/2022.findings-acl.50.pdf)]; [[code](https://github.com/allenai/reframing/)]. \n   \n14. **What Makes Good In-Context Examples for GPT-3?** *Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen.* \u003cins\u003eACL Workshop\u003c/ins\u003e 2022. [[pdf](https://aclanthology.org/2022.deelio-1.10.pdf)]; [[code](https://github.com/jiachangliu/KATEGPT3)]. \n   \n15. **Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity.** *Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp.* \u003cins\u003eACL\u003c/ins\u003e 2022. [[pdf](https://aclanthology.org/2022.acl-long.556.pdf)].\n   \n16. **Calibrate Before Use: Improving Few-shot Performance of Language Models.** *Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh.* \u003cins\u003eICML\u003c/ins\u003e 2021. [[pdf](https://arxiv.org/pdf/2102.09690.pdf)]; [[code](https://github.com/tonyzhaozh/few-shot-learning)].\n\n### 5.3 Robustness and Safety\n\n\u003c!-- **Paper Title.** *Author 1, Author 2, and Author 3.* \u003cins\u003eConference/Journal/Preprint\u003c/ins\u003e Year. [[pdf](link)]; [[other resources](link)]. --\u003e\n   \n1. **Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection.** *Jun Yan, Vikas Yadav, Shiyang Li, and et al.* \u003cins\u003eWorkshop @ NeurIPS\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/abs/2307.16888)].\n   \n2. **Evaluating the Zero-shot Robustness ofInstruction-tuned Language Models.** *Jiuding Sun, Chantal Shaib, and Byron C. Wallace.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2306.11270.pdf)].\n   \n3. **Poisoning Language Models During Instruction Tuning.** *Alexander Wan, Eric Wallace, Sheng Shen, and Dan Klein.* \u003cins\u003eICML\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2305.00944.pdf)]; [[code](https://github.com/AlexWan0/Poisoning-Instruction-Tuned-Models)].\n   \n4. **Multi-step Jailbreaking Privacy Attacks on ChatGPT.** *Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2304.05197.pdf)].\n   \n5. **More than you've asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models.** *Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.12173.pdf)]; [[code](https://github.com/greshake/llm-security)]. \n   \n6. **Robustness of Learning from Task Instructions.** *Jiasheng Gu, Hanzi Xu, Liangyu Nie, and Wenpeng Yin.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.03813.pdf)]. \n\n7. **Learning from Task Descriptions.** *Orion Weller, Nicholas Lourie, Matt Gardner, and Matthew E. Peters.* \u003cins\u003eEMNLP\u003c/ins\u003e 2020. [[pdf](https://aclanthology.org/2020.emnlp-main.105.pdf)]; [[code](https://github.com/allenai/zest)]; [[corpus](https://allenai.org/data/zest)]. \n\n\n### 5.4 Evaluation\nStop using old-school automatic metrics to evaluate your instruction-tuned system; try more advanced methods to do it comprehensively!\n\n1. **Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2.** *Hamish Ivison, Yizhong Wang, Valentina Pyatkin, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2311.10702.pdf)]; [[model\u0026data](https://huggingface.co/collections/allenai/tulu-v2-suite-6551b56e743e6349aab45101)]\n   \n2. **How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources.** *Yizhong Wang, Hamish Ivison, Pradeep Dasigi, and et al.* \u003cins\u003eNeurIPS Datasets and Benchmarks\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2306.04751.pdf)]; [[code](https://github.com/allenai/open-instruct)].\n\n3. **Instruction-following Evaluation through Verbalizer Manipulation.** *Shiyang Li, Jun Yan, Hai Wang, Zheng Tang, Xiang Ren, Vijay Srinivasan, Hongxia Jin* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2307.10558.pdf)].\n   \n4. **INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models.** *Yew Ken Chia, Pengfei Hong, Lidong Bing, and Soujanya Poria.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2306.04757.pdf)]; [[code](https://github.com/declare-lab/instruct-eval)]; [[leaderboard](https://declare-lab.net/instruct-eval/)].\n\n\n### 5.5 Negation\n\nNegation expressions, such as `do not` and `avoid doing`, are difficult for models to corretly understand and follow.\n\n1. **Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts.** *Joel Jang, Seonghyeon Ye, and Minjoon Seo.* \u003cins\u003eICML Workshop\u003c/ins\u003e 2023. [[pdf](https://proceedings.mlr.press/v203/jang23a/jang23a.pdf)].\n   \n2. **Understanding by Understanding Not: Modeling Negation in Language Models.** *Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, and et al.* \u003cins\u003eNAACL\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.naacl-main.102.pdf)]; [[code](https://github.com/arianhosseini/negation-learning)]. \n\n\n### 5.6 Complexity\n\n\u003c!-- **Paper Title.** *Author 1, Author 2, and Author 3.* \u003cins\u003eConference/Journal/Preprint\u003c/ins\u003e Year. [[pdf](link)]; [[other resources](link)]. --\u003e\nPapers are focusing on enhancing the complexity of instructions to enhance model competence. More complex data in the mix of instruction data, more competent performance model could achieve.\n\n1. **Wizardlm: Empowering large language models to follow complex instructions.** *Xu, Can and Sun, Qingfeng and Zheng, Kai and Geng, Xiubo and Zhao, Pu and Feng, Jiazhan and Tao, Chongyang and Jiang, Daxin*. \u003cins\u003ePrepint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2304.12244.pdf)]; [[code](https://github.com/nlpxucan/WizardLM)]. \n\n2. **Orca: Progressive learning from complex explanation traces of gpt-4.** *Mukherjee, Subhabrata and Mitra, Arindam and Jawahar, Ganesh and Agarwal, Sahaj and Palangi, Hamid and Awadallah, Ahmed*. \u003cins\u003ePrepint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2306.02707.pdf)].\n\n3. **A Preliminary Study of the Intrinsic Relationship between Complexity and Alignment.** *Zhao, Yingxiu and Yu, Bowen and Hui, Binyuan and Yu, Haiyang and Huang, Fei and Li, Yongbin and Zhang, Nevin L*. \u003cins\u003ePrepint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2308.05696.pdf)]; [[code](https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/tree-instruct)].\n\n### 5.7 Other Papers\n\n\u003c!-- **Paper Title.** *Author 1, Author 2, and Author 3.* \u003cins\u003eConference/Journal/Preprint\u003c/ins\u003e Year. [[pdf](link)]; [[other resources](link)]. --\u003e\n   \n1. **Don't Blame the Annotator: Bias Already Starts in the Annotation Instructions.** *Mihir Parmar, Swaroop Mishra, Mor Geva, and Chitta Baral.* \u003cins\u003eEACL\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2205.00415.pdf)]; [[code](https://github.com/Mihir3009/instruction-bias)].\n2. **Instruction Tuned Models are Quick Learners.** *Himanshu Gupta, Saurabh Arjun Sawant, Swaroop Mishra, et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2306.05539.pdf)]; [[code](https://github.com/srsawant34/efficient_instruction_learning)].\n3. **Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning.** *Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin Raffel.* \u003cins\u003eNeurIPS\u003c/ins\u003e 2022. [[pdf](https://openreview.net/pdf?id=rBCvMG-JsPd)]; [[code](https://github.com/r-three/t-few)]. \n4. **A Survey of NLP-Related Crowdsourcing HITs: what works and what does not.** *Jessica Huynh, Jeffrey Bigham, and Maxine Eskenazi.* \u003cins\u003ePreprint\u003c/ins\u003e 2021. [[pdf](https://arxiv.org/pdf/2111.05241.pdf)].\n   \n\n   \n\n   \n\n\n## 6. 🤖 Applications\n\n### 6.1 Human-Computer Interaction\n\nInstructions are used in various human-computer interaction (HCI) tasks, such as virtual assistants, chatbots, etc. \n\n\n1. **Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing.** *Tuhin Chakrabarty, Vishakh Padmakumar, and He He.* \u003cins\u003eEMNLP\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2210.13669.pdf)]; [[code](https://github.com/vishakhpk/creative-instructions)]. \n   \n2. **HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models.** *Swaroop Mishra, and Elnaz Nouri.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2208.08232.pdf)]. \n   \n3. **EditEval: An Instruction-Based Benchmark for Text Improvements.** *Jane Dwivedi-Yu, Timo Schick, Zhengbao Jiang, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2209.13331.pdf)]; [[code](https://github.com/facebookresearch/EditEval)]; [[website](https://eval.ai/web/challenges/challenge-page/1866/overview)].\n   \n4. **Communicating Natural Programs to Humans and Machines.** *Sam Acquaviva, Yewen Pu, Marta Kryven, and et al.* \u003cins\u003eNeurIPS Workshop\u003c/ins\u003e 2022. [[pdf](https://openreview.net/pdf?id=OxFoLTKDcNm)]; [[code](https://github.com/samacqua/LARC)]. \n   \n5. **Interactive Task Learning from GUI-Grounded Natural Language Instructions and Demonstrations.** *Toby Jia-Jun Li, Tom Mitchell, and Brad Myers.* \u003cins\u003eACL Demo\u003c/ins\u003e 2020. [[pdf](https://aclanthology.org/2020.acl-demos.25.pdf)]; [[code](https://github.com/tobyli/Sugilite_development)]; [[video](https://www.youtube.com/watch?v=tdHEk-GeaqE)].\n   \n6. **Multi-Modal Interactive Task Learning from Demonstrations and Natural Language Instructions.** *Toby Jia-Jun Li.* \u003cins\u003eUIST\u003c/ins\u003e 2020. [[pdf](https://dl.acm.org/doi/pdf/10.1145/3379350.3415803)]; [[code](https://github.com/tobyli/Sugilite_development)].\n   \n7. **Pre-Learning Environment Representations for Data-Efficient Neural Instruction Following.** *David Gaddy, and Dan Klein.* \u003cins\u003eACL\u003c/ins\u003e 2019. [[pdf](https://aclanthology.org/P19-1188.pdf)]. \n   \n8. **VirtualHome: Simulating Household Activities via Programs.** *Xavier Puig, Kevin Ra, Marko Boben, and et al.* \u003cins\u003eCVPR\u003c/ins\u003e 2018. [[pdf](https://openaccess.thecvf.com/content_cvpr_2018/papers/Puig_VirtualHome_Simulating_Household_CVPR_2018_paper.pdf)]; [[website](http://virtual-home.org/)]. \n   \n9.  **Natural Language Communication with Robots.** *Yonatan Bisk, Deniz Yuret, and Daniel Marcu.* \u003cins\u003eNAACL\u003c/ins\u003e 2016. [[pdf](https://aclanthology.org/N16-1089.pdf)]; [[website](https://groundedlanguage.github.io/)].\n    \n10. **Jointly Learning to Parse and Perceive: Connecting Natural Language to the Physical World.** *Jayant Krishnamurthy, and Thomas Kollar.* \u003cins\u003eTACL\u003c/ins\u003e 2013. [[pdf](http://rtw.ml.cmu.edu/tacl2013_lsp/tacl2013-krishnamurthy-kollar.pdf)]; [[code](http://rtw.ml.cmu.edu/tacl2013_lsp/)]. \n\n11. **Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions.** *Yoav Artzi, and Luke Zettlemoyer.* \u003cins\u003eTACL\u003c/ins\u003e 2013. [[pdf](https://aclanthology.org/Q13-1005.pdf)].\n    \n12. **Unsupervised PCFG Induction for Grounded Language Learning with Highly Ambiguous Supervision.** *Joohyun Kim, and Raymond Mooney.* \u003cins\u003eEMNLP\u003c/ins\u003e 2012. [[pdf](https://aclanthology.org/D12-1040.pdf)].\n    \n13. **A joint model of language and perception for grounded attribute learning.** *Cynthia Matuszek, Nicholas FitzGerald, Luke Zettlemoyer, Liefeng Bo, and Dieter Fox.* \u003cins\u003eICML\u003c/ins\u003e 2012. [[pdf](https://arxiv.org/pdf/1206.6423.pdf)]. \n    \n14. **Learning to Interpret Natural Language Instructions.** *Monica Babeş-Vroman, James MacGlashan, Ruoyuan Gao, and et al.* \u003cins\u003eACL Workshop\u003c/ins\u003e 2012. [[pdf](https://aclanthology.org/W12-2801.pdf)]. \n    \n15. **Fast Online Lexicon Learning for Grounded Language Acquisition.** *David Chen.* \u003cins\u003eACL\u003c/ins\u003e 2012. [[pdf](https://aclanthology.org/P12-1045.pdf)].\n    \n16. **Learning to Win by Reading Manuals in a Monte-Carlo Framework.** *S.R.K. Branavan, David Silver, and Regina Barzilay.* \u003cins\u003eACL\u003c/ins\u003e 2011. [[pdf](https://aclanthology.org/P11-1028.pdf)]; [[website](http://groups.csail.mit.edu/rbg/code/civ/)].\n    \n17. **Learning from natural instructions.** *Dan Goldwasse, and Dan Roth.* \u003cins\u003eIJCAI\u003c/ins\u003e 2011. [[pdf](https://citeseerx.ist.psu.edu/document?repid=rep1\u0026type=pdf\u0026doi=2aba84801935041774c1e2b749e0331efa322ed8)].  \n    \n18. **Learning to Interpret Natural Language Navigation Instructions from Observations.** *David L. Chen and Raymond J. Mooney.* \u003cins\u003eAAAI\u003c/ins\u003e 2011. [[pdf](https://www.cs.utexas.edu/users/ml/papers/chen.aaai11.pdf)]. \n    \n19. **Approaching the Symbol Grounding Problem with Probabilistic Graphical Models.** *Stefanie Tellex, Thomas Kollar, Steven Dickerson, and et al.* \u003cins\u003eAAAI\u003c/ins\u003e 2011. [[pdf](https://cs.brown.edu/people/stellex/publications/tellex11a.pdf)]. \n    \n20. **Driving Semantic Parsing from the World’s Response.** *James Clarke, Dan Goldwasser, Ming-Wei Chang, and Dan Roth.* \u003cins\u003eCoNLL\u003c/ins\u003e 2010. [[pdf](https://aclanthology.org/W10-2903.pdf)]. \n    \n21. **Learning to Follow Navigational Directions.** *Adam Vogel, and Daniel Jurafsky.* \u003cins\u003eACL\u003c/ins\u003e 2010. [[pdf](https://aclanthology.org/P10-1083.pdf)].\n    \n22. **Reading between the Lines: Learning to Map High-Level Instructions to Commands.** *S.R.K. Branavan, Luke Zettlemoyer, and Regina Barzilay.* \u003cins\u003eACL\u003c/ins\u003e 2010. [[pdf](https://aclanthology.org/P10-1129.pdf)]; [[website](http://groups.csail.mit.edu/rbg/code/rl-hli/)]. \n    \n23. **Reading to Learn: Constructing Features from Semantic Abstracts.** *Jacob Eisenstein, James Clarke, Dan Goldwasser, and Dan Roth.* \u003cins\u003eEMNLP\u003c/ins\u003e 2009. [[pdf](https://aclanthology.org/D09-1100.pdf)]; [[website](http://www.comlab.ox.ac.uk/activities/machinelearning/Aleph/)]. \n    \n24. **Learning Semantic Correspondences with Less Supervision.** *Percy Liang, Michael Jordan, and Dan Klein.* \u003cins\u003eACL\u003c/ins\u003e 2009. [[pdf](https://aclanthology.org/P09-1011.pdf)]. \n    \n25. **Reinforcement Learning for Mapping Instructions to Actions.** *S.R.K. Branavan, Harr Chen, Luke Zettlemoyer, and Regina Barzilay.* \u003cins\u003eACL\u003c/ins\u003e 2009. [[pdf](https://aclanthology.org/P09-1010.pdf)]; [[website](http://groups.csail.mit.edu/rbg/code/rl/)]. \n    \n26. **Learning to sportscast: a test of grounded language acquisition.** *David L. Chen and Raymond J. Mooney.* \u003cins\u003eICML\u003c/ins\u003e 2008. [[pdf](https://dl.acm.org/doi/pdf/10.1145/1390156.1390173)]. \n    \n27. **Guiding a Reinforcement Learner with Natural Language Advice: Initial Results in RoboCup Soccer.** *Gregory Kuhlmann, Peter Stone, Raymond Mooney, and Jude Shavlik.* \u003cins\u003eAAAI Workshop\u003c/ins\u003e 2004. [[pdf](https://ftp.cs.wisc.edu/machine-learning/shavlik-group/kuhlmann-aaai04.pdf)]; [[website](http://www.cs.utexas.edu/AustinVilla/sim/keepaway/)]. \n\n\n### 6.2 Data and Feature Augmentation\n\nSome instructions (e.g., label explanations) are also be used for automatic annotation (i.e., data augmentation), or for enriching feature.\n\n1. **One Embedder, Any Task: Instruction-Finetuned Text Embeddings.** *Hongjin Su, Weijia Shi, Jungo Kasai, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.09741.pdf)]; [[website](https://instructor-embedding.github.io/)]. \n   \n2. **Prompt Consistency for Zero-Shot Task Generalization.** *Chunting Zhou, Junxian He, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig.* \u003cins\u003eFindings of EMNLP\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2205.00049.pdf)]; [[code](https://github.com/violet-zct/swarm-distillation-zero-shot)]. \n   \n3. **Teaching Machine Comprehension with Compositional Explanations.** *Qinyuan Ye, Xiao Huang, Elizabeth Boschee, and Xiang Ren.* \u003cins\u003eFindings of EMNLP\u003c/ins\u003e 2020. [[pdf](https://aclanthology.org/2020.findings-emnlp.145.pdf)]; [[code](https://github.com/INK-USC/mrc-explanation)]. \n   \n4. **Learning from Explanations with Neural Execution Tree.** *Ziqi Wang, Yujia Qin, Wenxuan Zhou, Jun Yan, Qinyuan Ye, Leonardo Neves, Zhiyuan Liu, and Xiang Ren.* \u003cins\u003eICLR\u003c/ins\u003e 2020. [[pdf](https://openreview.net/pdf?id=rJlUt0EYwS)]; [[website](http://inklab.usc.edu/project-NExT/)]. \n   \n5. **Training Classifiers with Natural Language Explanations.** *Braden Hancock, Paroma Varma, Stephanie Wang, Martin Bringmann, Percy Liang, and Christopher Ré.* \u003cins\u003eACL\u003c/ins\u003e 2018. [[pdf](https://aclanthology.org/P18-1175.pdf)]; [[code](https://github.com/HazyResearch/babble)]. \n   \n6. **Zero-shot Learning of Classifiers from Natural Language Quantification.** *Shashank Srivastava, Igor Labutov, and Tom Mitchell.* \u003cins\u003eACL\u003c/ins\u003e 2018. [[pdf](https://aclanthology.org/P18-1029.pdf)]. \n   \n7. **Joint Concept Learning and Semantic Parsing from Natural Language Explanations.** *Shashank Srivastava, Igor Labutov, and Tom Mitchell.* \u003cins\u003eEMNLP\u003c/ins\u003e 2017. [[pdf](https://aclanthology.org/D17-1161.pdf)]. \n\n### 6.3 General-purpose Language Models\n\nGeneral-purpose language models are also one of the most attractive applications of instruction learning, e.g., [ChatGPT](https://chat.openai.com/chat), which can align nicely with human values.\n\n\n1. **Sparks of Artificial General Intelligence: Early experiments with GPT-4.** *Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2303.12712.pdf)]. \n   \n2. **GPT-4 Technical Report.** *OpenAI.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://cdn.openai.com/papers/gpt-4.pdf)]; [[blog](https://openai.com/research/gpt-4)].  \n   \n3. **The Wisdom of Hindsight Makes Language Models Better Instruction Followers.** *Tianjun Zhang, Fangchen Liu, Justin Wong, Pieter Abbeel, and Joseph E. Gonzalez.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.05206.pdf)]; [[code](https://github.com/tianjunz/HIR)].\n    \n4. **Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models.** *Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.07388.pdf)]. \n   \n5. **Training language models to follow instructions with human feedback.** *Long Ouyang, Jeffrey Wu, Xu Jiang, and et al.* \u003cins\u003eNeurIPS\u003c/ins\u003e 2022. [[pdf](https://openreview.net/pdf?id=TG8KACxEON)]. \n\n\n### 6.4 Other Papers\n\n\u003c!-- **Paper Title.** *Author 1, Author 2, and Author 3.* \u003cins\u003eConference/Journal/Preprint\u003c/ins\u003e Year. [[pdf](link)]; [[other resources](link)]. --\u003e\n1. **GPTScore: Evaluate as You Desire.** *Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.04166.pdf)]; [[code](https://github.com/jinlanfu/GPTScore)]. \n   \n2. **MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning.** *Zhiyang Xu, Ying Shen, and Lifu Huang.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.10773.pdf)].\n   \n3. **Task-aware Retrieval with Instructions.** *Akari Asai, Timo Schick, Patrick Lewis, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2211.09260.pdf)]; [[code](https://github.com/facebookresearch/tart)]. \n   \n4. **UnifiedABSA: A Unified ABSA Framework Based on Multi-task Instruction Tuning.** *Zengzhi Wang, Rui Xia, and Jianfei Yu.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2211.10986.pdf)].  \n \n5. **In-Context Learning for Few-Shot Dialogue State Tracking.** *Yushi Hu, Chia-Hsuan Lee, Tianbao Xie, Tao Yu, Noah A. Smith, and Mari Ostendorf.* \u003cins\u003eFindings of EMNLP\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2203.08568.pdf)]; [[code](https://github.com/Yushi-Hu/IC-DST)]. \n   \n6. **Few-shot Learning with Multilingual Language Models.** *Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, and et al.* \u003cins\u003eEMNLP\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2112.10668.pdf)]; [[code](https://github.com/facebookresearch/fairseq/tree/main/examples/xglm)].\n   \n7. **UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models.** *Tianbao Xie, Chen Henry Wu, Peng Shi, and et al.* \u003cins\u003eEMNLP\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2201.05966.pdf)]; [[code](https://github.com/HKUNLP/UnifiedSKG)]; [[website](https://unifiedskg.com/)]. \n   \n8. **In-BoXBART: Get Instructions into Biomedical Multi-Task Learning .** *Mihir Parmar, Swaroop Mishra, Mirali Purohit, Man Luo, M. Hassan Murad, and Chitta Baral.* \u003cins\u003eFindings of NAACL\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2204.07600.pdf)]; [[code](https://github.com/Mihir3009/In-BoXBART)].\n\n\n\n## 7. 📖 Extended Reading\n\n\nWe also share some other awesome papers that might inspire the future work.\n\n### 7.1 Instruction Induction\n\n   \n1. **Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners.** *Seonghyeon Ye, Doyoung Kim, Joel Jang, Joongbo Shin, and Minjoon Seo.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2210.02969.pdf)]; [[code](https://github.com/seonghyeonye/Flipped-Learning)]. \n   \n2. **Instruction Induction: From Few Examples to Natural Language Task Descriptions.** *Or Honovich, Uri Shaham, Samuel R. Bowman, and Omer Levy.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2205.10782.pdf)]; [[code](https://github.com/orhonovich/instruction-induction)].\n   \n3. **Learning to Decompose and Organize Complex Tasks.** *Yi Zhang, Sujay Kumar Jauhar, Julia Kiseleva, Ryen White, and Dan Roth.* \u003cins\u003eNAACL\u003c/ins\u003e 2021. [[pdf](https://aclanthology.org/2021.naacl-main.217.pdf)]; [[corpus](https://github.com/microsoft/MSComplexTasks)]. \n   \n4. **Analogous Process Structure Induction for Sub-event Sequence Prediction.** *Hongming Zhang, Muhao Chen, Haoyu Wang, Yangqiu Song, and Dan Roth.* \u003cins\u003eEMNLP\u003c/ins\u003e 2020. [[pdf](https://aclanthology.org/2020.emnlp-main.119.pdf)]; [[code](https://cogcomp.github.io/APSI/)]. \n\n\n### 7.2 ChatGPT-related Papers\n\n\u003c!-- **ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks.** *Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2303.15056.pdf)]; [[other resources](link)]. --\u003e\n\nNowdays, ChatGPT is a super star 🌟 in the NLP community. Since there is no official paper for ChatGPT, we share some frontier works that can provide deep insights into ChatGPT.\n   \n1. **When do you need Chain-of-Thought Prompting for ChatGPT?** *Jiuhai Chen, Lichang Chen, Heng Huang, and Tianyi Zhou.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2304.03262.pdf)].\n   \n2. **Toxicity in ChatGPT: Analyzing Persona-assigned Language Models.** *Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2304.05335.pdf)].\n   \n3. **Is ChatGPT a General-Purpose Natural Language Processing Task Solver?** *Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.06476.pdf)].\n   \n4. **How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection.** *Biyang Guo, Xin Zhang, Ziyuan Wang, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2301.07597.pdf)]; [[corpus](https://github.com/Hello-SimpleAI/chatgpt-comparison-detection)]. \n   \n5. **ChatGPT: Jack of all trades, master of none.** *Jan Kocoń, Igor Cichecki, Oliwier Kaszyca, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.10724.pdf)].\n   \n6. **On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective.** *Jindong Wang, Xixu Hu, Wenxin Hou, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.12095.pdf)]; [[code](https://github.com/microsoft/robustlearn)]. \n\n\n### 7.3 Human Feedback vs. Model Feedback\n\n\n\u003c!-- **Paper Title.** *Author 1, Author 2, and Author 3.* \u003cins\u003eConference/Journal/Preprint\u003c/ins\u003e Year. [[pdf](link)]; [[other resources](link)]. --\u003e\n   \n1. **Aligning Large Language Models through Synthetic Feedback.** *Sungdong Kim, Sanghwan Bae, Jamin Shin, Soyoung Kang, Donghyun Kwak, Kang Min Yoo, and Minjoon Seo.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2305.13735.pdf)].\n   \n2. **LIMA: Less Is More for Alignment.** *Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2305.11206.pdf)].\n   \n3. **Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision.** *Zhiqing Sun, Yikang Shen, Qinhong Zhou, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2305.03047.pdf)]; [[code](https://github.com/IBM/Dromedary)].\n   \n4. **Chain of Hindsight Aligns Language Models with Feedback.** *Hao Liu, Carmelo Sferrazza, and Pieter Abbeel.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.02676.pdf)]; [[code](https://github.com/lhao499/CoH)]. \n   \n5. **Pretraining Language Models with Human Preferences.** *Tomasz Korbak, Kejian Shi, Angelica Chen, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.08582.pdf)].\n   \n6. **Constitutional AI: Harmlessness from AI Feedback.** *Yuntao Bai, Saurav Kadavath, Sandipan Kundu, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2212.08073.pdf)]; [[corpus](https://github.com/anthropics/ConstitutionalHarmlessnessPaper)].\n   \n7. **Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback.** *Yuntao Bai, Andy Jones, Kamal Ndousse, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2204.05862.pdf)]; [[corpus](https://github.com/anthropics/hh-rlhf)]. \n\n\n### 7.4 Scalable Oversight and Alignment\n\n\n1. **Measuring Progress on Scalable Oversight for Large Language Models.** *Samuel R. Bowman, Jeeyoon Hyun, Ethan Perez, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2211.03540.pdf)].\n\n2. **Aligning AI With Shared Human Values.** *Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt.* \u003cins\u003eICLR\u003c/ins\u003e 2021. [[pdf](https://openreview.net/pdf?id=dNy_RKzJacY)].\n\n\n### 7.5 Other Papers\n\n\n1. **Navigating the Grey Area: Expressions of Overconfidence and Uncertainty in Language Models.** *Kaitlyn Zhou, Dan Jurafsky, and Tatsunori Hashimoto.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.13439.pdf)].\n   \n2. **The Capacity for Moral Self-Correction in Large Language Models.** *Deep Ganguli, Amanda Askell, Nicholas Schiefer, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.07459.pdf)]. \n   \n3. **Large Language Models Can Be Easily Distracted by Irrelevant Context.** *Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou.* \u003cins\u003ePreprint\u003c/ins\u003e 2023. [[pdf](https://arxiv.org/pdf/2302.00093.pdf)]; [[corpus](https://github.com/google-research-datasets/GSM-IC)].\n\n4. **Language Models (Mostly) Know What They Know.** *Saurav Kadavath, Tom Conerly, Amanda Askell, and et al.* \u003cins\u003ePreprint\u003c/ins\u003e 2022. [[pdf](https://arxiv.org/pdf/2207.05221.pdf)].\n\n\n\n\u003c!-- TODO: tweets \u0026 slides? --\u003e\n\n---\n\n\u003c!-- omit in toc --\u003e\n## ⭐ Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=RenzeLou/awesome-instruction-learning\u0026type=Date)](https://star-history.com/#RenzeLou/awesome-instruction-learning\u0026Date)\n","projects_url":"https://awesome.ecosyste.ms/api/v1/lists/renzelou%2Fawesome-instruction-learning/projects"}