{"id":23735362,"url":"https://github.com/SEU-COIN/LLMPapers","last_synced_at":"2025-09-04T09:34:11.776Z","repository":{"id":67155317,"uuid":"592646691","full_name":"SEU-COIN/LLMPapers","owner":"SEU-COIN","description":"Papers \u0026 Works for large languange models (OpenAI GPT-4, Meta Llama, etc.).","archived":false,"fork":false,"pushed_at":"2025-05-11T06:22:13.000Z","size":13794,"stargazers_count":318,"open_issues_count":1,"forks_count":27,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-08-30T21:09:00.509Z","etag":null,"topics":["chatgpt","codex","gpt-3","large-language-models"],"latest_commit_sha":null,"homepage":"","language":"TeX","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SEU-COIN.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-24T07:54:38.000Z","updated_at":"2025-08-23T13:35:55.000Z","dependencies_parsed_at":"2023-11-06T07:48:22.813Z","dependency_job_id":"66fdcdaa-b0f0-45de-a879-210f21f30018","html_url":"https://github.com/SEU-COIN/LLMPapers","commit_stats":null,"previous_names":["seu-coin/llmpapers","kseseu/llmpapers"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SEU-COIN/LLMPapers","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SEU-COIN%2FLLMPapers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SEU-COIN%2FLLMPapers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SEU-COIN%2FLLMPapers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SEU-COIN%2FLLMPapers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SEU-COIN","download_url":"https://codeload.github.com/SEU-COIN/LLMPapers/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SEU-COIN%2FLLMPapers/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273583667,"owners_count":25131885,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-04T02:00:08.968Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatgpt","codex","gpt-3","large-language-models"],"created_at":"2024-12-31T06:03:15.191Z","updated_at":"2025-09-04T09:34:11.726Z","avatar_url":"https://github.com/SEU-COIN.png","language":"TeX","funding_links":[],"categories":["TeX"],"sub_categories":[],"readme":"# Resources on ChatGPT and Large Language Models\r\nCollection of  papers and related works for Large Language Models (ChatGPT, GPT-3, Codex etc.).\r\n## Contributors\r\nThis repository is contributed by the following contributors.\r\n- **Organizers**: [Guilin Qi (漆桂林)](https://cse.seu.edu.cn/2019/0103/c23024a257135/page.htm), [Xiaofang Qi (戚晓芳)](https://cse.seu.edu.cn/2019/0103/c23024a257134/page.htm)\r\n- **Paper Collectors**: Zafar Ali, [Sheng Bi (毕胜)](https://github.com/bisheng), [Yongrui Chen (陈永锐)](https://github.com/Bahuia), Zizhuo Chen (陈孜卓), [Xinbang Dai (戴鑫邦)](https://github.com/OBriennnnn), Huan Gao (高桓), [Nan Hu (胡楠)](https://github.com/HuuuNan), Shilong Hu (胡世龙), [Jingqi Kang (康婧淇)](https://github.com/JingqiKang), [Jiaqi Li (李嘉琦)](https://github.com/aoluming), [Dehai Min (闵德海)](https://github.com/ZhishanQ), [Guilin Qi (漆桂林)](https://cse.seu.edu.cn/2019/0103/c23024a257135/page.htm), Yiming Tan (谭亦鸣), [Tongtong Wu (吴桐桐)](http://wutong8023.site/), [Songlin Zhai (翟松林)](https://github.com/SonglinZhai), [Shenyu Zhang (张沈昱)](https://github.com/ZSY-SZ), [Yuxin Zhang (张裕欣)](https://github.com/Zzyx1996)\r\n- **Maintainers**: [Runzhe Wang (王润哲)](https://github.com/sid0527), [Shenyu Zhang (张沈昱)](https://github.com/ZSY-SZ) \r\n\r\nThe automation script of this repo is powered by [Auto-Bibfile](https://github.com/wutong8023/Auto-Bibfile.git). If you'd like to commit to this repo, please modify [bibtex.bib](https://github.com/KSESEU/LLMPapers/blob/main/bibtex.bib) or [related_works.json](https://github.com/KSESEU/LLMPapers/blob/main/related_works.json) and re-generate [README.md](https://github.com/KSESEU/LLMPapers/blob/main/README.md) using `python scripts/run.py`.\r\n\r\n\r\n\r\n## Papers\r\n\r\n### Outline \r\n- [\u003cimg src=https://img.shields.io/badge/Evaluation-31-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#evaluation)\r\n- [\u003cimg src=https://img.shields.io/badge/Survey-47-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#survey)\r\n- [\u003cimg src=https://img.shields.io/badge/In--Context_Learning-44-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#in-context-learning)\r\n- [\u003cimg src=https://img.shields.io/badge/Instruction_Tuning-18-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#instruction-tuning)\r\n- [\u003cimg src=https://img.shields.io/badge/RLHF-20-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#rlhf)\r\n- [\u003cimg src=https://img.shields.io/badge/Pre--Training_Techniques-19-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#pre-training-techniques)\r\n  - [\u003cimg src=https://img.shields.io/badge/Mixtures_of_Experts-4-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#mixtures-of-experts)\r\n- [\u003cimg src=https://img.shields.io/badge/Knowledge_Enhanced-23-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#knowledge-enhanced)\r\n- [\u003cimg src=https://img.shields.io/badge/Knowledge_Distillation-24-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#knowledge-distillation)\r\n- [\u003cimg src=https://img.shields.io/badge/Knowledge_Generation-11-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#knowledge-generation)\r\n- [\u003cimg src=https://img.shields.io/badge/Knowledge_Editing-16-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#knowledge-editing)\r\n- [\u003cimg src=https://img.shields.io/badge/Reasoning-163-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#reasoning)\r\n  - [\u003cimg src=https://img.shields.io/badge/Chain_of_Thought-70-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#chain-of-thought)\r\n  - [\u003cimg src=https://img.shields.io/badge/Multi--Step_Reasoning-6-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#multi-step-reasoning)\r\n  - [\u003cimg src=https://img.shields.io/badge/Arithmetic_Reasoning-5-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#arithmetic-reasoning)\r\n  - [\u003cimg src=https://img.shields.io/badge/Symbolic_Reasoning-23-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#symbolic-reasoning)\r\n  - [\u003cimg src=https://img.shields.io/badge/Chain_of_Verification-1-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#chain-of-verification)\r\n  - [\u003cimg src=https://img.shields.io/badge/Knowledge_Graph_Embedding-5-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#knowledge-graph-embedding)\r\n  - [\u003cimg src=https://img.shields.io/badge/Slow_Thinking-7-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#slow-thinking)\r\n- [\u003cimg src=https://img.shields.io/badge/Federated_Learning-14-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#federated-learning)\r\n- [\u003cimg src=https://img.shields.io/badge/Distributed_AI-9-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#distributed-ai)\r\n- [\u003cimg src=https://img.shields.io/badge/Selective_Annotation-2-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#selective-annotation)\r\n- [\u003cimg src=https://img.shields.io/badge/Program_and_Code_Generation-45-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#program-and-code-generation)\r\n  - [\u003cimg src=https://img.shields.io/badge/Code_Representation-5-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#code-representation)\r\n  - [\u003cimg src=https://img.shields.io/badge/Code_Fixing-8-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#code-fixing)\r\n  - [\u003cimg src=https://img.shields.io/badge/Code_Review-5-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#code-review)\r\n  - [\u003cimg src=https://img.shields.io/badge/Program_Generation-3-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#program-generation)\r\n- [\u003cimg src=https://img.shields.io/badge/Software_Engineering-5-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#software-engineering)\r\n- [\u003cimg src=https://img.shields.io/badge/AIGC-78-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#aigc)\r\n  - [\u003cimg src=https://img.shields.io/badge/Controllable_Text_Generation-9-deepskyblue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#controllable-text-generation)\r\n- [\u003cimg src=https://img.shields.io/badge/Continual_Learning-46-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#continual-learning)\r\n- [\u003cimg src=https://img.shields.io/badge/Prompt_Engineering-35-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#prompt-engineering)\r\n- [\u003cimg src=https://img.shields.io/badge/Natural_Language_Understanding-8-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#natural-language-understanding)\r\n- [\u003cimg src=https://img.shields.io/badge/Multimodal-26-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#multimodal)\r\n- [\u003cimg src=https://img.shields.io/badge/Multilingual-1-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#multilingual)\r\n- [\u003cimg src=https://img.shields.io/badge/Reliability-5-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#reliability)\r\n- [\u003cimg src=https://img.shields.io/badge/Robustness-2-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#robustness)\r\n- [\u003cimg src=https://img.shields.io/badge/Dialogue_System-16-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#dialogue-system)\r\n- [\u003cimg src=https://img.shields.io/badge/Recommender_System-10-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#recommender-system)\r\n- [\u003cimg src=https://img.shields.io/badge/Event_Extraction-6-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#event-extraction)\r\n- [\u003cimg src=https://img.shields.io/badge/Event_Relation_Extraction-7-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#event-relation-extraction)\r\n- [\u003cimg src=https://img.shields.io/badge/Data_Argumentation-4-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#data-argumentation)\r\n- [\u003cimg src=https://img.shields.io/badge/Data_Annotation-2-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#data-annotation)\r\n- [\u003cimg src=https://img.shields.io/badge/Information_Extraction-56-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#information-extraction)\r\n- [\u003cimg src=https://img.shields.io/badge/Domain_Adaptive-3-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#domain-adaptive)\r\n- [\u003cimg src=https://img.shields.io/badge/Question_Answering-26-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#question-answering)\r\n- [\u003cimg src=https://img.shields.io/badge/Application-10-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#application)\r\n- [\u003cimg src=https://img.shields.io/badge/Meta_Learning-2-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#meta-learning)\r\n- [\u003cimg src=https://img.shields.io/badge/Generalizability-3-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#generalizability)\r\n- [\u003cimg src=https://img.shields.io/badge/Language_Model_as_Knowledge_Base-20-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#language-model-as-knowledge-base)\r\n- [\u003cimg src=https://img.shields.io/badge/Retrieval--Augmented_Language_Model-14-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#retrieval-augmented-language-model)\r\n- [\u003cimg src=https://img.shields.io/badge/Quality-2-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#quality)\r\n- [\u003cimg src=https://img.shields.io/badge/Interpretability/Explainability-3-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#interpretability/explainability)\r\n- [\u003cimg src=https://img.shields.io/badge/Data_Generation-3-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#data-generation)\r\n- [\u003cimg src=https://img.shields.io/badge/Safety-1-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#safety)\r\n- [\u003cimg src=https://img.shields.io/badge/Graph_Learning-6-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#graph-learning)\r\n- [\u003cimg src=https://img.shields.io/badge/Knowledge_Storage_and_Locating-3-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#knowledge-storage-and-locating)\r\n- [\u003cimg src=https://img.shields.io/badge/Knowledge_Fusion-16-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#knowledge-fusion)\r\n- [\u003cimg src=https://img.shields.io/badge/Agent-2-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#agent)\r\n- [\u003cimg src=https://img.shields.io/badge/LLM_and_GNN-5-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#llm-and-gnn)\r\n- [\u003cimg src=https://img.shields.io/badge/Vision_LLM-1-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#vision-llm)\r\n- [\u003cimg src=https://img.shields.io/badge/LLM_and_KG-25-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#llm-and-kg)\r\n- [\u003cimg src=https://img.shields.io/badge/Others-6-blue style=\"zoom:100%; vertical-align: middle\"\u003e](https://github.com/KSESEU/LLMPapers/blob/main/./README.md#others)\r\n### Hyperlinks \r\n- [[Overview]](https://github.com/KSESEU/LLMPapers/blob/main/README.md) -- [Homepage](https://github.com/KSESEU/LLMPapers/blob/main/README.md)\r\n-  -- [Summary](https://github.com/KSESEU/LLMPapers/blob/main/taxonomy/./)\r\n-  -- [Author](https://github.com/KSESEU/LLMPapers/blob/main/taxonomy/author)\r\n-  -- [Techniques](https://github.com/KSESEU/LLMPapers/blob/main/taxonomy/techniques)\r\n-  -- [Published Time](https://github.com/KSESEU/LLMPapers/blob/main/taxonomy/time)\r\n-  -- [Published Venue](https://github.com/KSESEU/LLMPapers/blob/main/taxonomy/venue)\r\n\r\n### Evaluation\r\n\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2404.00942) [**Evaluating the Factuality of Large Language Models using Large-Scale\r\nKnowledge Graphs**](https://doi.org/10.48550/arXiv.2404.00942),\u003cbr\u003e by *Xiaoze Liu, Feijie Wu, Tianyang Xu, Zhuo Chen, Yichi Zhang, Xiaoqian Wang and Jing Gao*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.04023) [**A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning,\r\nHallucination, and Interactivity**](https://doi.org/10.48550/arXiv.2302.04023),\u003cbr\u003e by *Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji et al.*\r\n\u003cbr\u003e``\r\n本文提出了一个使用公开数据集定量评估交互式LLM（如ChatGPT）的框架。我们使用涵盖8个不同的常见NLP应用任务的21个数据集对ChatGPT进行了广泛的技术评估。我们基于这些数据集和一个新设计的多模态数据集评估了ChatGPT的多任务、多语言和多模态方面。\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2302.06476) [**Is ChatGPT a General-Purpose Natural Language Processing Task Solver?**](https://arxiv.org/abs/2302.06476),\u003cbr\u003e by *Qin, Chengwei, Zhang, Aston, Zhang, Zhuosheng, Chen, Jiaao, Yasunaga, Michihiro and Yang, Diyi*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.06466) [**ChatGPT versus Traditional Question Answering for Knowledge Graphs:\r\nCurrent Status and Future Directions Towards Knowledge Graph Chatbots**](https://doi.org/10.48550/arXiv.2302.06466),\u003cbr\u003e by *Reham Omar, Omij Mangukiya, Panos Kalnis and Essam Mansour*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2301.13867) [**Mathematical Capabilities of ChatGPT**](https://doi.org/10.48550/arXiv.2301.13867),\u003cbr\u003e by *Simon Frieder, Luca Pinchetti, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Christian Petersen, Alexis Chevalier and Julius Berner*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.08081) [**Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization**](https://doi.org/10.48550/arXiv.2302.08081),\u003cbr\u003e by *Xianjun Yang, Yan Li, Xinlu Zhang, Haifeng Chen and Wei Cheng*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.12095) [**On the Robustness of ChatGPT: An Adversarial and Out-of-distribution\r\nPerspective**](https://doi.org/10.48550/arXiv.2302.12095),\u003cbr\u003e by *Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2301.04655) [**ChatGPT is not all you need. A State of the Art Review of large\r\nGenerative AI models**](https://doi.org/10.48550/arXiv.2301.04655),\u003cbr\u003e by *Roberto Gozalo-Brizuela and Eduardo C. Garrido-Merch\\'an*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2302.10198) [**Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned\r\nBERT**](https://arxiv.org/abs/2302.10198),\u003cbr\u003e by *Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du and Dacheng Tao*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2303.07992) [**Evaluation of ChatGPT as a Question Answering System for Answering\r\nComplex Questions**](https://doi.org/10.48550/arXiv.2303.07992),\u003cbr\u003e by *Yiming Tan, Dehai Min, Yu Li, Wenbo Li, Nan Hu, Yongrui Chen and Guilin Qi*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2303.16421) [**ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models**](https://arxiv.org/abs/2303.16421),\u003cbr\u003e by *Ning Bian, Xianpei Han, Le Sun, Hongyu Lin, Yaojie Lu and Ben He*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2308.07902) [**Through the Lens of Core Competency: Survey on Evaluation of Large\r\nLanguage Models**](https://doi.org/10.48550/arXiv.2308.07902),\u003cbr\u003e by *Ziyu Zhuang, Qiguang Chen, Longxuan Ma, Mingda Li, Yi Han, Yushan Qian, Haopeng Bai, Zixian Feng et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2305.17306) [**Chain-of-Thought Hub: A Continuous Effort to Measure Large Language\r\nModels' Reasoning Performance**](https://doi.org/10.48550/arXiv.2305.17306),\u003cbr\u003e by *Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao Peng and Tushar Khot*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ACL-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2023.findings-acl.29) [**A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark\r\nDatasets**](https://doi.org/10.18653/v1/2023.findings-acl.29),\u003cbr\u003e by *Md. Tahmid Rahman Laskar, M. Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty and Jimmy X. Huang*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2308.12488) [**GPTEval: A Survey on Assessments of ChatGPT and GPT-4**](https://doi.org/10.48550/arXiv.2308.12488),\u003cbr\u003e by *Rui Mao, Guanyi Chen, Xulang Zhang, Frank Guerin and Erik Cambria*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2211.09110) [**Holistic Evaluation of Language Models**](https://doi.org/10.48550/arXiv.2211.09110),\u003cbr\u003e by *Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2204.00498) [**Evaluating the Text-to-SQL Capabilities of Large Language Models**](https://doi.org/10.48550/arXiv.2204.00498),\u003cbr\u003e by *Nitarshan Rajkumar, Raymond Li and Dzmitry Bahdanau*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/COLING-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.coling-1.491) [**Are Visual-Linguistic Models Commonsense Knowledge Bases?**](https://aclanthology.org/2022.coling-1.491),\u003cbr\u003e by *Hsiu-Yu Yang and Carina Silberer*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.10529) [**Is GPT-3 a Psychopath? Evaluating Large Language Models from a Psychological\r\nPerspective**](https://doi.org/10.48550/arXiv.2212.10529),\u003cbr\u003e by *Xingxuan Li, Yutong Li, Linlin Liu, Lidong Bing and Shafiq R. Joty*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.emnlp-main.132) [**GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained\r\nLanguage Models**](https://aclanthology.org/2022.emnlp-main.132),\u003cbr\u003e by *Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li and Kai-Wei Chang*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.emnlp-main.653) [**RobustLR: A Diagnostic Benchmark for Evaluating Logical Robustness\r\nof Deductive Reasoners**](https://aclanthology.org/2022.emnlp-main.653),\u003cbr\u003e by *Soumya Sanyal, Zeyi Liao and Xiang Ren*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2202.13169) [**A Systematic Evaluation of Large Language Models of Code**](https://arxiv.org/abs/2202.13169),\u003cbr\u003e by *Frank F. Xu, Uri Alon, Graham Neubig and Vincent J. Hellendoorn*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2022.findings-emnlp.445) [**Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts**](https://doi.org/10.18653/v1/2022.findings-emnlp.445),\u003cbr\u003e by *Arshiya Aggarwal, Jiao Sun and Nanyun Peng*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2021-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2107.03374) [**Evaluating Large Language Models Trained on Code**](https://arxiv.org/abs/2107.03374),\u003cbr\u003e by *Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond\\'e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ACL-2021-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2021.findings-acl.36) [**GLGE: A New General Language Generation Evaluation Benchmark**](https://doi.org/10.18653/v1/2021.findings-acl.36),\u003cbr\u003e by *Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2021-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2104.05861) [**Evaluating Pre-Trained Models for User Feedback Analysis in Software\r\nEngineering: A Study on Classification of App-Reviews**](https://arxiv.org/abs/2104.05861),\u003cbr\u003e by *Mohammad Abdul Hadi and Fatemeh H. Fard*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ACL_Findings-2021-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2021.findings-acl.322) [**Do Language Models Perform Generalizable Commonsense Inference?**](https://doi.org/10.18653/v1/2021.findings-acl.322), [\u003cimg src=https://img.shields.io/badge/Code-skyblue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://github.com/wangpf3/LM-for-CommonsenseInference)\u003cbr\u003e by *Peifeng Wang, Filip Ilievski, Muhao Chen and Xiang Ren*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2021-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2021.emnlp-main.598) [**RICA: Evaluating Robust Inference Capabilities Based on Commonsense\r\nAxioms**](https://doi.org/10.18653/v1/2021.emnlp-main.598),\u003cbr\u003e by *Pei Zhou, Rahul Khanna, Seyeon Lee, Bill Yuchen Lin, Daniel Ho, Jay Pujara and Xiang Ren*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2006.14799) [**Evaluation of Text Generation: A Survey**](https://arxiv.org/abs/2006.14799),\u003cbr\u003e by *Asli Celikyilmaz, Elizabeth Clark and Jianfeng Gao*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2007.15780) [**Neural Language Generation: Formulation, Methods, and Evaluation**](https://arxiv.org/abs/2007.15780),\u003cbr\u003e by *Cristina Garbacea and Qiaozhu Mei*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ICLR-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://openreview.net/forum?id=SkeHuCVFDr) [**BERTScore: Evaluating Text Generation with BERT**](https://openreview.net/forum?id=SkeHuCVFDr),\u003cbr\u003e by *Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger and Yoav Artzi*\r\n\u003cbr\u003e\u003cbr\u003e\r\n### Survey\r\n\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2025-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2501.09686) [**Towards Large Reasoning Models: A Survey of Reinforced Reasoning\r\nwith Large Language Models**](https://doi.org/10.48550/arXiv.2501.09686),\u003cbr\u003e by *Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2025-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2502.17419) [**From System 1 to System 2: A Survey of Reasoning Large Language\r\nModels**](https://doi.org/10.48550/arXiv.2502.17419),\u003cbr\u003e by *Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2402.18041) [**Datasets for Large Language Models: A Comprehensive Survey**](https://doi.org/10.48550/arXiv.2402.18041),\u003cbr\u003e by *Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding and Lianwen Jin*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2403.14734) [**A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond**](https://doi.org/10.48550/arXiv.2403.14734),\u003cbr\u003e by *Qiushi Sun, Zhirui Chen, Fangzhi Xu, Kanzhi Cheng, Chang Ma, Zhangyue Yin, Jianing Wang, Chengcheng Han et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2024.emnlp-main.54) [**Large Language Models for Data Annotation and Synthesis: A Survey**](https://aclanthology.org/2024.emnlp-main.54),\u003cbr\u003e by *Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2301.00234) [**A Survey for In-context Learning**](https://doi.org/10.48550/arXiv.2301.00234),\u003cbr\u003e by *Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu et al.*\r\n\u003cbr\u003e``\r\nThis paper surveys and summarizes the progress and challenges of ICL, including ICL's formal definition, correlation to related studies, advanced techniques (training strategies, related analysis) and potential directions.\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.09419) [**A Comprehensive Survey on Pretrained Foundation Models: A History\r\nfrom BERT to ChatGPT**](https://doi.org/10.48550/arXiv.2302.09419),\u003cbr\u003e by *Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.09051) [**Complex QA and language models hybrid architectures, Survey**](https://doi.org/10.48550/arXiv.2302.09051),\u003cbr\u003e by *Xavier Daull, Patrice Bellot, Emmanuel Bruno, Vincent Martin and Elisabeth Murisasco*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.07842) [**Augmented Language Models: a Survey**](https://doi.org/10.48550/arXiv.2302.07842),\u003cbr\u003e by *Gr\\'egoire Mialon, Roberto Dess\\`\\i, Maria Lomeli, Christoforos Nalmpantis, Ramakanth Pasunuru, Roberta Raileanu, Baptiste Rozi\\`ere, Timo Schick et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2303.07616) [**The Life Cycle of Knowledge in Big Language Models: A Survey**](https://doi.org/10.48550/arXiv.2303.07616),\u003cbr\u003e by *Boxi Cao, Hongyu Lin, Xianpei Han and Le Sun*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2303.18223) [**A Survey of Large Language Models**](https://doi.org/10.48550/arXiv.2303.18223),\u003cbr\u003e by *Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ACM_Comput._Surv.-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.1145/3571730) [**Survey of Hallucination in Natural Language Generation**](https://doi.org/10.1145/3571730),\u003cbr\u003e by *Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2307.12966) [**Aligning Large Language Models with Human: A Survey**](https://doi.org/10.48550/arXiv.2307.12966),\u003cbr\u003e by *Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2308.11432) [**A Survey on Large Language Model based Autonomous Agents**](https://doi.org/10.48550/arXiv.2308.11432),\u003cbr\u003e by *Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/Internet_of_Things_and_Cyber--Physical_Systems-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://www.sciencedirect.com/science/article/pii/S266734522300024X) [**ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope**](https://www.sciencedirect.com/science/article/pii/S266734522300024X),\u003cbr\u003e by *Ray, Partha Pratim*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2308.10620) [**Large Language Models for Software Engineering: A Systematic Literature\r\nReview**](https://doi.org/10.48550/arXiv.2308.10620),\u003cbr\u003e by *Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2306.08302) [**Unifying Large Language Models and Knowledge Graphs: A Roadmap**](https://doi.org/10.48550/arXiv.2306.08302),\u003cbr\u003e by *Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang and Xindong Wu*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2308.07107) [**Large Language Models for Information Retrieval: A Survey**](https://doi.org/10.48550/arXiv.2308.07107),\u003cbr\u003e by *Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Zhicheng Dou and Ji-Rong Wen*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2307.03109) [**A Survey on Evaluation of Large Language Models**](https://doi.org/10.48550/arXiv.2307.03109),\u003cbr\u003e by *Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2308.14177) [**AIGC for Various Data Modalities: A Survey**](https://doi.org/10.48550/arXiv.2308.14177),\u003cbr\u003e by *Lin Geng Foo, Hossein Rahmani and Jun Liu*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/arXiv_preprint_arXiv:2305.18703-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/pdf/2305.18703.pdf) [**Domain specialization as the key to make large language models disruptive: A comprehensive survey**](https://arxiv.org/pdf/2305.18703.pdf),\u003cbr\u003e by *Ling, Chen, Zhao, Xujiang, Lu, Jiaying, Deng, Chengyuan, Zheng, Can, Wang, Junxiang, Chowdhury, Tanmoy, Li, Yun et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2303.04226) [**A Comprehensive Survey of AI-Generated Content (AIGC): A History\r\nof Generative AI from GAN to ChatGPT**](https://doi.org/10.48550/arXiv.2303.04226),\u003cbr\u003e by *Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S. Yu and Lichao Sun*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2308.10792) [**Instruction Tuning for Large Language Models: A Survey**](https://doi.org/10.48550/arXiv.2308.10792),\u003cbr\u003e by *Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2310.07521) [**Survey on Factuality in Large Language Models: Knowledge, Retrieval\r\nand Domain-Specificity**](https://doi.org/10.48550/arXiv.2310.07521),\u003cbr\u003e by *Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Jiayang Cheng, Yunzhi Yao, Wenyang Gao et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2309.15698) [**Deep Model Fusion: A Survey**](https://doi.org/10.48550/arXiv.2309.15698),\u003cbr\u003e by *Weishi Li, Yong Peng, Miao Zhang, Liang Ding, Han Hu and Li Shen*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2309.15402) [**A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future**](https://doi.org/10.48550/arXiv.2309.15402),\u003cbr\u003e by *Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2309.01029) [**Explainability for Large Language Models: A Survey**](https://doi.org/10.48550/arXiv.2309.01029),\u003cbr\u003e by *Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2312.17617) [**Large Language Models for Generative Information Extraction: A Survey**](https://arxiv.org/abs/2312.17617),\u003cbr\u003e by *Derong Xu, Wei Chen, Wenjun Peng, Chao Zhang, Tong Xu, Xiangyu Zhao, Xian Wu, Yefeng Zheng et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2311.12399) [**A Survey of Graph Meets Large Language Model: Progress and Future\r\nDirections**](https://doi.org/10.48550/arXiv.2311.12399),\u003cbr\u003e by *Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng and Jeffrey Xu Yu*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2311.05232) [**A Survey on Hallucination in Large Language Models: Principles, Taxonomy,\r\nChallenges, and Open Questions**](https://doi.org/10.48550/arXiv.2311.05232),\u003cbr\u003e by *Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.09420) [**When Neural Model Meets NL2Code: A Survey**](https://doi.org/10.48550/arXiv.2212.09420),\u003cbr\u003e by *Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Yongji Wang and Jian-Guang Lou*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/TKDE-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.13428) [**A Survey on Knowledge-Enhanced Pre-trained Language Models**](https://doi.org/10.48550/arXiv.2212.13428),\u003cbr\u003e by *Chaoqi Zhen, Yanlei Shang, Xiangyu Liu, Yifei Li, Yong Chen and Dell Zhang*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/T--PAMI-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.1109/TPAMI.2021.3057446) [**A Continual Learning Survey: Defying Forgetting in Classification\r\nTasks**](https://doi.org/10.1109/TPAMI.2021.3057446),\u003cbr\u003e by *Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory G. Slabaugh and Tinne Tuytelaars*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/JKSUCIS-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.1016/j.jksuci.2020.04.001) [**The survey: Text generation models in deep learning**](https://doi.org/10.1016/j.jksuci.2020.04.001),\u003cbr\u003e by *Touseef Iqbal and Shaima Qureshi*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/KIS-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.1007/s10115-022-01664-x) [**From distributed machine learning to federated learning: a survey**](https://doi.org/10.1007/s10115-022-01664-x),\u003cbr\u003e by *Ji Liu, Jizhou Huang, Yang Zhou, Xuhong Li, Shilei Ji, Haoyi Xiong and Dejing Dou*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/IJCAI-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.24963/ijcai.2022/775) [**Deep Learning Meets Software Engineering: A Survey on Pre-Trained\r\nModels of Source Code**](https://doi.org/10.24963/ijcai.2022/775),\u003cbr\u003e by *Changan Niu, Chuanyi Li, Bin Luo and Vincent Ng*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/TKDE-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.1109/TKDE.2020.3028705) [**A Survey on Knowledge Graph-Based Recommender Systems**](https://doi.org/10.1109/TKDE.2020.3028705),\u003cbr\u003e by *Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong and Qing He*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.09252) [**Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue\r\nSystems**](https://doi.org/10.48550/arXiv.2212.09252),\u003cbr\u003e by *Sagi Shaier, Lawrence Hunter and Katharina Kann*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.10403) [**Towards Reasoning in Large Language Models: A Survey**](https://doi.org/10.48550/arXiv.2212.10403),\u003cbr\u003e by *Jie Huang and Kevin Chen-Chuan Chang*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.09597) [**Reasoning with Language Model Prompting: A Survey**](https://doi.org/10.48550/arXiv.2212.09597),\u003cbr\u003e by *Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2202.01110) [**A Survey on Retrieval-Augmented Text Generation**](https://arxiv.org/abs/2202.01110),\u003cbr\u003e by *Huayang Li, Yixuan Su, Deng Cai, Yan Wang and Lemao Liu*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/AAAI-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://ojs.aaai.org/index.php/AAAI/article/view/21496) [**Commonsense Knowledge Reasoning and Generation with Pre-trained Language\r\nModels: A Survey**](https://ojs.aaai.org/index.php/AAAI/article/view/21496),\u003cbr\u003e by *Prajjwal Bhargava and Vincent Ng*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2204.06031) [**A Review on Language Models as Knowledge Bases**](https://doi.org/10.48550/arXiv.2204.06031),\u003cbr\u003e by *Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona T. Diab and Marjan Ghazvininejad*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2021-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2101.09459) [**Advances and Challenges in Conversational Recommender Systems: A\r\nSurvey**](https://arxiv.org/abs/2101.09459),\u003cbr\u003e by *Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke and Tat-Seng Chua*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2021-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2105.04387) [**Recent Advances in Deep Learning Based Dialogue Systems: A Systematic\r\nSurvey**](https://arxiv.org/abs/2105.04387),\u003cbr\u003e by *Jinjie Ni, Tom Young, Vlad Pandelea, Fuzhao Xue, Vinay Adiga and Erik Cambria*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2021-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2021.emnlp-main.81) [**Relational World Knowledge Representation in Contextual Language Models:\r\nA Review**](https://doi.org/10.18653/v1/2021.emnlp-main.81),\u003cbr\u003e by *Tara Safavi and Danai Koutra*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2006.14799) [**Evaluation of Text Generation: A Survey**](https://arxiv.org/abs/2006.14799),\u003cbr\u003e by *Asli Celikyilmaz, Elizabeth Clark and Jianfeng Gao*\r\n\u003cbr\u003e\u003cbr\u003e\r\n### In-Context Learning\r\n\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2401.11624) [**In-context Learning with Retrieved Demonstrations for Language Models:\r\nA Survey**](https://doi.org/10.48550/arXiv.2401.11624),\u003cbr\u003e by *Man Luo, Xin Xu, Yue Liu, Panupong Pasupat and Mehran Kazemi*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2403.06402) [**'One size doesn't fit all': Learning how many Examples to use for\r\nIn-Context Learning for Improved Text Classification**](https://doi.org/10.48550/arXiv.2403.06402),\u003cbr\u003e by *Manish Chandra, Debasis Ganguly, Yiwen Li and Iadh Ounis*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2405.01116) [**\"In-Context Learning\" or: How I learned to stop worrying and love \"Applied Information Retrieval\"**](https://arxiv.org/abs/2405.01116),\u003cbr\u003e by *Andrew Parry, Debasis Ganguly and Manish Chandra*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2406.11194) [**In-Context Editing: Learning Knowledge from Self-Induced Distributions**](https://doi.org/10.48550/arXiv.2406.11194),\u003cbr\u003e by *Siyuan Qi, Bangcheng Yang, Kailin Jiang, Xiaobo Wang, Jiaqi Li, Yifan Zhong, Yaodong Yang and Zilong Zheng*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2301.00234) [**A Survey for In-context Learning**](https://doi.org/10.48550/arXiv.2301.00234),\u003cbr\u003e by *Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu et al.*\r\n\u003cbr\u003e``\r\nThis paper surveys and summarizes the progress and challenges of ICL, including ICL's formal definition, correlation to related studies, advanced techniques (training strategies, related analysis) and potential directions.\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.04813) [**Explanation Selection Using Unlabeled Data for In-Context Learning**](https://doi.org/10.48550/arXiv.2302.04813),\u003cbr\u003e by *Xi Ye and Greg Durrett*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.04931) [**In-Context Learning with Many Demonstration Examples**](https://doi.org/10.48550/arXiv.2302.04931),\u003cbr\u003e by *Mukai Li, Shansan Gong, Jiangtao Feng, Yiheng Xu, Jun Zhang, Zhiyong Wu and Lingpeng Kong*\r\n\u003cbr\u003e``\r\nThis paper proposes a LM named EvaLM to scale up the sequence length (trained with 8k tokens per batch line). Experiments based on EvaLM prove that in-context learning can achieve higher performance with more demonstrations under many-shot instruction tuning (8k) and further extending the length of instructions (16k) can further improve the upper bound of scaling in-context  learning.\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2301.11916) [**Large Language Models Are Implicitly Topic Models: Explaining and\r\nFinding Good Demonstrations for In-Context Learning**](https://doi.org/10.48550/arXiv.2301.11916),\u003cbr\u003e by *Xinyi Wang, Wanrong Zhu and William Yang Wang*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.13539) [**Finding Supporting Examples for In-Context Learning**](https://doi.org/10.48550/arXiv.2302.13539),\u003cbr\u003e by *Xiaonan Li and Xipeng Qiu*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2303.07895) [**The Learnability of In-Context Learning**](https://doi.org/10.48550/arXiv.2303.07895),\u003cbr\u003e by *Noam Wies, Yoav Levine and Amnon Shashua*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.14691) [**In-Context Instruction Learning**](https://doi.org/10.48550/arXiv.2302.14691),\u003cbr\u003e by *Seonghyeon Ye, Hyeonbin Hwang, Sohee Yang, Hyeongu Yun, Yireun Kim and Minjoon Seo*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.11521) [**How Does In-Context Learning Help Prompt Tuning?**](https://doi.org/10.48550/arXiv.2302.11521),\u003cbr\u003e by *Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu and Mohit Iyyer*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2303.13217) [**Fairness-guided Few-shot Prompting for Large Language Models**](https://doi.org/10.48550/arXiv.2303.13217),\u003cbr\u003e by *Huan Ma, Changqing Zhang, Yatao Bian, Lemao Liu, Zhirui Zhang, Peilin Zhao, Shu Zhang, Huazhu Fu et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2305.14128) [**Dr.ICL: Demonstration-Retrieved In-context Learning**](https://doi.org/10.48550/arXiv.2305.14128),\u003cbr\u003e by *Man Luo, Xin Xu, Zhuyun Dai, Panupong Pasupat, Seyed Mehran Kazemi, Chitta Baral, Vaiva Imbrasaite and Vincent Y. Zhao*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ACL-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2023.acl-long.256) [**Unified Demonstration Retriever for In-Context Learning**](https://doi.org/10.18653/v1/2023.acl-long.256),\u003cbr\u003e by *Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ESWC-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://ceur-ws.org/Vol-3447/Text2KG\\_Paper\\_9.pdf) [**Exploring In-Context Learning Capabilities of Foundation Models for\r\nGenerating Knowledge Graphs from Text**](https://ceur-ws.org/Vol-3447/Text2KG\\_Paper\\_9.pdf),\u003cbr\u003e by *Hanieh Khorashadizadeh, Nandana Mihindukulasooriya, Sanju Tiwari, Jinghua Groppe and Sven Groppe*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2307.01137) [**Exploring the In-context Learning Ability of Large Language Model\r\nfor Biomedical Concept Linking**](https://doi.org/10.48550/arXiv.2307.01137),\u003cbr\u003e by *Qinyong Wang, Zhenxiang Gao and Rong Xu*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2305.14726) [**In-Context Demonstration Selection with Cross Entropy Difference**](https://doi.org/10.48550/arXiv.2305.14726),\u003cbr\u003e by *Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Yichong Xu and Chenguang Zhu*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/the_61st_Annual_Meeting_of_the_Association_for_Computational\r\nLinguistics_(Volume_2:_Short_Papers),_{ACL}_2023,_Toronto,_Canada,\r\nJuly_9--14,_2023-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2023.acl-short.43) [**MetaVL: Transferring In-Context Learning Ability From Language Models\r\nto Vision-Language Models**](https://doi.org/10.18653/v1/2023.acl-short.43),\u003cbr\u003e by *Masoud Monajatipoor, Liunian Harold Li, Mozhdeh Rouhsedaghat, Lin Yang and Kai-Wei Chang*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2307.07742) [**SINC: Self-Supervised In-Context Learning for Vision-Language Tasks**](https://doi.org/10.48550/arXiv.2307.07742),\u003cbr\u003e by *Yi-Syuan Chen, Yun-Zhu Song, Cheng Yu Yeo, Bei Liu, Jianlong Fu and Hong-Han Shuai*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/pdf/2303.08119.pdf) [**How Many Demonstrations Do You Need for In-context Learning?**](https://arxiv.org/pdf/2303.08119.pdf),\u003cbr\u003e by *Jiuhai Chen, Lichang Chen, Chen Zhu and Tianyi Zhou*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2305.12766) [**Explaining Emergent In-Context Learning as Kernel Regression**](https://arxiv.org/abs/2305.12766),\u003cbr\u003e by *Chi Han, Ziqi Wang, Han Zhao and Heng Ji*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NeurIPS-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](http://papers.nips.cc/paper\\_files/paper/2023/hash/cda04d7ea67ea1376bf8c6962d8541e0-Abstract-Conference.html) [**Meta-in-context learning in large language models**](http://papers.nips.cc/paper\\_files/paper/2023/hash/cda04d7ea67ea1376bf8c6962d8541e0-Abstract-Conference.html),\u003cbr\u003e by *Julian Coda-Forno, Marcel Binz, Zeynep Akata, Matt M. Botvinick, Jane X. Wang and Eric Schulz*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2311.08993) [**When does In-context Learning Fall Short and Why? A Study on Specification-Heavy\r\nTasks**](https://doi.org/10.48550/arXiv.2311.08993),\u003cbr\u003e by *Hao Peng, Xiaozhi Wang, Jianhui Chen, Weikai Li, Yunjia Qi, Zimu Wang, Zhili Wu, Kaisheng Zeng et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ACL-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2022.acl-long.53) [**Meta-learning via Language Model In-context Tuning**](https://doi.org/10.18653/v1/2022.acl-long.53), [\u003cimg src=https://img.shields.io/badge/BERT-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/N19-1423/) [\u003cimg src=https://img.shields.io/badge/DeBERTa-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2006.03654) [\u003cimg src=https://img.shields.io/badge/GPT--2-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) \u003cbr\u003e by *Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis and He He*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NAACL-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2022.naacl-main.201) [**MetaICL: Learning to Learn In Context**](https://doi.org/10.18653/v1/2022.naacl-main.201), [\u003cimg src=https://img.shields.io/badge/Code-skyblue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://github.com/facebookresearch/MetaICL) [\u003cimg src=https://img.shields.io/badge/GPT--2-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) \u003cbr\u003e by *Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi*\r\n\u003cbr\u003e``\r\nMetaICL proposes a supervised meta-training framework to enable LMs to more effectively learn a new task in context. In MetaICL, each meta-training example includes several training examples from one task that will be presented together as a single sequence to the LM, and the prediction of the final example is used to calculate the loss.\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2209.01975) [**Selective Annotation Makes Language Models Better Few-Shot Learners**](https://doi.org/10.48550/arXiv.2209.01975), [\u003cimg src=https://img.shields.io/badge/Code-skyblue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://github.com/HKUNLP/icl-selective-annotation) [\u003cimg src=https://img.shields.io/badge/SBERT-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/D19-1410/) [\u003cimg src=https://img.shields.io/badge/GPT--J-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://huggingface.co/docs/transformers/model_doc/gptj) [\u003cimg src=https://img.shields.io/badge/GPT--Neo-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://huggingface.co/docs/transformers/model_doc/gpt_neo) [\u003cimg src=https://img.shields.io/badge/GPT--3-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html) [\u003cimg src=https://img.shields.io/badge/Codex-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2107.03374) [\u003cimg src=https://img.shields.io/badge/OPT-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2205.01068) \u003cbr\u003e by *Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf et al.*\r\n\u003cbr\u003e``\r\nThis paper proposes a graph-based selective annotation method named vote-k to\r\n``\u003cbr\u003e``\r\n(1) select a pool of examples to annotate from unlabeled data,\r\n``\u003cbr\u003e``\r\n(2) retrieve prompts (contexts) from the annotated data pool for in-context learning.\r\n``\u003cbr\u003e``\r\nSpecifically, the selection method first selects a small set of unlabeled examples iteratively and then labels them to serve as contexts for LLMs to predict the labels of the rest unlabeled data. The method selects the predictions with highest confidence (log probability of generation output) to fill up the selective annotation pool.\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NAACL-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2022.naacl-main.260) [**Improving In-Context Few-Shot Learning via Self-Supervised Training**](https://doi.org/10.18653/v1/2022.naacl-main.260), [\u003cimg src=https://img.shields.io/badge/MoE-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2022.naacl-main.260) \u003cbr\u003e by *Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov and Zornitsa Kozareva*\r\n\u003cbr\u003e``\r\nThis paper proposes to use self-supervision (MLM, NSP, CL, etc.) between pre-training and downstream usage to teach the LM to perform in-context learning. Analysis reveals that:\r\n``\u003cbr\u003e``\r\n(1) benefits of self-supervised depends on the amount of training data,\r\n``\u003cbr\u003e``\r\n(2) semantic similarity between training and evaluation tasks matters,\r\n``\u003cbr\u003e``\r\n(3) adding training objectives without diversity does not help,\r\n``\u003cbr\u003e``\r\n(4) model performance improves when choosing similar templates for both self-supervised and downstream tasks,\r\n``\u003cbr\u003e``\r\n(5) self-supervised  tasks and human-annotated datasets are complementary,\r\n``\u003cbr\u003e``\r\n(6) self-supervised-trained models are better at following task instructions.\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2205.10782) [**Instruction Induction: From Few Examples to Natural Language Task\r\nDescriptions**](https://doi.org/10.48550/arXiv.2205.10782),\u003cbr\u003e by *Or Honovich, Uri Shaham, Samuel R. Bowman and Omer Levy*\r\n\u003cbr\u003e``\r\n(1) 探索了利用LLM在几个样本的情况下归纳出任务指令的能力；\r\n``\u003cbr\u003e``\r\n(2) 测量两个指标：1. 模型归纳指令与人类归纳的指令对比，2. 利用模型归纳的指令作为prompt进行预测的执行准确率；\r\n``\u003cbr\u003e``\r\n(3) 相比于GPT-3，InstructGPT效果更好，理所当然。\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ACL-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2022.acl-long.556) [**Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot\r\nPrompt Order Sensitivity**](https://doi.org/10.18653/v1/2022.acl-long.556), [\u003cimg src=https://img.shields.io/badge/GPT--2-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) [\u003cimg src=https://img.shields.io/badge/GPT--3-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html) \u003cbr\u003e by *Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel and Pontus Stenetorp*\r\n\u003cbr\u003e``\r\n(1) This work demonstrates that few-shot prompts suffer from order sensitivity, in that for the same prompt the order in which samples are provided can make a difference to model performance.\r\n``\u003cbr\u003e``\r\n(2) This work introduces a probing method which constructs an artificial development set by language models themselves to alleviate the order sensitivity problem.\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NAACL-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2022.naacl-main.191) [**Learning To Retrieve Prompts for In-Context Learning**](https://doi.org/10.18653/v1/2022.naacl-main.191), [\u003cimg src=https://img.shields.io/badge/GPT--3-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html) [\u003cimg src=https://img.shields.io/badge/GPT--Neo-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://huggingface.co/docs/transformers/model_doc/gpt_neo) [\u003cimg src=https://img.shields.io/badge/Codex-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2107.03374) [\u003cimg src=https://img.shields.io/badge/GPT--J-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://huggingface.co/docs/transformers/model_doc/gptj) [\u003cimg src=https://img.shields.io/badge/SBERT-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/D19-1410/) [\u003cimg src=https://img.shields.io/badge/BERT-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/N19-1423/) \u003cbr\u003e by *Ohad Rubin, Jonathan Herzig and Jonathan Berant*\r\n\u003cbr\u003e``\r\nThis paper proposes a method to retrieve good contexts for in-context learning. Specifically, the method\r\n``\u003cbr\u003e``\r\n(1) uses an unsupervised retriever (BM25/SBERT) to obtain a set of context candidates,\r\n``\u003cbr\u003e``\r\n(2) passes the candidates to a scoring model (GPT-Neo/GPT-J/GPT-3/Codex) and select the top/bottom k as positive/negative examples,\r\n``\u003cbr\u003e``\r\n(3) uses the examples to train a dense retriever (BERT-based).\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.emnlp-main.622) [**Active Example Selection for In-Context Learning**](https://aclanthology.org/2022.emnlp-main.622), [\u003cimg src=https://img.shields.io/badge/Code-skyblue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://github.com/ChicagoHAI/active-example-selection) [\u003cimg src=https://img.shields.io/badge/GPT--2-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) [\u003cimg src=https://img.shields.io/badge/GPT--3-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html) \u003cbr\u003e by *Yiming Zhang, Shi Feng and Chenhao Tan*\r\n\u003cbr\u003e``\r\n(1) This paper revisits the  effect of example selection (re-ordering \u0026 calibration) for ICL, observing that a large variance across set of demonstration examples still exists.\r\n``\u003cbr\u003e``\r\n(2) This paper applies reinforcement learning (Q-Learning) to optimize example selection by formulating this task as sequential decision-making problem, which is appropriate for example selection from unlabeled datasets. \r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2206.08082) [**Self-Generated In-Context Learning: Leveraging Auto-regressive Language\r\nModels as a Demonstration Generator**](https://doi.org/10.48550/arXiv.2206.08082),\u003cbr\u003e by *Hyuhng Joon Kim, Hyunsoo Cho, Junyeob Kim, Taeuk Kim, Kang Min Yoo and Sang-goo Lee*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ISoLA-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.1007/978-3-031-19759-8\\_15) [**Measuring Convergence Inertia: Online Learning in Self-adaptive Systems\r\nwith Context Shifts**](https://doi.org/10.1007/978-3-031-19759-8\\_15),\u003cbr\u003e by *Elvin Alberts and Ilias Gerostathopoulos*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ICLR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://openreview.net/forum?id=RdJVFCHjUMI) [**An Explanation of In-context Learning as Implicit Bayesian Inference**](https://openreview.net/forum?id=RdJVFCHjUMI),\u003cbr\u003e by *Sang Michael Xie, Aditi Raghunathan, Percy Liang and Tengyu Ma*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.emnlp-main.759) [**Rethinking the Role of Demonstrations: What Makes In-Context Learning\r\nWork?**](https://aclanthology.org/2022.emnlp-main.759),\u003cbr\u003e by *Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi and Luke Zettlemoyer*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.08686) [**The Impact of Symbolic Representations on In-context Learning for\r\nFew-shot Reasoning**](https://doi.org/10.48550/arXiv.2212.08686),\u003cbr\u003e by *Hanlin Zhang, Yi-Fan Zhang, Li Erran Li and Eric P. Xing*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NAACL-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2022.deelio-1.10) [**What Makes Good In-Context Examples for GPT-3?**](https://doi.org/10.18653/v1/2022.deelio-1.10), [\u003cimg src=https://img.shields.io/badge/Code-skyblue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://github.com/jiachangliu/KATEGPT3) [\u003cimg src=https://img.shields.io/badge/RoBERTa-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/1907.11692) [\u003cimg src=https://img.shields.io/badge/T5-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](http://jmlr.org/papers/v21/20-074.html) [\u003cimg src=https://img.shields.io/badge/SBERT-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/D19-1410/) [\u003cimg src=https://img.shields.io/badge/GPT--3-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html) \u003cbr\u003e by *Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin and Weizhu Chen*\r\n\u003cbr\u003e``\r\n(1) 探索了在in-context learning中什么样的demonstration example可以对GPT-3的效果取得帮助；\r\n``\u003cbr\u003e``\r\n(2) 利用roberta对样本进行编码，并计算demonstration与test example的向量距离（欧氏距离），最终发现与test example越相近的demonstration越能取得较好的效果。\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP_Findings-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.findings-emnlp.329) [**Thinking about GPT-3 In-Context Learning for Biomedical IE? Think\r\nAgain**](https://aclanthology.org/2022.findings-emnlp.329),\u003cbr\u003e by *Bernal Jimenez Gutierrez, Nikolas McNeal, Clayton Washington, You Chen, Lang Li, Huan Sun and Yu Su*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.10559) [**Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient\r\nDescent as Meta-Optimizers**](https://doi.org/10.48550/arXiv.2212.10559),\u003cbr\u003e by *Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui and Furu Wei*\r\n\u003cbr\u003e``\r\n(1) 与The Dual Form of Neural Networks Revisited结合一起看，可以进一步理解in-context learning，通过与NN线性层对偶形式的类比，可以将ICL流程描述为：1. 基于Transformer的预训练语言模型作为元优化器；2. 通过正向计算，根据示范例子产生元梯度；3. 通过关注，将元梯度应用于原始语言模型，建立一个ICL模型；\r\n``\u003cbr\u003e``\r\n(2)与Fine-tune类似，ICL也是在zero-shot learning参数的基础上，提供了一个更新量。\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ICML-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://proceedings.mlr.press/v162/irie22a.html) [**The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions\r\nto Training Patterns via Spotlights of Attention**](https://proceedings.mlr.press/v162/irie22a.html),\u003cbr\u003e by *Kazuki Irie, R\\'obert Csord\\'as and J\\\"urgen Schmidhuber*\r\n\u003cbr\u003e``\r\n(1) 很有意思的一篇，回顾神经网络（NN）线性层Y=WX（省略偏置b）的原始形式与对偶形式，两种形式完全等价；\r\n``\u003cbr\u003e``\r\n(2) 从对偶形式中可以发现，通过反向传播训练的NN线性层的输出主要是该层在训练期间的训练误差信号et的线性组合，其中权重是通过比较测试查询x和每个训练输入计算出来的；进一步可以得出，如果测试时输入的x和训练时的输入是正交的，那么梯度下降所得到的参数更新对于该样本x完全没有影响。\r\n``\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.10375) [**Self-adaptive In-context Learning**](https://doi.org/10.48550/arXiv.2212.10375),\u003cbr\u003e by *Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye and Lingpeng Kong*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.10378) [**Careful Data Curation Stabilizes In-context Learning**](https://doi.org/10.48550/arXiv.2212.10378),\u003cbr\u003e by *Ting-Yun Chang and Robin Jia*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.09095) [**Rethinking the Role of Scale for In-Context Learning: An Interpretability-based\r\nCase Study at 66 Billion Scale**](https://doi.org/10.48550/arXiv.2212.09095),\u003cbr\u003e by *Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, Sravan Bodapati, Katrin Kirchhoff and Dan Roth*\r\n\u003cbr\u003e\u003cbr\u003e\r\n### Instruction Tuning\r\n\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2402.18243) [**Learning or Self-aligning? Rethinking Instruction Fine-tuning**](https://doi.org/10.48550/arXiv.2402.18243),\u003cbr\u003e by *Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Guanglu Wan, Xunliang Cai et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/AAAI-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.1609/aaai.v38i16.29777) [**Can Large Language Models Understand Real-World Complex Instructions?**](https://doi.org/10.1609/aaai.v38i16.29777),\u003cbr\u003e by *Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Jiaqing Liang et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2303.10475) [**Large Language Model Instruction Following: A Survey of Progresses and Challenges**](https://arxiv.org/abs/2303.10475),\u003cbr\u003e by *Renze Lou, Kai Zhang and Wenpeng Yin*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.00093) [**Large Language Models Can Be Easily Distracted by Irrelevant Context**](https://doi.org/10.48550/arXiv.2302.00093),\u003cbr\u003e by *Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Sch\\\"arli and Denny Zhou*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.07459) [**The Capacity for Moral Self-Correction in Large Language Models**](https://doi.org/10.48550/arXiv.2302.07459),\u003cbr\u003e by *Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamile Lukosiute, Anna Chen, Anna Goldie, Azalia Mirhoseini et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2302.03202) [**Exploring the Benefits of Training Expert Language Models over Instruction Tuning**](https://arxiv.org/abs/2302.03202),\u003cbr\u003e by *Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee and Minjoon Seo*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2304.07987) [**Chinese Open Instruction Generalist: A Preliminary Release**](https://doi.org/10.48550/arXiv.2304.07987),\u003cbr\u003e by *Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2304.03277) [**Instruction Tuning with GPT-4**](https://doi.org/10.48550/arXiv.2304.03277),\u003cbr\u003e by *Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley and Jianfeng Gao*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NeurIPS-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](http://papers.nips.cc/paper\\_files/paper/2023/hash/6dcf277ea32ce3288914faf369fe6de0-Abstract-Conference.html) [**Visual Instruction Tuning**](http://papers.nips.cc/paper\\_files/paper/2023/hash/6dcf277ea32ce3288914faf369fe6de0-Abstract-Conference.html),\u003cbr\u003e by *Haotian Liu, Chunyuan Li, Qingyang Wu and Yong Jae Lee*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NeurIPS-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](http://papers.nips.cc/paper\\_files/paper/2023/hash/9a6a435e75419a836fe47ab6793623e6-Abstract-Conference.html) [**InstructBLIP: Towards General-purpose Vision-Language Models with\r\nInstruction Tuning**](http://papers.nips.cc/paper\\_files/paper/2023/hash/9a6a435e75419a836fe47ab6793623e6-Abstract-Conference.html),\u003cbr\u003e by *Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NeurIPS-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](http://papers.nips.cc/paper\\_files/paper/2023/hash/e393677793767624f2821cec8bdd02f1-Abstract-Conference.html) [**GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction**](http://papers.nips.cc/paper\\_files/paper/2023/hash/e393677793767624f2821cec8bdd02f1-Abstract-Conference.html),\u003cbr\u003e by *Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li and Ying Shan*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NeurIPS-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](http://papers.nips.cc/paper\\_files/paper/2023/hash/548a41b9cac6f50dccf7e63e9e1b1b9b-Abstract-Datasets\\_and\\_Benchmarks.html) [**LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset,\r\nFramework, and Benchmark**](http://papers.nips.cc/paper\\_files/paper/2023/hash/548a41b9cac6f50dccf7e63e9e1b1b9b-Abstract-Datasets\\_and\\_Benchmarks.html),\u003cbr\u003e by *Zhenfei Yin, Jiong Wang, Jianjian Cao, Zhelun Shi, Dingning Liu, Mukai Li, Xiaoshui Huang, Zhiyong Wang et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ICLR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://openreview.net/forum?id=gEZrGCozdqR) [**Finetuned Language Models are Zero-Shot Learners**](https://openreview.net/forum?id=gEZrGCozdqR),\u003cbr\u003e by *Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2201.08239) [**LaMDA: Language Models for Dialog Applications**](https://arxiv.org/abs/2201.08239),\u003cbr\u003e by *Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2210.11416) [**Scaling Instruction-Finetuned Language Models**](https://doi.org/10.48550/arXiv.2210.11416),\u003cbr\u003e by *Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.emnlp-main.340) [**Super-NaturalInstructions: Generalization via Declarative Instructions\r\non 1600+ NLP Tasks**](https://aclanthology.org/2022.emnlp-main.340),\u003cbr\u003e by *Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.10560) [**Self-Instruct: Aligning Language Model with Self Generated Instructions**](https://doi.org/10.48550/arXiv.2212.10560),\u003cbr\u003e by *Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi and Hannaneh Hajishirzi*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2203.09161) [**How Many Data Samples is an Additional Instruction Worth?**](https://doi.org/10.48550/arXiv.2203.09161),\u003cbr\u003e by *Ravsehaj Singh Puri, Swaroop Mishra, Mihir Parmar and Chitta Baral*\r\n\u003cbr\u003e\u003cbr\u003e\r\n### RLHF\r\n\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.07459) [**The Capacity for Moral Self-Correction in Large Language Models**](https://doi.org/10.48550/arXiv.2302.07459),\u003cbr\u003e by *Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamile Lukosiute, Anna Chen, Anna Goldie, Azalia Mirhoseini et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.12192) [**Aligning Text-to-Image Models using Human Feedback**](https://doi.org/10.48550/arXiv.2302.12192),\u003cbr\u003e by *Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2307.04964) [**Secrets of RLHF in Large Language Models Part I: PPO**](https://doi.org/10.48550/arXiv.2307.04964),\u003cbr\u003e by *Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei Shen, Binghai Wang, Yan Liu, Senjie Jin et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2307.15217) [**Open Problems and Fundamental Limitations of Reinforcement Learning\r\nfrom Human Feedback**](https://doi.org/10.48550/arXiv.2307.15217),\u003cbr\u003e by *Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, J\\'er\\'emy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2309.00267) [**RLAIF: Scaling Reinforcement Learning from Human Feedback with AI\r\nFeedback**](https://doi.org/10.48550/arXiv.2309.00267),\u003cbr\u003e by *Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune and Abhinav Rastogi*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2204.05862) [**Training a Helpful and Harmless Assistant with Reinforcement Learning\r\nfrom Human Feedback**](https://doi.org/10.48550/arXiv.2204.05862),\u003cbr\u003e by *Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2210.01241) [**Is Reinforcement Learning (Not) for Natural Language Processing?:\r\nBenchmarks, Baselines, and Building Blocks for Natural Language Policy\r\nOptimization**](https://doi.org/10.48550/arXiv.2210.01241),\u003cbr\u003e by *Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kiant\\'e Brantley, Jack Hessel, Rafet Sifa, Christian Bauckhage, Hannaneh Hajishirzi and Yejin Choi*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2203.11147) [**Teaching language models to support answers with verified quotes**](https://doi.org/10.48550/arXiv.2203.11147),\u003cbr\u003e by *Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, H. Francis Song, Martin Chadwick, Mia Glaese, Susannah Young et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2209.14375) [**Improving alignment of dialogue agents via targeted human judgements**](https://doi.org/10.48550/arXiv.2209.14375),\u003cbr\u003e by *Amelia Glaese, Nat McAleese, Maja Trebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2210.10760) [**Scaling Laws for Reward Model Overoptimization**](https://arxiv.org/abs/2210.10760),\u003cbr\u003e by *Gao, Leo, Schulman, John and Hilton, Jacob*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2209.07858) [**Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors,\r\nand Lessons Learned**](https://doi.org/10.48550/arXiv.2209.07858),\u003cbr\u003e by *Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2208.02294) [**Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning**](https://doi.org/10.48550/arXiv.2208.02294),\u003cbr\u003e by *Deborah Cohen, Moonkyung Ryu, Yinlam Chow, Orgad Keller, Ido Greenberg, Avinatan Hassidim, Michael Fink, Yossi Matias et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2203.02155) [**Training language models to follow instructions with human feedback**](https://doi.org/10.48550/arXiv.2203.02155),\u003cbr\u003e by *Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/IJRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.1177/02783649211041652) [**Learning reward functions from diverse sources of human feedback:\r\nOptimally integrating demonstrations and preferences**](https://doi.org/10.1177/02783649211041652),\u003cbr\u003e by *Erdem Biyik, Dylan P. Losey, Malayandi Palan, Nicholas C. Landolfi, Gleb Shevchuk and Dorsa Sadigh*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2021-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2112.09332) [**WebGPT: Browser-assisted question-answering with human feedback**](https://arxiv.org/abs/2112.09332),\u003cbr\u003e by *Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2021-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2109.10862) [**Recursively Summarizing Books with Human Feedback**](https://arxiv.org/abs/2109.10862),\u003cbr\u003e by *Jeff Wu, Long Ouyang, Daniel M. Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike and Paul F. Christiano*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NeurIPS-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://proceedings.neurips.cc/paper/2020/hash/1f89885d556929e98d3ef9b86448f951-Abstract.html) [**Learning to summarize with human feedback**](https://proceedings.neurips.cc/paper/2020/hash/1f89885d556929e98d3ef9b86448f951-Abstract.html),\u003cbr\u003e by *Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2020.emnlp-main.28) [**Dialogue Response Ranking Training with Large-Scale Human Feedback\r\nData**](https://doi.org/10.18653/v1/2020.emnlp-main.28),\u003cbr\u003e by *Xiang Gao, Yizhe Zhang, Michel Galley, Chris Brockett and Bill Dolan*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2019-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](http://arxiv.org/abs/1909.08593) [**Fine-Tuning Language Models from Human Preferences**](http://arxiv.org/abs/1909.08593),\u003cbr\u003e by *Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul F. Christiano and Geoffrey Irving*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NeurIPS-2017-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://proceedings.neurips.cc/paper/2017/hash/d5e2c0adad503c91f91df240d0cd4e49-Abstract.html) [**Deep Reinforcement Learning from Human Preferences**](https://proceedings.neurips.cc/paper/2017/hash/d5e2c0adad503c91f91df240d0cd4e49-Abstract.html),\u003cbr\u003e by *Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg and Dario Amodei*\r\n\u003cbr\u003e\u003cbr\u003e\r\n### Pre-Training Techniques\r\n\r\n- [\u003cimg src=https://img.shields.io/badge/OpenAI-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/papers/gpt-4.pdf) [**GPT-4 Technical Report**](https://cdn.openai.com/papers/gpt-4.pdf), [\u003cimg src=https://img.shields.io/badge/GPT--4-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/papers/gpt-4.pdf) \u003cbr\u003e by *OpenAI*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/OpenAI-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/papers/gpt-4-system-card.pdf) [**GPT-4 System Card**](https://cdn.openai.com/papers/gpt-4-system-card.pdf), [\u003cimg src=https://img.shields.io/badge/GPT--4-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/papers/gpt-4.pdf) \u003cbr\u003e by *OpenAI*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2205.01068) [**OPT: Open Pre-trained Transformer Language Models**](https://doi.org/10.48550/arXiv.2205.01068), [\u003cimg src=https://img.shields.io/badge/OPT-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2205.01068) \u003cbr\u003e by *Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona T. Diab et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2209.10372) [**WeLM: A Well-Read Pre-trained Language Model for Chinese**](https://doi.org/10.48550/arXiv.2209.10372), [\u003cimg src=https://img.shields.io/badge/Code-skyblue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://welm.weixin.qq.com/docs/api/)\u003cbr\u003e by *Hui Su, Xiao Zhou, Houjin Yu, Yuwen Chen, Zilin Zhu, Yang Yu and Jie Zhou*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NeurIPS-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html) [**Language Models are Few-Shot Learners**](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html), [\u003cimg src=https://img.shields.io/badge/GPT--3-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html) \u003cbr\u003e by *Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/ICLR-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://openreview.net/forum?id=r1xMH1BtvB) [**ELECTRA: Pre-training Text Encoders as Discriminators Rather Than\r\nGenerators**](https://openreview.net/forum?id=r1xMH1BtvB), [\u003cimg src=https://img.shields.io/badge/ELECTRA-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://openreview.net/forum?id=r1xMH1BtvB) \u003cbr\u003e by *Kevin Clark, Minh-Thang Luong, Quoc V. Le and Christopher D. Manning*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP_Findings-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2020.findings-emnlp.58) [**Revisiting Pre-Trained Models for Chinese Natural Language Processing**](https://doi.org/10.18653/v1/2020.findings-emnlp.58),\u003cbr\u003e by *Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang and Guoping Hu*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2006.03654) [**DeBERTa: Decoding-enhanced BERT with Disentangled Attention**](https://arxiv.org/abs/2006.03654), [\u003cimg src=https://img.shields.io/badge/DeBERTa-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/2006.03654) \u003cbr\u003e by *Pengcheng He, Xiaodong Liu, Jianfeng Gao and Weizhu Chen*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/JMLR-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](http://jmlr.org/papers/v21/20-074.html) [**Exploring the Limits of Transfer Learning with a Unified Text-to-Text\r\nTransformer**](http://jmlr.org/papers/v21/20-074.html), [\u003cimg src=https://img.shields.io/badge/T5-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](http://jmlr.org/papers/v21/20-074.html) \u003cbr\u003e by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/TACL-2020-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.1162/tacl\\_a\\_00349) [**A Primer in BERTology: What We Know About How BERT Works**](https://doi.org/10.1162/tacl\\_a\\_00349),\u003cbr\u003e by *Anna Rogers, Olga Kovaleva and Anna Rumshisky*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/OpenAI-2019-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) [**Language Models are Unsupervised Multitask Learners**](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf), [\u003cimg src=https://img.shields.io/badge/GPT--2-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) \u003cbr\u003e by *Radford, Alec, Wu, Jeffrey, Child, Rewon, Luan, David, Amodei, Dario and Sutskever, Ilya*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NAACL-2019-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/n19-1423) [**BERT: Pre-training of Deep Bidirectional Transformers for Language\r\nUnderstanding**](https://doi.org/10.18653/v1/n19-1423), [\u003cimg src=https://img.shields.io/badge/BERT-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/N19-1423/) \u003cbr\u003e by *Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2019-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](http://arxiv.org/abs/1907.11692) [**RoBERTa: A Robustly Optimized BERT Pretraining Approach**](http://arxiv.org/abs/1907.11692), [\u003cimg src=https://img.shields.io/badge/RoBERTa-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://arxiv.org/abs/1907.11692) \u003cbr\u003e by *Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2019-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/D19-1410) [**Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks**](https://doi.org/10.18653/v1/D19-1410), [\u003cimg src=https://img.shields.io/badge/SBERT-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/D19-1410/) \u003cbr\u003e by *Nils Reimers and Iryna Gurevych*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/OpenAI-2018-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) [**Improving language understanding by generative pre-training**](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf), [\u003cimg src=https://img.shields.io/badge/GPT--1-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) \u003cbr\u003e by *Radford, Alec, Narasimhan, Karthik, Salimans, Tim, Sutskever, Ilya and others*\r\n\u003cbr\u003e\u003cbr\u003e\r\n#### Mixtures of Experts\r\n\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.emnlp-main.804) [**Efficient Large Scale Language Modeling with Mixtures of Experts**](https://aclanthology.org/2022.emnlp-main.804), [\u003cimg src=https://img.shields.io/badge/Code-skyblue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://github.com/facebookresearch/fairseq/tree/main/examples/moe_lm) [\u003cimg src=https://img.shields.io/badge/MoE-yellow alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.emnlp-main.804) \u003cbr\u003e by *Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NAACL-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.18653/v1/2022.naacl-main.116) [**MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation**](https://doi.org/10.18653/v1/2022.naacl-main.116),\u003cbr\u003e by *Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He, Tuo Zhao and Weizhu Chen*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.05055) [**Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints**](https://doi.org/10.48550/arXiv.2212.05055),\u003cbr\u003e by *Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani et al.*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2210.03885) [**Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts**](https://doi.org/10.48550/arXiv.2210.03885),\u003cbr\u003e by *Tao Zhong, Zhixiang Chi, Li Gu, Yang Wang, Yuanhao Yu and Jin Tang*\r\n\u003cbr\u003e\u003cbr\u003e\r\n### Knowledge Enhanced\r\n\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2024-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2401.05669) [**ConcEPT: Concept-Enhanced Pre-Training for Language Models**](https://doi.org/10.48550/arXiv.2401.05669),\u003cbr\u003e by *Xintao Wang, Zhouhong Gu, Jiaqing Liang, Dakuan Lu, Yanghua Xiao and Wei Wang*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2023-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2302.02093) [**Knowledge-enhanced Neural Machine Reasoning: A Review**](https://doi.org/10.48550/arXiv.2302.02093),\u003cbr\u003e by *Tanmoy Chowdhury, Chen Ling, Xuchao Zhang, Xujiang Zhao, Guangji Bai, Jian Pei, Haifeng Chen and Liang Zhao*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/NeurIPS-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2210.09338) [**Deep Bidirectional Language-Knowledge Graph Pretraining**](https://doi.org/10.48550/arXiv.2210.09338),\u003cbr\u003e by *Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D. Manning, Percy Liang and Jure Leskovec*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/TKDE-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.13428) [**A Survey on Knowledge-Enhanced Pre-trained Language Models**](https://doi.org/10.48550/arXiv.2212.13428),\u003cbr\u003e by *Chaoqi Zhen, Yanlei Shang, Xiangyu Liu, Yifei Li, Yong Chen and Dell Zhang*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/FCST-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.3778/j.issn.1673-9418.2108105) [**Review of Knowledge-Enhanced Pre-trained Language Models**](https://doi.org/10.3778/j.issn.1673-9418.2108105),\u003cbr\u003e by *Yi, HAN, Linbo, QIAO, Dongsheng, LI and Xiangke, LIAO*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/CoRR-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://doi.org/10.48550/arXiv.2212.09252) [**Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue\r\nSystems**](https://doi.org/10.48550/arXiv.2212.09252),\u003cbr\u003e by *Sagi Shaier, Lawrence Hunter and Katharina Kann*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/COLING-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.coling-1.85) [**A Domain Knowledge Enhanced Pre-Trained Language Model for Vertical\r\nSearch: Case Study on Medicinal Products**](https://aclanthology.org/2022.coling-1.85),\u003cbr\u003e by *Kesong Liu, Jianhui Jiang and Feifei Lyu*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.shields.io/badge/EMNLP-2022-blue alt=\"img\" style=\"zoom:100%; vertical-align: middle\" /\u003e](https://aclanthology.org/2022.emnlp-main.207) [**Knowledge Prompting in Pre-trained Language Model for Natural Language\r\nUnderstanding**](https://aclanthology.org/2022.emnlp-main.207),\u003cbr\u003e by *Jianing Wang, Wenkang Huang, Minghui Qiu, Qiuhui Shi, Hongbin Wang, Xiang Li and Ming Gao*\r\n\u003cbr\u003e\u003cbr\u003e\r\n- [\u003cimg src=https://img.s","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSEU-COIN%2FLLMPapers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSEU-COIN%2FLLMPapers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSEU-COIN%2FLLMPapers/lists"}