{"id":13623604,"url":"https://github.com/lmmlzn/Awesome-LLMs-Datasets","last_synced_at":"2025-04-15T15:30:33.466Z","repository":{"id":217596926,"uuid":"744306661","full_name":"lmmlzn/Awesome-LLMs-Datasets","owner":"lmmlzn","description":"Summarize existing representative LLMs text datasets.","archived":false,"fork":false,"pushed_at":"2025-03-25T10:30:58.000Z","size":1331,"stargazers_count":1233,"open_issues_count":4,"forks_count":126,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-10T09:08:51.573Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lmmlzn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-01-17T02:52:51.000Z","updated_at":"2025-04-08T10:24:53.000Z","dependencies_parsed_at":"2024-02-15T04:25:06.031Z","dependency_job_id":"bc4f0a8d-4dee-4bfc-80dd-2a31ec2483ca","html_url":"https://github.com/lmmlzn/Awesome-LLMs-Datasets","commit_stats":null,"previous_names":["lmmlzn/awesome-llms-datasets"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmmlzn%2FAwesome-LLMs-Datasets","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmmlzn%2FAwesome-LLMs-Datasets/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmmlzn%2FAwesome-LLMs-Datasets/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmmlzn%2FAwesome-LLMs-Datasets/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lmmlzn","download_url":"https://codeload.github.com/lmmlzn/Awesome-LLMs-Datasets/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249097708,"owners_count":21212339,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T21:01:33.672Z","updated_at":"2025-04-15T15:30:33.437Z","avatar_url":"https://github.com/lmmlzn.png","language":null,"funding_links":[],"categories":["Others","NLP语料和数据集","NLP","Datasets","Topics","Other Lists","Natural Language Processing"],"sub_categories":["其他_文本生成、文本对话","数据集","LLM Training Datasets","TeX Lists"],"readme":"\u003cdiv align=\"center\"\u003e\n    \u003ch1\u003eAwesome LLMs Datasets\u003c/h1\u003e\n\u003c/div\u003e\n\n- Summarize existing representative LLMs text datasets across five dimensions: **Pre-training Corpora, Fine-tuning Instruction Datasets, Preference Datasets, Evaluation Datasets, and Traditional NLP Datasets**. (Regular updates)\n- New dataset sections have been added: **Multi-modal Large Language Models (MLLMs) Datasets, Retrieval Augmented Generation (RAG) Datasets**. (Gradual updates)\n- To improve update efficiency and frequency, starting from 2025, only key details of datasets will be updated (e.g., dataset name, paper link, etc.). For more comprehensive information, please refer to the corresponding research papers.\n\n## Paper\nThe paper **[\"Datasets for Large Language Models: A Comprehensive Survey\"](https://arxiv.org/abs/2402.18041)** has been released.（2024/2）\n\n**Abstract:**\n\nThis paper embarks on an exploration into the Large Language Model (LLM) datasets, which play a crucial role in the remarkable advancements of LLMs. The datasets serve as the foundational infrastructure analogous to a root system that sustains and nurtures the development of LLMs. Consequently, examination of these datasets emerges as a critical topic in research. In order to address the current lack of a comprehensive overview and thorough analysis of LLM datasets, and to gain insights into their current status and future trends, this survey consolidates and categorizes the fundamental aspects of LLM datasets from five perspectives: (1) Pre-training Corpora; (2) Instruction Fine-tuning Datasets; (3) Preference Datasets; (4) Evaluation Datasets; (5) Traditional Natural Language Processing (NLP) Datasets. The survey sheds light on the prevailing challenges and points out potential avenues for future investigation. Additionally, a comprehensive review of the existing available dataset resources is also provided, including statistics from 444 datasets, covering 8 language categories and spanning 32 domains. Information from 20 dimensions is incorporated into the dataset statistics. The total data size surveyed surpasses 774.5 TB for pre-training corpora and 700M instances for other datasets. We aim to present the entire landscape of LLM text datasets, serving as a comprehensive reference for researchers in this field and contributing to future studies.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"Fig_1.jpg\" width=\"800\"/\u003e\n\u003cp\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003cstrong\u003eFig 1. The overall architecture of the survey. Zoom in for better view\u003c/strong\u003e\n\u003cp\u003e\n  \n## Dataset Information Module\nThe following is a summary of the dataset information module.\n\n- Corpus/Dataset name\n- Publisher\n- Release Time\n  - “X” indicates unknown month. \n- Size\n- Public or Not\n  - “All” indicates full open source; \n  - “Partial” indicates partially open source; \n  - “Not” indicates not open source. \n- License\n- Language\n  - “EN” indicates English;\n  - “ZH” indicates Chinese;\n  - “AR” indicates Arabic;\n  - “ES” indicates Spanish;\n  - “RU” indicates Russian;\n  - “DE” indicates German;\n  - “KO” indicates Korean;\n  - “LT” indicates Lithuanian;\n  - “FA” indicates Persian/Farsi;\n  - “PL” indicates Programming Language;\n  - “Multi” indicates Multilingual, and the number in parentheses indicates the number of languages included. \n- Construction Method\n  - “HG” indicates Human Generated Corpus/Dataset;\n  - “MC” indicates Model Constructed Corpus/Dataset;\n  - “CI” indicates Collection and Improvement of Existing Corpus/Dataset.\n- Category\n- Source\n- Domain\n- Instruction Category\n- Preference Evaluation Method\n  - “VO” indicates Vote;\n  - “SO” indicates Sort;\n  - “SC” indicates Score;\n  - “-H” indicates Conducted by Humans;\n  - “-M” indicates Conducted by Models.\n- Question Type\n  - “SQ” indicates Subjective Questions;\n  - “OQ” indicates Objective Questions;\n  - “Multi” indicates Multiple Question Types.\n- Evaluation Method\n  - “CE” indicates Code Evaluation;\n  - “HE” indicates Human Evaluation;\n  - “ME” indicates Model Evaluation.\n- Focus\n- Numbers of Evaluation Categories/Subcategories\n- Evaluation Category\n- Number of Entity Categories (NER Task)\n- Number of Relationship Categories (RE Task)\n\n## Changelog\n\u003cdetails\u003e\n\u003csummary\u003eClick to expand and view hidden content.\u003c/summary\u003e\n    \n- （2024/01/17）Create the **Awesome-LLMs-Datasets** dataset repository.\n- （2024/02/02）Revise information for some datasets; add **[Dolma](https://browse.arxiv.org/abs/2402.00159)** (Pre-training Corpora | General Pre-training Corpora | Multi-category).\n- （2024/02/15）Add **[Aya Collection](https://arxiv.org/abs/2402.06619)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | HG \u0026 CI \u0026 MC); **[Aya Dataset](https://arxiv.org/abs/2402.06619)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | HG).\n- （2024/02/22）Add **[OpenMathInstruct-1](https://arxiv.org/abs/2402.10176)** (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Math); **[FinBen](https://arxiv.org/abs/2402.12659)** (Evaluation Datasets | Financial).\n- （2024/04/05）\n  - Add new dataset sections: **(1) Multi-modal Large Language Models (MLLMs) Datasets; (2) Retrieval Augmented Generation (RAG) Datasets**.\n  - Add **[MMRS-1M](https://arxiv.org/abs/2401.16822)** (MLLMs Datasets | Instruction Fine-tuning Datasets); **[VideoChat2-IT](https://arxiv.org/abs/2311.17005)** (MLLMs Datasets | Instruction Fine-tuning Datasets); **[InstructDoc](https://arxiv.org/abs/2401.13313)** (MLLMs Datasets | Instruction Fine-tuning Datasets); **[ALLaVA-4V Data](https://arxiv.org/abs/2402.11684)** (MLLMs Datasets | Instruction Fine-tuning Datasets); **[MVBench](https://arxiv.org/abs/2311.17005)** (MLLMs Datasets | Evaluation Datasets); **[OlympiadBench](https://arxiv.org/abs/2402.14008)** (MLLMs Datasets | Evaluation Datasets); **[MMMU](https://arxiv.org/abs/2311.16502)** (MLLMs Datasets | Evaluation Datasets).\n  - Add **[CLUE Benchmark Series](https://github.com/CLUEbenchmark)** (Evaluation Datasets | Evaluation Platform); **[OpenLLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)** (Evaluation Datasets | Evaluation Platform); **[OpenCompass](https://opencompass.org.cn/home)** (Evaluation Datasets | Evaluation Platform); **[MTEB Leaderboard](https://huggingface.co/spaces/mteb/leaderboard)** (Evaluation Datasets | Evaluation Platform); **[C-MTEB Leaderboard](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB)** (Evaluation Datasets | Evaluation Platform).\n  - Add **[NAH (Needle-in-a-Haystack)](https://github.com/gkamradt/LLMTest_NeedleInAHaystack)** (Evaluation Datasets | Long Text); **[ToolEyes](https://arxiv.org/abs/2401.00741)** (Evaluation Datasets | Tool); **[UHGEval](https://arxiv.org/abs/2311.15296)** (Evaluation Datasets | Factuality); **[CLongEval](https://arxiv.org/abs/2403.03514)** (Evaluation Datasets | Long Text).\n  - Add **[MathPile](https://arxiv.org/abs/2312.17120)** (Pre-training Corpora | Domain-specific Pre-training Corpora | Math); **[WanJuan-CC](https://arxiv.org/abs/2402.19282)** (Pre-training Corpora | General Pre-training Corpora | Webpages).\n  - Add **[IEPile](https://arxiv.org/abs/2402.14710)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | CI); **[InstructIE](https://arxiv.org/abs/2305.11527)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | HG).\n  - Add **[CRUD-RAG](https://arxiv.org/abs/2401.17043)** (RAG Datasets); **[WikiEval](https://arxiv.org/abs/2309.15217)** (RAG Datasets); **[RGB](https://arxiv.org/abs/2309.01431)** (RAG Datasets); **[RAG-Instruct-Benchmark-Tester](https://huggingface.co/datasets/llmware/rag_instruct_benchmark_tester)** (RAG Datasets); **[ARES](https://arxiv.org/abs/2311.09476)** (RAG Datasets).\n- （2024/04/06）\n  - Add **[GPQA](https://arxiv.org/abs/2311.12022)** (Evaluation Datasets | Subject); **[MGSM](https://arxiv.org/abs/2210.03057)** (Evaluation Datasets | Multilingual); **[HaluEval-Wild](https://arxiv.org/abs/2403.04307)** (Evaluation Datasets | Factuality); **[CMATH](https://arxiv.org/abs/2306.16636)** (Evaluation Datasets | Subject); **[FineMath](https://arxiv.org/abs/2403.07747)** (Evaluation Datasets | Subject); **[RealTime QA](https://arxiv.org/abs/2207.13332)** (Evaluation Datasets | Factuality); **[WYWEB](https://aclanthology.org/2023.findings-acl.204/)** (Evaluation Datasets | Subject); **[ChineseFactEval](https://gair-nlp.github.io/ChineseFactEval/)** (Evaluation Datasets | Factuality); **[Counting-Stars](https://arxiv.org/abs/2403.11802)** (Evaluation Datasets | Long Text).\n  - Add **[SlimPajama](https://www.cerebras.net/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama)** (Pre-training Corpora | General Pre-training Corpora | Multi-category); **[MassiveText](https://arxiv.org/abs/2112.11446)** (Pre-training Corpora | General Pre-training Corpora | Multi-category); **[MADLAD-400](https://arxiv.org/abs/2309.04662)** (Pre-training Corpora | General Pre-training Corpora | Webpages); **[Minerva](https://arxiv.org/abs/2206.14858)** (Pre-training Corpora | General Pre-training Corpora | Multi-category); **[CCAligned](https://aclanthology.org/2020.emnlp-main.480/)** (Pre-training Corpora | General Pre-training Corpora | Parallel Corpus); **[WikiMatrix](https://aclanthology.org/2021.eacl-main.115/)** (Pre-training Corpora | General Pre-training Corpora | Parallel Corpus); **[OpenWebMath](https://arxiv.org/abs/2310.06786)** (Pre-training Corpora | Domain-specific Pre-training Corpora | Math).\n  - Add **[WebQuestions](https://aclanthology.org/D13-1160.pdf)** (Traditional NLP Datasets | Question Answering | Knowledge QA).\n  - Add **[ALCE](https://aclanthology.org/2023.emnlp-main.398/)** (RAG Datasets).\n  - Add **[AlphaFin](https://arxiv.org/abs/2403.12582)** (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Other); **[COIG-CQIA](https://arxiv.org/abs/2403.18058)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | HG \u0026 CI).\n- （2024/06/15）\n  - Add **[CLUE](https://arxiv.org/abs/2404.04067)** (Evaluation Datasets | Medical); **[CHC-Bench](https://arxiv.org/abs/2404.04167)** (Evaluation Datasets | General); **[CIF-Bench](https://arxiv.org/abs/2402.13109)** (Evaluation Datasets | General); **[ACLUE](https://aclanthology.org/2023.alp-1.9/)** (Evaluation Datasets | Subject); **[LeSC](https://arxiv.org/abs/2405.05741)** (Evaluation Datasets | NLU); **[AlignBench](https://arxiv.org/abs/2311.18743)** (Evaluation Datasets | Multitask); **[SciKnowEval](https://arxiv.org/abs/2406.09098)** (Evaluation Datasets | Subject).\n  - Add **[MAP-CC](https://arxiv.org/abs/2404.04167)** (Pre-training Corpora | General Pre-training Corpora | Multi-category); **[FineWeb](https://huggingface.co/datasets/HuggingFaceFW/fineweb)** (Pre-training Corpora | General Pre-training Corpora | Webpages); **[CCI 2.0](https://huggingface.co/datasets/BAAI/CCI2-Data)** (Pre-training Corpora | General Pre-training Corpora | Webpages).\n  - Add **[WildChat](https://arxiv.org/abs/2405.01470)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | MC).\n  - Add **[OpenHermesPreferences](https://huggingface.co/datasets/argilla/OpenHermesPreferences)** (Preference Datasets | Sort); **[huozi_rlhf_data](https://github.com/HIT-SCIR/huozi/blob/main/data/huozi-rlhf/huozi_rlhf_data.csv)** (Preference Datasets | Vote); **[HelpSteer](https://arxiv.org/abs/2311.09528)** (Preference Datasets | Score); **[HelpSteer2](https://arxiv.org/abs/2406.08673)** (Preference Datasets | Score).\n  - Add **[MMT-Bench](https://arxiv.org/abs/2404.16006)** (MLLMs Datasets | Evaluation Datasets); **[mOSCAR](https://arxiv.org/abs/2406.08707)** (MLLMs Datasets | Pre-training Corpora); **[MM-NIAH](https://arxiv.org/abs/2406.07230)** (MLLMs Datasets | Evaluation Datasets).\n  - Add **[CRAG](https://arxiv.org/abs/2406.04744)** (RAG Datasets).\n- （2024/08/29）\n  - Add **[GameBench](https://arxiv.org/abs/2406.06613)** (Evaluation Datasets | Reasoning); **[HalluDial](https://arxiv.org/abs/2406.07070)** (Evaluation Datasets | Factuality); **[WildBench](https://arxiv.org/abs/2406.04770)** (Evaluation Datasets | General); **[DomainEval](https://arxiv.org/abs/2408.13204)** (Evaluation Datasets | Code); **[SysBench](https://arxiv.org/abs/2408.10943)** (Evaluation Datasets | General); **[KoBEST](https://aclanthology.org/2022.coling-1.325/)** (Evaluation Datasets | NLU); **[SarcasmBench](https://arxiv.org/abs/2408.11319)** (Evaluation Datasets | NLU); **[C\u003csup\u003e3\u003c/sup\u003e Bench](https://arxiv.org/abs/2405.17732)** (Evaluation Datasets | Subject); **[TableBench](https://www.arxiv.org/abs/2408.09174)** (Evaluation Datasets | Reasoning); **[ArabLegalEval](https://www.arxiv.org/abs/2408.07983)** (Evaluation Datasets | Law).\n  - Add **[MultiTrust](https://arxiv.org/abs/2406.07057)** (MLLMs Datasets | Evaluation Datasets); **[OBELISC](https://arxiv.org/abs/2306.16527)** (MLLMs Datasets | Pre-training Corpora); **[MultiMed](https://www.arxiv.org/abs/2408.12682)** (MLLMs Datasets | Evaluation Datasets).\n  - Add **[DCLM](https://arxiv.org/abs/2406.11794)** (Pre-training Corpora | General Pre-training Corpora | Webpages).\n  - Add **[Lithuanian-QA-v1](https://www.arxiv.org/abs/2408.12963)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | CI \u0026 MC); **[REInstruct](https://www.arxiv.org/abs/2408.10663)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | HG \u0026 CI \u0026 MC); **[KoLLM-Converations](https://huggingface.co/datasets/davidkim205/kollm-converations)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | CI).\n- （2024/09/04）\n  - Add **[LongWriter-6K](https://arxiv.org/abs/2408.07055)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | CI \u0026 MC).\n  - Add **[MedTrinity-25M](https://www.arxiv.org/abs/2408.02900)** (MLLMs Datasets | Evaluation Datasets); **[MMIU](https://www.arxiv.org/abs/2408.02718)** (MLLMs Datasets | Evaluation Datasets).\n  - Add **[Expository-Prose-V1](https://www.arxiv.org/abs/2408.03506)** (Pre-training Corpora | General Pre-training Corpora | Multi-category).\n  - Add **[DebateQA](https://arxiv.org/abs/2408.01419)** (Evaluation Datasets | Knowledge); **[NeedleBench](https://arxiv.org/pdf/2407.11963)** (Evaluation Datasets | Long Text); **[ArabicMMLU](https://aclanthology.org/2024.findings-acl.334/)** (Evaluation Datasets | Subject); **[PersianMMLU](https://arxiv.org/abs/2404.06644)** (Evaluation Datasets | Subject); **[TMMLU+](https://arxiv.org/abs/2403.01858)** (Evaluation Datasets | Subject).\n  - Add **[RAGEval](https://arxiv.org/abs/2408.01262)** (RAG Datasets); **[LFRQA](https://www.arxiv.org/abs/2407.13998)** (RAG Datasets); **[MultiHop-RAG](https://arxiv.org/abs/2401.15391)** (RAG Datasets).\n- （2024/12/14）\n  - Add **[Future-Idea-Generation](https://arxiv.org/abs/2409.06185)** (Evaluation Datasets | Other); **[WenMind](https://openreview.net/pdf?id=0G5OK5vmmg)** (Evaluation Datasets | Subject); **[Chinese SimpleQA](https://arxiv.org/abs/2411.07140)** (Evaluation Datasets | Knowledge).\n  - Add **[MME-RealWorld](https://arxiv.org/abs/2408.13257)** (MLLMs Datasets | Evaluation Datasets); **[II-Bench](https://arxiv.org/abs/2406.05862)** (MLLMs Datasets | Evaluation Datasets); **[CII-Bench](https://arxiv.org/abs/2410.13854)** (MLLMs Datasets | Evaluation Datasets); **[ALM-Bench](https://arxiv.org/abs/2411.16508)** (MLLMs Datasets | Evaluation Datasets).\n  - Add **[MaLA](https://arxiv.org/abs/2409.17892)** (Pre-training Corpora | General Pre-training Corpora | Multi-category); **[CCI3.0-HQ](https://arxiv.org/abs/2410.18505)** (Pre-training Corpora | General Pre-training Corpora | Multi-category); **[GlotCC](https://arxiv.org/abs/2410.23825)** (Pre-training Corpora | General Pre-training Corpora | Webpages); **[ChineseWebText 2.0](https://arxiv.org/abs/2411.19668)** (Pre-training Corpora | General Pre-training Corpora | Webpages); **[ChineseWebText 1.0](https://arxiv.org/abs/2311.01149)** (Pre-training Corpora | General Pre-training Corpora | Webpages); **[SkyPile](https://arxiv.org/abs/2310.19341)** (Pre-training Corpora | General Pre-training Corpora | Webpages).\n  - Add **[ViDoRe](https://arxiv.org/abs/2407.01449)** (RAG Datasets); **[M2KR](https://arxiv.org/abs/2402.08327)** (RAG Datasets); **[M-BEIR](https://arxiv.org/abs/2311.17136)** (RAG Datasets); **[MRAG-Bench](https://arxiv.org/abs/2410.08182)** (RAG Datasets).\n  - Add **[SlimOrca](https://huggingface.co/datasets/Open-Orca/SlimOrca)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | CI \u0026 MC); **[GPTeacher](https://huggingface.co/datasets/teknium/GPTeacher-General-Instruct)** (Instruction Fine-tuning Datasets | General Instruction Fine-tuning Datasets | MC); **[OrcaMathWordProblems](https://arxiv.org/abs/2402.14830)** (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Math); **[MathInstruct](https://arxiv.org/abs/2309.05653)** (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Math); **[MetaMathQA](https://arxiv.org/abs/2309.12284)** (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Math); **[Magicoder-OSS-Instruct-75K](https://arxiv.org/abs/2312.02120)** (Instruction Fine-tuning Datasets | Domain-specific Instruction Fine-tuning Datasets | Code).\n  - Add **[UltraInteract](https://arxiv.org/abs/2404.02078)** (Preference Datasets | Other).\n\n\u003c/details\u003e\n\n- （2025/03/25）Add **[GneissWeb](https://arxiv.org/abs/2502.14907)** (Pre-training Corpora | General Pre-training Corpora | Webpages).\n- We will release the dataset information in CSV format (2025).\n\n## Table of Contents\n- **[Pre-training Corpora](#pre-training-corpora)**\n  - [General Pre-training Corpora](#general-pre-training-corpora)\n    - [Webpages](#webpages)\n    - [Language Texts](#language-texts)\n    - [Books](#books)\n    - [Academic Materials](#academic-materials)\n    - [Code](#code01)\n    - [Parallel Corpus](#parallel-corpus)\n    - [Social Media](#social-media)\n    - [Encyclopedia](#encyclopedia)\n    - [Multi-category](#multi-category)\n  - [Domain-specific Pre-training Corpora](#domain-specific-pre-training-corpora)\n    - [Financial](#financial01)\n    - [Medical](#medical01)\n    - [Math](#math03)\n    - [Other](#other01)\n- **[Instruction Fine-tuning Datasets](#instruction-fine-tuning-datasets)**\n  - [General Instruction Fine-tuning Datasets](#general-instruction-fine-tuning-datasets)\n    - [Human Generated Datasets (HG)](#human-generated-datasets-hg)\n    - [Model Constructed Datasets (MC)](#model-constructed-datasets-mc)\n    - [Collection and Improvement of Existing Datasets (CI)](#collection-and-improvement-of-existing-datasets-ci)\n    - [HG \u0026 CI](#hg--ci)\n    - [HG \u0026 MC](#hg--mc)\n    - [CI \u0026 MC](#ci--mc)\n    - [HG \u0026 CI \u0026 MC](#hg--ci--mc)\n  - [Domain-specific Instruction Fine-tuning Datasets](#domain-specific-instruction-fine-tuning-datasets)\n    - [Medical](#medical02)\n    - [Code](#code02)\n    - [Legal](#legal)\n    - [Math](#math01)\n    - [Education](#education)\n    - [Other](#other02)\n- **[Preference Datasets](#preference-datasets)**\n  - [Preference Evaluation Methods](#preference-evaluation-methods)\n    - [Vote](#vote)\n    - [Sort](#sort)\n    - [Score](#score)\n    - [Other](#other03)\n- **[Evaluation Datasets](#evaluation-datasets)**\n  - [General](#general)\n  - [Exam](#exam)\n  - [Subject](#subject)\n  - [NLU](#nlu)\n  - [Reasoning](#reasoning)\n  - [Knowledge](#knowledge)\n  - [Long Text](#long-text)\n  - [Tool](#tool)\n  - [Agent](#agent)\n  - [Code](#code03)\n  - [OOD](#ood)\n  - [Law](#law)\n  - [Medical](#medical03)\n  - [Financial](#financial02)\n  - [Social Norms](#social-norms)\n  - [Factuality](#factuality)\n  - [Evaluation](#evaluation)\n  - [Multitask](#multitask01)\n  - [Multilingual](#multilingual)\n  - [Other](#other04)\n  - [Evaluation Platform](#evaluation-platform)\n- **[Traditional NLP Datasets](#traditional-nlp-datasets)**\n  - [Question Answering](#question-answering)\n    - [Reading Comprehension](#reading-comprehension)\n      - [Selection \u0026 Judgment](#selection--judgment)\n      - [Cloze Test](#cloze-test)\n      - [Answer Extraction](#answer-extraction)\n      - [Unrestricted QA](#unrestricted-qa)\n    - [Knowledge QA](#knowledge-qa)\n    - [Reasoning QA](#reasoning-qa)\n  - [Recognizing Textual Entailment](#recognizing-textual-entailment)\n  - [Math](#math02)\n  - [Coreference Resolution](#coreference-resolution)\n  - [Sentiment Analysis](#sentiment-analysis)\n  - [Semantic Matching](#semantic-matching)\n  - [Text Generation](#text-generation)\n  - [Text Translation](#text-translation)\n  - [Text Summarization](#text-summarization)\n  - [Text Classification](#text-classification)\n  - [Text Quality Evaluation](#text-quality-evaluation)\n  - [Text-to-Code](#text-to-code)\n  - [Named Entity Recognition](#named-entity-recognition)\n  - [Relation Extraction](#relation-extraction)\n  - [Multitask](#multitask02)\n- **[Multi-modal Large Language Models (MLLMs) Datasets](#multi-modal-large-language-models-mllms-datasets)**\n  - [Pre-training Corpora](#mllmpre)\n  - [Instruction Fine-tuning Datasets](#instruction02)\n  - [Evaluation Datasets](#evaluation02)\n- **[Retrieval Augmented Generation (RAG) Datasets](#retrieval-augmented-generation-rag-datasets)**\n\n## Pre-training Corpora\nThe pre-training corpora are large collections of text data used during the pre-training process of LLMs.\n\n### General Pre-training Corpora\nThe general pre-training corpora are large-scale datasets composed of extensive text from diverse domains and sources. Their primary characteristic is that the text content is not confined to a single domain, making them more suitable for training general foundational models. **Corpora are classified based on data categories.**\n\n**Dataset information format：**\n\n```\n- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website\n  - Publisher:\n  - Size:\n  - License:\n  - Source:\n```\n\n#### Webpages\n\n- **CC-Stories  2018-6 | Not | EN | CI | [Paper](https://arxiv.org/pdf/1806.02847.pdf) | [Github](https://github.com/tensorflow/models/tree/archive/research/lm_commonsense) | [Dataset](https://huggingface.co/datasets/spacemanidol/cc-stories)**\n  - Publisher: Google Brain\n  - Size: 31 GB\n  - License: -\n  - Source: Common Crawl\n\n- **CC100  2020-7 | All | Multi (100) | CI | [Paper](https://aclanthology.org/2020.acl-main.747.pdf) | [Dataset](https://huggingface.co/datasets/cc100)**\n  - Publisher: Facebook AI\n  - Size: 2.5 TB\n  - License: Common Crawl Terms of Use\n  - Source: Common Crawl\n\n- **CLUECorpus2020  2020-3 | All | ZH | CI | [Paper](https://arxiv.org/pdf/2003.01355.pdf) | [Dataset](https://github.com/CLUEbenchmark/CLUECorpus2020)**\n  - Publisher: CLUE Organization\n  - Size: 100 GB\n  - License: MIT\n  - Source: Common Crawl\n\n- **Common Crawl  2007-X | All | Multi | HG | [Website](https://commoncrawl.org/)**\n  - Publisher: Common Crawl\n  - Size: -\n  - License: Common Crawl Terms of Use\n  - Source: Web crawler data\n\n- **CulturaX  2023-9 | All | Multi (167) | CI | [Paper](https://arxiv.org/pdf/2309.09400.pdf) | [Dataset](https://huggingface.co/datasets/uonlp/CulturaX)**\n  - Publisher: University of Oregon et al.\n  - Size: 27 TB\n  - License: mC4 \u0026 OSCAR license\n  - Source: mC4, OSCAR\n\n- **C4  2019-10 | All | EN | CI | [Paper](https://arxiv.org/pdf/1910.10683.pdf) | [Dataset](https://huggingface.co/datasets/allenai/c4)**\n  - Publisher: Google Research\n  - Size: 12.68 TB\n  - License: ODC-BY \u0026 Common Crawl Terms of Use\n  - Source: Common Crawl\n\n- **mC4  2021-6 | All | Multi (108) | CI | [Paper](https://aclanthology.org/2021.naacl-main.41.pdf) | [Dataset](https://huggingface.co/datasets/mc4)**\n  - Publisher: Google Research\n  - Size: 251 GB\n  - License: ODC-BY \u0026 Common Crawl Terms of Use\n  - Source: Common Crawl\n\n- **OSCAR 22.01  2022-1 | All | Multi (151) | CI | [Paper](https://arxiv.org/pdf/2201.06642.pdf) | [Dataset](https://huggingface.co/datasets/oscar-corpus/OSCAR-2201) | [Website](https://oscar-project.org/)**\n  - Publisher: Inria\n  - Size: 8.41 TB\n  - License: CC0\n  - Source: Common Crawl\n\n- **RealNews  2019-5 | All | EN | CI | [Paper](https://arxiv.org/abs/1905.12616) | [Github](https://github.com/rowanz/grover)**\n  - Publisher: University of Washington et al.\n  - Size: 120 GB\n  - License: Apache-2.0\n  - Source: Common Crawl\n\n- **RedPajama-V2  2023-10 | All | Multi (5) | CI | [Github](https://github.com/togethercomputer/RedPajama-Data) | [Dataset](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-V2) | [Website](https://together.ai/blog/redpajama-data-v2)**\n  - Publisher: Together Computer\n  - Size: 30.4 T Tokens\n  - License: Common Crawl Terms of Use\n  - Source: Common Crawl, C4, etc.\n\n- **RefinedWeb  2023-6 | Partial | EN | CI | [Paper](https://arxiv.org/pdf/2306.01116.pdf) | [Dataset](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)**\n  - Publisher: The Falcon LLM team\n  - Size: 5000 GB\n  - License: ODC-BY-1.0\n  - Source: Common Crawl\n\n- **WuDaoCorpora-Text  2021-6 | Partial | ZH | HG | [Paper](https://www.sciencedirect.com/science/article/pii/S2666651021000152) | [Dataset](https://data.baai.ac.cn/details/WuDaoCorporaText)**\n  - Publisher: BAAI et al.\n  - Size: 200 GB\n  - License: MIT\n  - Source: Chinese webpages\n\n- **WanJuan-CC  2024-2 | Partial | EN | HG | [Paper](https://arxiv.org/abs/2402.19282) | [Dataset](https://opendatalab.org.cn/OpenDataLab/WanJuanCC)**\n  - Publisher: Shanghai Artifcial Intelligence Laboratory\n  - Size: 1 T Tokens\n  - License: CC-BY-4.0\n  - Source: Common Crawl\n\n- **MADLAD-400  2023-9 | All | Multi (419) | HG | [Paper](https://arxiv.org/abs/2309.04662) | [Github](https://github.com/google-research/google-research/tree/master/madlad_400) | [Dataset](https://huggingface.co/datasets/allenai/MADLAD-400)**\n  - Publisher: Google DeepMind et al.\n  - Size: 2.8 T Tokens\n  - License: ODL-BY\n  - Source: Common Crawl\n\n- **FineWeb 2024-4 | All | EN | CI | [Dataset](https://huggingface.co/datasets/HuggingFaceFW/fineweb)**\n  - Publisher: HuggingFaceFW\n  - Size: 15 TB Tokens\n  - License: ODC-BY-1.0\n  - Source: Common Crawl\n\n- **CCI 2.0 2024-4 | All | ZH | HG | [Dataset1](https://huggingface.co/datasets/BAAI/CCI2-Data) | [Dataset2](http://open.flopsera.com/flopsera-open/details/BAAI-CCI2)**\n  - Publisher: BAAI\n  - Size: 501 GB\n  - License: CCI Usage Aggrement\n  - Source: Chinese webpages\n\n- **DCLM 2024-6 | All | EN | CI | [Paper](https://arxiv.org/abs/2406.11794) | [Github](https://github.com/mlfoundations/dclm) | [Dataset](http://data.commoncrawl.org/contrib/datacomp/index.html) | [Website](https://www.datacomp.ai/dclm/)**\n  - Publisher: University of Washington et al.\n  - Size: 279.6 TB\n  - License: Common Crawl Terms of Use\n  - Source: Common Crawl\n\n- **GlotCC 2024-10 | All | Multi (1275) | CI | [Paper](https://arxiv.org/abs/2410.23825) | [Github](https://github.com/cisnlp/GlotCC) | [Dataset](https://huggingface.co/datasets/cis-lmu/GlotCC-V1)**\n  - Publisher: LMU Munich \u0026 Munich Center for Machine Learning et al.\n  - Size: 2 TB\n  - License: Common Crawl Terms of Use\n  - Source: Common Crawl\n\n- **ChineseWebText 2.0 2024-11 | All | ZH | CI | [Paper](https://arxiv.org/abs/2411.19668) | [Github](https://github.com/CASIA-LM/ChineseWebText-2.0) | [Dataset](https://huggingface.co/datasets/CASIA-LM/ChineseWebText2.0)**\n  - Publisher: Chinese Academy of Sciences et al.\n  - Size: 3.8 TB\n  - License: Apache-2.0\n  - Source: MAP-CC, WanJuan, WuDao, etc.\n\n- **ChineseWebText 1.0 2023-11 | All | ZH | CI | [Paper](https://arxiv.org/abs/2311.01149) | [Github](https://github.com/CASIA-LM/ChineseWebText) | [Dataset](https://huggingface.co/datasets/CASIA-LM/ChineseWebText)**\n  - Publisher: Chinese Academy of Sciences et al.\n  - Size: 1.42 TB\n  - License: -\n  - Source: Common Crawl\n\n- **SkyPile 2023-10 | Partial | ZH | HG | [Paper](https://arxiv.org/abs/2310.19341) | [Github](https://github.com/SkyworkAI/Skywork) | [Dataset](https://huggingface.co/datasets/Skywork/SkyPile-150B)**\n  - Publisher: Kunlun Inc\n  - Size: 150 B Tokens\n  - License: Skywork Community License\n  - Source: Publicly accessible Chinese Internet webpages\n\n- **GneissWeb 2025-2 | [Paper](https://arxiv.org/abs/2502.14907) | [Dataset](https://huggingface.co/datasets/ibm-granite/GneissWeb)**\n  - Publisher: IBM Research et al. \n\n#### Language Texts\n\n- **ANC  2003-X | All | EN | HG | [Website](https://anc.org/)**\n  - Publisher: The US National Science Foundation et al.\n  - Size: -\n  - License: -\n  - Source: American English texts\n\n- **BNC  1994-X | All | EN | HG | [Website](http://www.natcorp.ox.ac.uk/)**\n  - Publisher: Oxford University Press et al.\n  - Size: 4124 Texts\n  - License: -\n  - Source: British English texts\n\n- **News-crawl  2019-1 | All | Multi (59) | HG | [Dataset](https://data.statmt.org/news-crawl/)**\n  - Publisher: UKRI et al.\n  - Size: 110 GB\n  - License: CC0\n  - Source: Newspapers\n\n\n#### Books\n\n- **Anna’s Archive  2023-X | All | Multi | HG | [Website](https://annas-archive.org/datasets)**\n  - Publisher: Anna\n  - Size: 586.3 TB\n  - License: -\n  - Source: Sci-Hub, Library Genesis, Z-Library, etc.\n\n- **BookCorpusOpen  2021-5 | All | EN | CI | [Paper](https://arxiv.org/pdf/2105.05241.pdf) | [Github](https://github.com/jackbandy/bookcorpus-datasheet) | [Dataset](https://huggingface.co/datasets/bookcorpusopen)**\n  - Publisher: Jack Bandy et al.\n  - Size: 17,868 Books\n  - License: Smashwords Terms of Service\n  - Source: Toronto Book Corpus\n\n- **PG-19  2019-11 | All | EN | HG | [Paper](https://arxiv.org/pdf/1911.05507.pdf) | [Github](https://github.com/google-deepmind/pg19) | [Dataset](https://huggingface.co/datasets/pg19)**\n  - Publisher: DeepMind\n  - Size: 11.74 GB\n  - License: Apache-2.0\n  - Source: Project Gutenberg\n\n- **Project Gutenberg  1971-X | All | Multi | HG | [Website](https://www.gutenberg.org/)**\n  - Publisher: Ibiblio et al.\n  - Size: -\n  - License: The Project Gutenberg\n  - Source: Ebook data\n\n- **Smashwords  2008-X | All | Multi | HG | [Website](https://www.smashwords.com/)**\n  - Publisher: Draft2Digital et al.\n  - Size: -\n  - License: Smashwords Terms of Service\n  - Source: Ebook data\n\n- **Toronto Book Corpus  2015-6 | Not | EN | HG | [Paper](https://arxiv.org/pdf/1506.06724.pdf) | [Website](https://yknzhu.wixsite.com/mbweb)**\n  - Publisher: University of Toronto et al.\n  - Size: 11,038 Books\n  - License: MIT \u0026 Smashwords Terms of Service\n  - Source: Smashwords\n\n\n#### Academic Materials\n\n- **arXiv  1991-X | All | EN | HG | [Website](https://arxiv.org/)**\n  - Publisher: Paul Ginsparg et al.\n  - Size: -\n  - License: Terms of Use for arXiv APIs\n  - Source: arXiv preprint\n\n- **S2ORC  2020-6 | All | EN | CI | [Paper](https://aclanthology.org/2020.acl-main.447.pdf) | [Github](https://github.com/allenai/s2orc)**\n  - Publisher: AI2 et al.\n  - Size: 81.1 MB\n  - License: ODC-BY-1.0\n  - Source: MAG, arXiv, PubMed, etc.\n\n#### Code \u003ca id=\"code01\"\u003e\u003c/a\u003e\n\n- **BIGQUERY  2022-3 | Not | PL | CI | [Paper](https://arxiv.org/pdf/2203.13474.pdf) | [Github](https://github.com/salesforce/CodeGen)**\n  - Publisher: Salesforce Research\n  - Size: 341.1 GB\n  - License: Apache-2.0\n  - Source: BigQuery\n\n- **Github  2008-4 | All | PL | HG | [Website](https://github.com/)**\n  - Publisher: Microsoft\n  - Size: -\n  - License: -\n  - Source: Various code projects\n\n- **phi-1  2023-6 | Not | EN \u0026 PL | HG \u0026 MC | [Paper](https://arxiv.org/pdf/2306.11644.pdf) | [Dataset](https://huggingface.co/datasets/teleprint-me/phi-1)**\n  - Publisher: Microsoft Research\n  - Size: 7 B Tokens\n  - License: CC-BY-NC-SA-3.0\n  - Source: The Stack, StackOverflow, GPT-3.5 Generation\n\n- **The Stack  2022-11 | All | PL (358) | HG | [Paper](https://arxiv.org/pdf/2211.15533.pdf) | [Dataset](https://huggingface.co/datasets/bigcode/the-stack)**\n  - Publisher: ServiceNow Research et al.\n  - Size: 6 TB\n  - License: The Terms of the Original Licenses\n  - Source: Permissively-licensed source code files\n\n\n#### Parallel Corpus\n\n- **MTP  2023-9 | All | EN \u0026 ZH | HG \u0026 CI | [Dataset](https://data.baai.ac.cn/details/BAAI-MTP)**\n  - Publisher: BAAI\n  - Size: 1.3 TB\n  - License: BAAI Data Usage Protocol\n  - Source: Chinese-English parallel text pairs on the web\n\n- **MultiUN  2010-5 | All | Multi (7) | HG | [Paper](http://www.lrec-conf.org/proceedings/lrec2010/pdf/686_Paper.pdf) | [Website](https://www.euromatrixplus.net/multi-un/)**\n  - Publisher: German Research Center for Artificial Intelligence (DFKI) GmbH\n  - Size: 4353 MB\n  - License: -\n  - Source: United Nations documents\n\n- **ParaCrawl  2020-7 | All | Multi (42) | HG | [Paper](https://aclanthology.org/2020.acl-main.417.pdf) | [Website](https://paracrawl.eu/)**\n  - Publisher: Prompsit et al.\n  - Size: 59996 Files\n  - License: CC0\n  - Source: Web crawler data\n\n- **UNCorpus v1.0  2016-5 | All | Multi (6) | HG | [Paper](https://conferences.unite.un.org/UNCorpus/Content/Doc/un.pdf) | [Website](https://conferences.unite.un.org/UNCorpus)**\n  - Publisher: United Nations et al.\n  - Size: 799276 Files\n  - License: -\n  - Source: United Nations documents\n\n- **CCAligned  2020-11 | All | Multi (138) | HG | [Paper](https://aclanthology.org/2020.emnlp-main.480/) | [Dataset](https://huggingface.co/datasets/ccaligned_multilingual)**\n  - Publisher: Facebook AI et al.\n  - Size: 392 M URL pairs\n  - License: -\n  - Source: Common Crawl\n\n- **WikiMatrix  2021-4 | All | Multi (85) | HG | [Paper](https://aclanthology.org/2021.eacl-main.115/) | [Github](https://github.com/facebookresearch/LASER/tree/main/tasks/WikiMatrix) | [Dataset](https://github.com/facebookresearch/LASER/tree/main/tasks/WikiMatrix)**\n  - Publisher: Facebook AI et al.\n  - Size: 134 M parallel sentences\n  - License: CC-BY-SA\n  - Source: Wikipedia\n\n#### Social Media\n\n- **OpenWebText  2019-4 | All | EN | HG | [Website](https://skylion007.github.io/OpenWebTextCorpus/)**\n  - Publisher: Brown University\n  - Size: 38 GB\n  - License: CC0\n  - Source: Reddit\n\n- **Pushshift Reddit  2020-1 | All | EN | CI | [Paper](https://arxiv.org/pdf/2001.08435.pdf) | [Website](https://files.pushshift.io/reddit/)**\n  - Publisher: Pushshift.io et al.\n  - Size: 2 TB\n  - License: -\n  - Source: Reddit\n\n- **Reddit  2005-6 | All | EN | HG | [Website](www.reddit.com)**\n  - Publisher: Condé Nast Digital et al.\n  - Size: -\n  - License: -\n  - Source: Social media posts\n\n- **StackExchange  2008-9 | All | EN | HG | [Dataset](https://archive.org/download/stackexchange) | [Website](https://stackexchange.com/)**\n  - Publisher: Stack Exchange\n  - Size: -\n  - License: CC-BY-SA-4.0\n  - Source: Community question and answer data\n\n- **WebText  2019-2 | Partial | EN | HG | [Paper](https://insightcivic.s3.us-east-1.amazonaws.com/language-models.pdf) | [Github](https://github.com/openai/gpt-2) | [Dataset](https://github.com/openai/gpt-2-output-dataset)**\n  - Publisher: OpenAI\n  - Size: 40 GB\n  - License: MIT\n  - Source: Reddit\n\n- **Zhihu  2011-1 | All | ZH | HG | [Website](https://www.zhihu.com/)**\n  - Publisher: Beijing Zhizhe Tianxia Technology Co., Ltd\n  - Size: -\n  - License: Zhihu User Agreement\n  - Source: Social media posts\n\n\n#### Encyclopedia\n\n- **Baidu baike  2008-4 | All | ZH | HG | [Website](https://baike.baidu.com/)**\n  - Publisher: Baidu\n  - Size: -\n  - License: Baidu baike User Agreement\n  - Source: Encyclopedic content data\n\n- **TigerBot-wiki  2023-5 | All | ZH | HG | [Paper](https://arxiv.org/abs/2312.08688) | [Github](https://github.com/TigerResearch/TigerBot) | [Dataset](https://huggingface.co/datasets/TigerResearch/tigerbot-wiki-plugin)**\n  - Publisher: TigerBot\n  - Size: 205 MB\n  - License: Apache-2.0\n  - Source: Baidu baike\n\n- **Wikipedia  2001-1 | All | Multi | HG | [Dataset](https://huggingface.co/datasets/wikipedia) | [Website](https://dumps.wikimedia.org/)**\n  - Publisher: Wikimedia Foundation\n  - Size: -\n  - License: CC-BY-SA-3.0 \u0026 GFDL\n  - Source: Encyclopedic content data\n\n\n#### Multi-category\n\n- **ArabicText 2022  2022-12 | All | AR | HG \u0026 CI | [Dataset](https://data.baai.ac.cn/details/ArabicText-2022)**\n  - Publisher: BAAI et al.\n  - Size: 201.9 GB\n  - License: CC-BY-SA-4.0\n  - Source: ArabicWeb, OSCAR, CC100, etc.\n\n- **MNBVC  2023-1 | All | ZH | HG \u0026 CI | [Github](https://github.com/esbatmop/MNBVC) | [Dataset](https://huggingface.co/datasets/liwu/MNBVC)**\n  - Publisher: Liwu Community\n  - Size: 20811 GB\n  - License: MIT\n  - Source: Chinese books, webpages, theses, etc.\n\n- **RedPajama-V1  2023-4 | All | Multi | HG \u0026 CI | [Github](https://github.com/togethercomputer/RedPajama-Data) | [Dataset](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T)**\n  - Publisher: Together Computer\n  - Size: 1.2 T Tokens\n  - License: -\n  - Source: Common Crawl, Github, books, etc.\n\n- **ROOTS  2023-3 | Partial | Multi (59) | HG \u0026 CI | [Paper](https://arxiv.org/pdf/2303.03915.pdf) | [Dataset](https://huggingface.co/bigscience-data)**\n  - Publisher: Hugging Face et al.\n  - Size: 1.61 TB\n  - License: BLOOM Open-RAIL-M\n  - Source: OSCAR, Github, etc.\n\n- **The Pile  2021-1 | All | EN | HG \u0026 CI | [Paper](https://arxiv.org/pdf/2101.00027.pdf) | [Github](https://github.com/EleutherAI/the-pile) | [Dataset](https://pile.eleuther.ai/)**\n  - Publisher: EleutherAI\n  - Size: 825.18 GB\n  - License: MIT\n  - Source: Books, arXiv, Github, etc.\n\n- **TigerBot_pretrain_en  2023-5 | Partial | EN | CI | [Paper](https://arxiv.org/abs/2312.08688) | [Github](https://github.com/TigerResearch/TigerBot) | [Dataset](https://huggingface.co/datasets/TigerResearch/pretrain_en)**\n  - Publisher: TigerBot\n  - Size: 51 GB\n  - License: Apache-2.0\n  - Source: English books, webpages, en-wiki, etc\n\n- **TigerBot_pretrain_zh  2023-5 | Partial | ZH | HG | [Paper](https://arxiv.org/abs/2312.08688) | [Github](https://github.com/TigerResearch/TigerBot) | [Dataset](https://huggingface.co/datasets/TigerResearch/pretrain_zh)**\n  - Publisher: TigerBot\n  - Size: 55 GB\n  - License: Apache-2.0\n  - Source: Chinese books, webpages, zh-wiki, etc.\n\n- **WanJuanText-1.0  2023-8 | All | ZH | HG | [Paper](https://arxiv.org/pdf/2308.10755.pdf) | [Github](https://github.com/opendatalab/WanJuan1.0) | [Dataset](https://opendatalab.org.cn/WanJuan1.0)**\n  - Publisher: Shanghai AI Laboratory\n  - Size: 1094 GB\n  - License: CC-BY-4.0\n  - Source: Webpages, Encyclopedia, Books, etc\n\n- **Dolma  2024-1 | All | EN | HG \u0026 CI | [Paper](https://browse.arxiv.org/abs/2402.00159) | [Github](https://github.com/allenai/dolma) | [Dataset](https://huggingface.co/datasets/allenai/dolma)**\n  - Publisher: AI2 et al.\n  - Size: 11519 GB\n  - License: MR Agreement\n  - Source: Project Gutenberg, C4, Reddit, etc.\n\n- **SlimPajama  2023-6 | All | EN | HG \u0026 CI | [Github](https://github.com/Cerebras/modelzoo/tree/main/modelzoo/transformers/data_processing/slimpajama) | [Dataset](https://huggingface.co/datasets/cerebras/SlimPajama-627B) | [Website](https://www.cerebras.net/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama)**\n  - Publisher: Cerebras et al.\n  - Size: 627 B Tokens\n  - License: -\n  - Source: Common Crawl, C4, Github, etc.\n\n- **MassiveText  2021-12 | Not | Multi | HG \u0026 CI | [Paper](https://arxiv.org/abs/2112.11446)**\n  - Publisher: Google DeepMind\n  - Size: 10.5 TB\n  - License: -\n  - Source: MassiveWeb, C4, books, etc.\n\n- **Minerva  2022-6 | Not | EN | HG | [Paper](https://arxiv.org/abs/2206.14858)**\n  - Publisher: Google Research\n  - Size: 38.5 B Tokens\n  - License: -\n  - Source: arXiv, Webpages, etc.\n\n- **MAP-CC 2024-4 | All | ZH | HG | [Paper](https://arxiv.org/abs/2404.04167) | [Github](https://github.com/Chinese-Tiny-LLM/Chinese-Tiny-LLM) | [Dataset](https://huggingface.co/datasets/m-a-p/MAP-CC) | [Website](https://chinese-tiny-llm.github.io/)**\n  - Publisher: Multimodal Art Projection Research Community et al.\n  - Size: 840.48 B Token\n  - License: CC-BY-NC-ND-4.0\n  - Source: Chinese Common Crawl, Chinese Encyclopedias, Chinese Books etc.\n\n- **Expository-Prose-V1 2024-8 | All | EN | HG \u0026 CI | [Paper](https://www.arxiv.org/abs/2408.03506) | [Github](https://github.com/Pints-AI/1.5-Pints) | [Dataset](https://huggingface.co/datasets/pints-ai/Expository-Prose-V1)**\n  - Publisher: Pints.ai Labs\n  - Size: 56 B Tokens\n  - License: MIT\n  - Source: ArXiv, Wikipedia, Gutenberg, etc.\n\n- **MaLA 2024-9 | All | Multi (939) | HG \u0026 CI | [Paper](https://arxiv.org/abs/2409.17892) | [Dataset](https://huggingface.co/collections/MaLA-LM/mala-corpus-66e05127641a51de34d39529)**\n  - Publisher: University of Helsinki et al.\n  - Size: 74 B Tokens\n  - License: ODC-BY\n  - Source: CC100, CulturaX, MADLAD-400 etc.\n\n- **CCI3.0-HQ 2024-9 | All | ZH | HG | [Paper](https://arxiv.org/abs/2410.18505) | [Dataset](https://huggingface.co/datasets/BAAI/CCI3-HQ)**\n  - Publisher: BAAI\n  - Size: 500 GB\n  - License: BAAI Data Usage Protocol\n  - Source: News, Social media, Blogs, etc.\n\n### Domain-specific Pre-training Corpora\nDomain-specific pre-training corpora are LLM datasets customized for specific fields or topics. The type of corpus is typically employed in the incremental pre-training phase of LLMs. **Corpora are classified based on data domains.**\n\n**Dataset information format：**\n\n```\n- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website\n  - Publisher:\n  - Size:\n  - License:\n  - Source:\n  - Category:\n  - Domain:\n```\n\n#### Financial \u003ca id=\"financial01\"\u003e\u003c/a\u003e\n\n- **BBT-FinCorpus  2023-2 | Partial | ZH | HG | [Paper](https://arxiv.org/pdf/2302.09432.pdf) | [Github](https://github.com/ssymmetry/BBT-FinCUGE-Applications) | [Website](https://bbt.ssymmetry.com/index.html)**\n  - Publisher: Fudan University et al.\n  - Size: 256 GB\n  - License: -\n  - Source: Company announcements, research reports, financial \n  - Category: Multi\n  - Domain: Finance\n\n- **FinCorpus  2023-9 | All | ZH | HG | [Paper](https://arxiv.org/pdf/2305.12002.pdf) | [Github](https://github.com/Duxiaoman-DI/XuanYuan) | [Dataset](https://huggingface.co/datasets/Duxiaoman-DI/FinCorpus)**\n  - Publisher: Du Xiaoman\n  - Size: 60.36 GB\n  - License: Apache-2.0\n  - Source: Company announcements, financial news, financial exam questions\n  - Category: Multi\n  - Domain: Finance\n\n- **FinGLM  2023-7 | All | ZH | HG | [Github](https://github.com/MetaGLM/FinGLM)**\n  - Publisher: Knowledge Atlas et al.\n  - Size: 69 GB\n  - License: Apache-2.0\n  - Source: Annual Reports of Listed Companies\n  - Category: Language Texts\n  - Domain: Finance\n\n- **TigerBot-earning  2023-5 | All | ZH | HG | [Paper](https://arxiv.org/abs/2312.08688) | [Github](https://github.com/TigerResearch/TigerBot) | [Dataset](https://huggingface.co/datasets/TigerResearch/tigerbot-earning-plugin)**\n  - Publisher: TigerBot\n  - Size: 488 MB\n  - License: Apache-2.0\n  - Source: Financial reports\n  - Category: Language Texts\n  - Domain: Finance\n\n- **TigerBot-research  2023-5 | All | ZH | HG | [Paper](https://arxiv.org/abs/2312.08688) | [Github](https://github.com/TigerResearch/TigerBot) | [Dataset](https://huggingface.co/datasets/TigerResearch/tigerbot-research-plugin)**\n  - Publisher: TigerBot\n  - Size: 696 MB\n  - License: Apache-2.0\n  - Source: Research reports\n  - Category: Language Texts\n  - Domain: Finance\n\n#### Medical \u003ca id=\"medical01\"\u003e\u003c/a\u003e\n\n- **Medical-pt  2023-5 | All | ZH | CI | [Github](https://github.com/shibing624/MedicalGPT) | [Dataset](https://huggingface.co/datasets/shibing624/medical)**\n  - Publisher: Ming Xu\n  - Size: 632.78 MB\n  - License: Apache-2.0\n  - Source: Medical encyclopedia data, medical textbooks\n  - Category: Multi\n  - Domain: Medical\n\n- **PubMed Central  2000-2 | All | EN | HG | [Website](https://www.ncbi.nlm.nih.gov/pmc/)**\n  - Publisher: NCBI\n  - Size: -\n  - License: PMC Copyright Notice\n  - Source: Biomedical scientific literature\n  - Category: Academic Materials\n  - Domain: Medical\n\n#### Math \u003ca id=\"math03\"\u003e\u003c/a\u003e\n\n- **Proof-Pile-2  2023-10 | All | EN | HG \u0026 CI | [Paper](https://arxiv.org/pdf/2310.10631.pdf) | [Github](https://github.com/EleutherAI/math-lm) | [Dataset](https://huggingface.co/datasets/EleutherAI/proof-pile-2) | [Website](https://blog.eleuther.ai/llemma/)**\n  - Publisher: Princeton University et al.\n  - Size: 55 B Tokens\n  - License: -\n  - Source: ArXiv, OpenWebMath, AlgebraicStack\n  - Category: Multi\n  - Domain: Mathematics\n\n- **MathPile  2023-12 | All | EN | HG | [Paper](https://arxiv.org/abs/2312.17120) | [Github](https://github.com/GAIR-NLP/MathPile) | [Dataset](https://huggingface.co/datasets/GAIR/MathPile)**\n  - Publisher: Shanghai Jiao Tong University et al.\n  - Size: 9.5 B Tokens\n  - License: CC-BY-NC-SA-4.0\n  - Source: Textbooks, Wikipedia, ProofWiki, CommonCrawl, StackExchange, arXiv\n  - Category: Multi\n  - Domain: Mathematics\n\n- **OpenWebMath  2023-10 | All | EN | HG | [Paper](https://arxiv.org/abs/2310.06786) | [Github](https://github.com/keirp/OpenWebMath) | [Dataset](https://huggingface.co/datasets/open-web-math/open-web-math)**\n  - Publisher: University of Toronto et al.\n  - Size: 14.7 B Tokens\n  - License: ODC-BY-1.0\n  - Source: Common Crawl\n  - Category: Webpages\n  - Domain: Mathematics\n\n#### Other \u003ca id=\"other01\"\u003e\u003c/a\u003e\n\n- **TigerBot-law  2023-5 | All | ZH | HG | [Paper](https://arxiv.org/abs/2312.08688) | [Github](https://github.com/TigerResearch/TigerBot) | [Dataset](https://huggingface.co/datasets/TigerResearch/tigerbot-law-plugin)**\n  - Publisher: TigerBot\n  - Size: 29.9 MB\n  - License: Apache-2.0\n  - Source: Legal clauses\n  - Category: Language Texts\n  - Domain: Law\n\n- **TransGPT-pt  2023-7 | All | ZH | HG | [Github](https://github.com/DUOMO/TransGPT) | [Dataset](https://huggingface.co/datasets/DUOMO-Lab/TransGPT-pt)**\n  - Publisher: Beijing Jiaotong University\n  - Size: 35.8 MB\n  - License: Apache-2.0\n  - Source: Technology documents, engineering construction information, statistical data, etc.\n  - Category: Multi\n  - Domain: Transportation\n\n## Instruction Fine-tuning Datasets\nThe instruction fine-tuning datasets consists of a series of text pairs comprising “instruction inputs” and “answer outputs.” “Instruction inputs” represent requests made by humans to the model. There are various types of instructions, such as classification, summarization, paraphrasing, etc. “Answer outputs” are the responses generated by the model following the instruction and aligning with human expectations.\n\n### General Instruction Fine-tuning Datasets\nGeneral instruction fine-tuning datasets contain one or more instruction categories with no domain restrictions, primarily aiming to enhance the instruction-following capability of LLMs in general tasks. **Datasets are classified based on construction methods.**\n\n**Dataset information format：**\n\n```\n- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website\n  - Publisher:\n  - Size:\n  - License:\n  - Source:\n  - Instruction Category:\n```\n\n#### Human Generated Datasets (HG)\n\n- **databricks-dolly-15K  2023-4 | All | EN | HG | [Dataset](https://huggingface.co/datasets/databricks/databricks-dolly-15k) | [Website](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm)**\n  - Publisher: Databricks\n  - Size: 15011 instances\n  - License: CC-BY-SA-3.0\n  - Source: Manually generated based on different instruction categories\n  - Instruction Category: Multi\n\n- **InstructionWild_v2  2023-6 | All | EN \u0026 ZH | HG | [Github](https://github.com/XueFuzhao/InstructionWild)**\n  - Publisher: National University of Singapore\n  - Size: 110K instances\n  - License: -\n  - Source: Collected on the web\n  - Instruction Category: Multi\n\n- **LCCC  2020-8 | All | ZH | HG | [Paper](https://arxiv.org/pdf/2008.03946.pdf) | [Github](https://github.com/thu-coai/CDial-GPT)**\n  - Publisher: Tsinghua University et al.\n  - Size: 12M instances\n  - License: MIT\n  - Source: Crawl user interactions on social media\n  - Instruction Category: Multi\n\n- **OASST1  2023-4 | All | Multi (35) | HG | [Paper](https://arxiv.org/pdf/2309.11235.pdf) | [Github](https://github.com/imoneoi/openchat) | [Dataset](https://huggingface.co/openchat)**\n  - Publisher: OpenAssistant\n  - Size: 161443 instances\n  - License: Apache-2.0\n  - Source: Generated and annotated by humans\n  - Instruction Category: Multi\n\n- **OL-CC  2023-6 | All | ZH | HG | [Dataset](https://data.baai.ac.cn/details/OL-CC)**\n  - Publisher: BAAI\n  - Size: 11655 instances\n  - License: Apache-2.0\n  - Source: Generated and annotated by humans\n  - Instruction Category: Multi\n\n- **Zhihu-KOL  2023-3 | All | ZH | HG | [Github](https://github.com/wangrui6/Zhihu-KOL) | [Dataset](https://huggingface.co/datasets/wangrui6/Zhihu-KOL)**\n  - Publisher: wangrui6\n  - Size: 1006218 instances\n  - License: MIT\n  - Source: Crawl from Zhihu\n  - Instruction Category: Multi\n\n- **Aya Dataset  2024-2 | All | Multi (65) | HG | [Paper](https://arxiv.org/abs/2402.06619) | [Dataset](https://hf.co/datasets/CohereForAI/aya_dataset) | [Website](https://aya.for.ai)**\n  - Publisher:  Cohere For AI Community et al.\n  - Size: 204K instances\n  - License: Apache-2.0\n  - Source: Manually collected and annotated via the Aya Annotation Platform\n  - Instruction Category: Multi\n\n- **InstructIE  2023-5 | All | EN \u0026 ZH | HG | [Paper](https://arxiv.org/abs/2305.11527) | [Github](https://github.com/zjunlp/KnowLM) | [Dataset](https://huggingface.co/datasets/zjunlp/InstructIE)**\n  - Publisher: Zhejiang University et al.\n  - Size: 371700 instances\n  - License: MIT\n  - Source: Baidu baike, Wikipedia\n  - Instruction Category: Extraction\n\n#### Model Constructed Datasets (MC)\n\n- **Alpaca_data  2023-3 | All | EN | MC | [Github](https://github.com/tatsu-lab/stanford_alpaca#data-release)**\n  - Publisher: Stanford Alpaca\n  - Size: 52K instances\n  - License: Apache-2.0\n  - Source: Generated by Text-Davinci-003 with Aplaca_data prompts\n  - Instruction Category: Multi\n\n- **BELLE_Generated_Chat  2023-5 | All | ZH | MC | [Github](https://github.com/LianjiaTech/BELLE/tree/main/data/10M) | [Dataset](https://huggingface.co/datasets/BelleGroup/generated_chat_0.4M)**\n  - Publisher: BELLE\n  - Size: 396004 instances\n  - License: GPL-3.0\n  - Source: Generated by ChatGPT\n  - Instruction Category: Generation\n\n- **BELLE_Multiturn_Chat  2023-5 | All | ZH | MC | [Github](https://github.com/LianjiaTech/BELLE/tree/main/data/10M) | [Dataset](https://huggingface.co/datasets/BelleGroup/multiturn_chat_0.8M)**\n  - Publisher: BELLE\n  - Size: 831036 instances\n  - License: GPL-3.0\n  - Source: Generated by ChatGPT\n  - Instruction Category: Multi\n\n- **BELLE_train_0.5M_CN  2023-4 | All | ZH | MC | [Github](https://github.com/LianjiaTech/BELLE/tree/main/data/1.5M) | [Dataset](https://huggingface.co/datasets/BelleGroup/train_0.5M_CN)**\n  - Publisher: BELLE\n  - Size: 519255 instances\n  - License: GPL-3.0\n  - Source: Generated by Text-Davinci-003\n  - Instruction Category: Multi\n\n- **BELLE_train_1M_CN  2023-4 | All | ZH | MC | [Github](https://github.com/LianjiaTech/BELLE/tree/main/data/1.5M) | [Dataset](https://huggingface.co/datasets/BelleGroup/train_1M_CN)**\n  - Publisher: BELLE\n  - Size: 917424 instances\n  - License: GPL-3.0\n  - Source: Generated by Text-Davinci-003\n  - Instruction Category: Multi\n\n- **BELLE_train_2M_CN  2023-5 | All | ZH | MC | [Github](https://github.com/LianjiaTech/BELLE/tree/main/data/10M) | [Dataset](https://huggingface.co/datasets/BelleGroup/train_2M_CN)**\n  - Publisher: BELLE\n  - Size: 2M instances\n  - License: GPL-3.0\n  - Source: Generated by ChatGPT\n  - Instruction Category: Multi\n\n- **BELLE_train_3.5M_CN  2023-5 | All | ZH | MC | [Github](https://github.com/LianjiaTech/BELLE/tree/main/data/10M) | [Dataset](https://huggingface.co/datasets/BelleGroup/train_3.5M_CN)**\n  - Publisher: BELLE\n  - Size: 3606402 instances\n  - License: GPL-3.0\n  - Source: Generated by ChatGPT\n  - Instruction Category: Multi\n\n- **CAMEL  2023-3 | All | Multi \u0026 PL | MC | [Paper](https://arxiv.org/pdf/2303.17760.pdf) | [Github](https://github.com/camel-ai/camel) | [Dataset](https://huggingface.co/camel-ai) | [Website](https://www.camel-ai.org/)**\n  - Publisher: KAUST\n  - Size: 1659328 instances\n  - License: CC-BY-NC-4.0\n  - Source: Dialogue generated by two GPT-3.5-Turbo agents\n  - Instruction Category: Multi\n\n- **Chatgpt_corpus  2023-6 | All | ZH | MC | [Github](https://github.com/PlexPt/chatgpt-corpus/releases/tag/3)**\n  - Publisher: PlexPt\n  - Size: 3270K instances\n  - License: GPL-3.0\n  - Source: Generated by GPT-3.5-Turbo\n  - Instruction Category: Multi\n\n- **InstructionWild_v1  2023-3 | All | EN \u0026 ZH | MC | [Github](https://github.com/XueFuzhao/InstructionWild)**\n  - Publisher: National University of Singapore\n  - Size: 104K instances\n  - License: -\n  - Source: Generated by OpenAI API\n  - Instruction Category: Multi\n\n- **LMSYS-Chat-1M  2023-9 | All | Multi | MC | [Paper](https://arxiv.org/pdf/2309.11998.pdf) | [Dataset](https://huggingface.co/datasets/lmsys/lmsys-chat-1m)**\n  - Publisher: UC Berkeley et al.\n  - Size: 1M instances\n  - License: LMSYS-Chat-1M license\n  - Source: Generated by multiple LLMs\n  - Instruction Category: Multi\n\n- **MOSS_002_sft_data  2023-4 | All | EN \u0026 ZH | MC | [Github](https://github.com/OpenLMLab/MOSS) | [Dataset](https://huggingface.co/datasets/fnlp/moss-002-sft-data)**\n  - Publisher: Fudan University\n  - Size: 1161137 instances\n  - License: CC-BY-NC-4.0\n  - Source: Generated by Text-Davinci-003\n  - Instruction Category: Multi\n\n- **MOSS_003_sft_data  2023-4 | All | EN \u0026 ZH | MC | [Github](https://github.com/OpenLMLab/MOSS) | [Dataset](https://github.com/OpenLMLab/MOSS/tree/main/SFT_data)**\n  - Publisher: Fudan University\n  - Size: 1074551 instances\n  - License: CC-BY-NC-4.0\n  - Source: Conversation data from MOSS-002 and generated by GPT-3.5-Turbo\n  - Instruction Category: Multi\n\n- **MOSS_003_sft_plugin_data  2023-4 | Partial | EN \u0026 ZH | MC | [Github](https://github.com/OpenLMLab/MOSS) | [Dataset](https://github.com/OpenLMLab/MOSS/tree/main/SFT_data/conversations/conversation_with_plugins)**\n  - Publisher: Fudan University\n  - Size: 300K instances\n  - License: CC-BY-NC-4.0\n  - Source: Generated by plugins and LLMs\n  - Instruction Category: Multi\n\n- **OpenChat  2023-7 | All | EN | MC | [Paper](https://arxiv.org/pdf/2309.11235.pdf) | [Github](https://github.com/imoneoi/openchat) | [Dataset](https://huggingface.co/openchat)**\n  - Publisher: Tsinghua University et al.\n  - Size: 70K instances\n  - License: MIT\n  - Source: ShareGPT\n  - Instruction Category: Multi\n\n- **RedGPT-Dataset-V1-CN  2023-4 | Partial | ZH | MC | [Github](https://github.com/DA-southampton/RedGPT)**\n  - Publisher: DA-southampton\n  - Size: 50K instances\n  - License: Apache-2.0\n  - Source: Generated by LLMs\n  - Instruction Category: Multi\n\n- **Self-Instruct  2022-12 | All | EN | MC | [Paper](https://aclanthology.org/2023.acl-long.754.pdf) | [Github](https://github.com/yizhongw/self-instruct)**\n  - Publisher: University of Washington et al.\n  - Size: 52445 instances\n  - License: Apache-2.0\n  - Source: Generated by GPT-3\n  - Instruction Category: Multi\n\n- **ShareChat  2023-4 | All | Multi | MC | [Website](https://paratranz.cn/projects/6725)**\n  - Publisher: Sharechat\n  - Size: 90K instances\n  - License: CC0\n  - Source: ShareGPT\n  - Instruction Category: Multi\n\n- **ShareGPT-Chinese-English-90k  2023-7 | All | EN \u0026 ZH | MC | [Github](https://github.com/CrazyBoyM/llama2-Chinese-chat) | [Dataset](https://huggingface.co/datasets/shareAI/ShareGPT-Chinese-English-90k)**\n  - Publisher: shareAI\n  - Size: 90K instances\n  - License: Apache-2.0\n  - Source: ShareGPT\n  - Instruction Category: Multi\n\n- **ShareGPT90K  2023-4 | All | EN | MC | [Dataset](https://huggingface.co/datasets/RyokoAI/ShareGPT52K)**\n  - Publisher: RyokoAI\n  - Size: 90K instances\n  - License: CC0\n  - Source: ShareGPT\n  - Instruction Category: Multi\n\n- **UltraChat  2023-5 | All | EN | MC | [Paper](https://arxiv.org/pdf/2305.14233.pdf) | [Github](https://github.com/thunlp/UltraChat#UltraLM)**\n  - Publisher: Tsinghua University\n  - Size: 1468352 instances\n  - License: CC-BY-NC-4.0\n  - Source: Dialogue generated by two ChatGPT agents\n  - Instruction Category: Multi\n\n- **Unnatural Instructions  2022-12 | All | EN | MC | [Paper](https://aclanthology.org/2023.acl-long.806.pdf) | [Github](https://github.com/orhonovich/unnatural-instructions)**\n  - Publisher: Tel Aviv University et al.\n  - Size: 240670 instances\n  - License: MIT\n  - Source: Generated by LLMs\n  - Instruction Category: Multi\n\n- **WebGLM-QA  2023-6 | All | EN | MC | [Paper](https://arxiv.org/pdf/2306.07906.pdf) | [Github](https://github.com/THUDM/WebGLM) | [Dataset](https://huggingface.co/datasets/THUDM/webglm-qa)**\n  - Publisher: Tsinghua University et al.\n  - Size: 44979 instances\n  - License: Apache-2.0\n  - Source: Construct WebGLM-QA via LLM in-context bootstrapping\n  - Instruction Category: Open QA\n\n- **Wizard_evol_instruct_196K  2023-6 | All | EN | MC | [Paper](https://arxiv.org/pdf/2304.12244.pdf) | [Github](https://github.com/nlpxucan/WizardLM) | [Dataset](https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_V2_196k)**\n  - Publisher: Microsoft et al.\n  - Size: 196K instances\n  - License: -\n  - Source: Evolve instructions through the Evol-Instruct method\n  - Instruction Category: Multi\n\n- **Wizard_evol_instruct_70K  2023-5 | All | EN | MC | [Paper](https://arxiv.org/pdf/2304.12244.pdf) | [Github](https://github.com/nlpxucan/WizardLM) | [Dataset](https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_70k)**\n  - Publisher: Microsoft et al.\n  - Size: 70K instances\n  - License: -\n  - Source: Evolve instructions through the Evol-Instruct method\n  - Instruction Category: Multi\n\n- **WildChat 2024-5 | Partial | Multi (68) | MC | [Paper](https://arxiv.org/abs/2405.01470) | [Dataset](https://huggingface.co/datasets/allenai/WildChat)**\n  - Publisher: Cornell University et al.\n  - Size: 1039785 instances\n  - License: AI2 ImpACT license\n  - Source: Conversations between users and ChatGPT, GPT-4\n  - Instruction Category: Multi\n\n- **GPTeacher 2023-4 | All | EN | MC | [Github](https://github.com/teknium1/GPTeacher) | [Dataset](https://huggingface.co/datasets/teknium/GPTeacher-General-Instruct\n)**\n  - Publisher: teknium\n  - Size: 89260 instances\n  - License: MIT\n  - Source: Generated by GPT-4\n  - Instruction Category: Multi\n\n#### Collection and Improvement of Existing Datasets (CI)\n\n- **CrossFit  2021-4 | All | EN | CI | [Paper](https://arxiv.org/pdf/2104.08835.pdf) | [Github](https://github.com/INK-USC/CrossFit)**\n  - Publisher: University of Southern California\n  - Size: 269 datasets\n  - License: -\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **DialogStudio  2023-7 | All | EN | CI | [Paper](https://arxiv.org/pdf/2307.10172.pdf) | [Github](https://github.com/salesforce/DialogStudio) | [Dataset](https://huggingface.co/datasets/Salesforce/dialogstudio)**\n  - Publisher: Salesforce AI et al.\n  - Size: 87 datasets\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **Dynosaur  2023-5 | All | EN | CI | [Paper](https://arxiv.org/pdf/2305.14327.pdf) | [Github](https://github.com/WadeYin9712/Dynosaur) | [Dataset](https://huggingface.co/datasets?search=dynosaur) | [Website](https://dynosaur-it.github.io/)**\n  - Publisher: UCLA et al.\n  - Size: 801900 instances\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **Flan-mini  2023-7 | All | EN | CI | [Paper](https://arxiv.org/pdf/2307.02053.pdf) | [Github](https://github.com/declare-lab/flacuna) | [Dataset](https://huggingface.co/datasets/declare-lab/flan-mini)**\n  - Publisher: Singapore University of Technology and Design\n  - Size: 1.34M instances\n  - License: CC\n  - Source: Collection and improvement of various instruction fine-tuning datasets\n  - Instruction Category: Multi\n\n- **Flan 2021  2021-9 | All | Multi | CI | [Paper](https://arxiv.org/pdf/2109.01652.pdf) | [Github](https://github.com/google-research/flan)**\n  - Publisher: Google Research\n  - Size: 62 datasets\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **Flan 2022  2023-1 | Partial | Multi | CI | [Paper](https://arxiv.org/pdf/2301.13688.pdf) | [Github](https://github.com/google-research/FLAN/tree/main/flan/v2) | [Dataset](https://huggingface.co/datasets/SirNeural/flan_v2)**\n  - Publisher: Google Research\n  - Size: 1836 datasets\n  - License: Apache-2.0\n  - Source: Collection and improvement of various instruction fine-tuning datasets\n  - Instruction Category: Multi\n\n- **InstructDial  2022-5 | All | EN | CI | [Paper](https://arxiv.org/pdf/2205.12673.pdf) | [Github](https://github.com/prakharguptaz/Instructdial)**\n  - Publisher: Carnegie Mellon University\n  - Size: 59 datasets\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **NATURAL INSTRUCTIONS  2021-4 | All | EN | CI | [Paper](https://aclanthology.org/2022.acl-long.244.pdf) | [Github](https://github.com/allenai/natural-instructions) | [Dataset](https://instructions.apps.allenai.org/)**\n  - Publisher: Allen Institute for AI et al.\n  - Size: 61 datasets\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **OIG  2023-3 | All | EN | CI | [Dataset](https://huggingface.co/datasets/laion/OIG)**\n  - Publisher: LAION\n  - Size: 3878622 instances\n  - License: Apache-2.0\n  - Source: Collection and improvement of various datasets\n  - Instruction Category: Multi\n\n- **Open-Platypus  2023-8 | All | EN | CI | [Paper](https://arxiv.org/pdf/2308.07317.pdf) | [Github](https://github.com/arielnlee/Platypus) | [Dataset](https://huggingface.co/datasets/garage-bAInd/Open-Platypus) | [Website](https://platypus-llm.github.io/)**\n  - Publisher: Boston University\n  - Size: 24926 instances\n  - License: -\n  - Source: Collection and improvement of various datasets\n  - Instruction Category: Multi\n\n- **OPT-IML Bench  2022-12 | Not | Multi | CI | [Paper](https://arxiv.org/pdf/2212.12017.pdf) | [Github](https://github.com/facebookresearch/metaseq)**\n  - Publisher: Meta AI\n  - Size: 2000 datasets\n  - License: MIT\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **PromptSource  2022-2 | All | EN | CI | [Paper](https://aclanthology.org/2022.acl-demo.9.pdf) | [Github](https://github.com/bigscience-workshop/promptsource)**\n  - Publisher: Brown University et al.\n  - Size: 176 datasets\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **SUPER-NATURAL INSTRUCTIONS  2022-4 | All | Multi | CI | [Paper](https://arxiv.org/pdf/2204.07705.pdf) | [Github](https://github.com/allenai/natural-instructions)**\n  - Publisher: Univ. of Washington et al.\n  - Size: 1616 datasets\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **T0  2021-10 | All | EN | CI | [Paper](https://arxiv.org/pdf/2110.08207.pdf) | [Dataset1](https://huggingface.co/bigscience/T0) | [Dataset2](https://huggingface.co/datasets/bigscience/P3)**\n  - Publisher: Hugging Face et al.\n  - Size: 62 datasets\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **UnifiedSKG  2022-3 | All | EN | CI | [Paper](https://arxiv.org/pdf/2201.05966.pdf) | [Github](https://github.com/xlang-ai/UnifiedSKG)**\n  - Publisher: The University of Hong Kong et al.\n  - Size: 21 datasets\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **xP3  2022-11 | All | Multi (46) | CI | [Paper](https://aclanthology.org/2023.acl-long.891.pdf) | [Github](https://github.com/bigscience-workshop/xmtf)**\n  - Publisher: Hugging Face et al.\n  - Size: 82 datasets\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n\n- **IEPile  2024-2 | All | EN \u0026 ZH | CI | [Paper](https://arxiv.org/abs/2402.14710) | [Github](https://github.com/zjunlp/IEPile) | [Dataset](https://huggingface.co/datasets/zjunlp/iepile)**\n  - Publisher: Zhejiang University et al.\n  - Size: 33 datasets\n  - License: CC-BY-NC-SA-4.0\n  - Source: Collection and improvement of various IE datasets\n  - Instruction Category: Extraction\n\n- **KOLLM-Conversations 2024-3 | All | KO | CI | [Dataset](https://huggingface.co/datasets/davidkim205/kollm-converations)**\n  - Publisher: davidkim205\n  - Size: 1122566 instances\n  - License: Apache-2.0\n  - Source: Collection and improvement of Korean datasets\n  - Instruction Category: Multi\n\n#### HG \u0026 CI\n\n- **Firefly  2023-4 | All | ZH | HG \u0026 CI | [Github](https://github.com/yangjianxin1/Firefly) | [Dataset](https://huggingface.co/datasets/YeungNLP/firefly-train-1.1M)**\n  - Publisher: YeungNLP\n  - Size: 1649399 instances\n  - License: -\n  - Source: Collect Chinese NLP datasets and manually generate data related to Chinese culture\n  - Instruction Category: Multi\n\n- **LIMA-sft  2023-5 | All | EN | HG \u0026 CI | [Paper](https://arxiv.org/pdf/2305.11206.pdf) | [Dataset](https://huggingface.co/datasets/GAIR/lima)**\n  - Publisher: Meta AI et al.\n  - Size: 1330 instances\n  - License: CC-BY-NC-SA\n  - Source: Manually select from various types of data\n  - Instruction Category: Multi\n\n- **COIG-CQIA  2024-3 | All | ZH | HG \u0026 CI | [Paper](https://arxiv.org/abs/2403.18058) | [Dataset](https://huggingface.co/datasets/m-a-p/COIG-CQIA)**\n  - Publisher: Shenzhen Institute of Advanced Technology et al.\n  - Size: 48375 instances\n  - License: -\n  - Source: Q\u0026A communities, Wikipedia, examinations, existing NLP datasets\n  - Instruction Category: Multi\n\n#### HG \u0026 MC\n\n- **InstructGPT-sft  2022-3 | Not | EN | HG \u0026 MC | [Paper](https://arxiv.org/pdf/2203.02155.pdf)**\n  - Publisher: OpenAI\n  - Size: 14378 instances\n  - License: -\n  - Source: Platform Q\u0026A data and manual labeling\n  - Instruction Category: Multi\n\n\n#### CI \u0026 MC\n\n- **Alpaca_GPT4_data   2023-4 | All | EN | CI \u0026 MC | [Paper](https://arxiv.org/pdf/2304.03277.pdf) | [Github](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM#data-release)**\n  - Publisher: Microsoft Research\n  - Size: 52K instances\n  - License: Apache-2.0\n  - Source: Generated by GPT-4 with Aplaca_data prompts\n  - Instruction Category: Multi\n\n- **Alpaca_GPT4_data_zh  2023-4 | All | ZH | CI \u0026 MC | [Github](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM#data-release) | [Dataset](https://huggingface.co/datasets/shibing624/alpaca-zh)**\n  - Publisher: Microsoft Research\n  - Size: 52K instances\n  - License: Apache-2.0\n  - Source: Generated by GPT-4 with Alpaca_data prompts translated into Chinese by ChatGPT\n  - Instruction Category: Multi\n\n- **Bactrain-X  2023-5 | All | Multi (52) | CI \u0026 MC | [Paper](https://arxiv.org/pdf/2305.15011.pdf) | [Github](https://github.com/mbzuai-nlp/bactrian-x) | [Dataset](https://huggingface.co/datasets/MBZUAI/Bactrian-X)**\n  - Publisher: MBZUAI\n  - Size: 3484884 instances\n  - License: CC-BY-NC-4.0\n  - Source: Generated by GPT-3.5-Turbo with Aplaca_data and databricks-dolly-15K prompts translated into 51 languages by Google Translate API\n  - Instruction Category: Multi\n\n- **Baize  2023-3 | Partial | EN | CI \u0026 MC | [Paper](https://arxiv.org/pdf/2304.01196.pdf) | [Github](https://github.com/project-baize/baize-chatbot) | [Dataset](https://github.com/project-baize/baize-chatbot/tree/main/data)**\n  - Publisher: University of California et al.\n  - Size: 210311 instances\n  - License: GPL-3.0\n  - Source: Sample seeds from specific datasets to create multi-turn dialogues using ChatGPT\n  - Instruction Category: Multi\n\n- **GPT4All  2023-3 | All | EN | CI \u0026 MC | [Paper](https://gpt4all.io/reports/GPT4All_Technical_Report_3.pdf) | [Github](https://github.com/nomic-ai/gpt4all) | [Dataset](https://huggingface.co/datasets/QingyiSi/Alpaca-CoT/tree/main/GPT4all)**\n  - Publisher: nomic-ai\n  - Size: 739259 instances\n  - License: MIT\n  - Source: Generated by GPT-3.5-Turbo with other datasets’ prompts\n  - Instruction Category: Multi\n\n- **GuanacoDataset  2023-3 | All | Multi | CI \u0026 MC | [Dataset](https://huggingface.co/datasets/JosephusCheung/GuanacoDataset) | [Website](https://guanaco-model.github.io/)**\n  - Publisher: JosephusCheung\n  - Size: 534530 instances\n  - License: GPL-3.0\n  - Source: Expand upon the initial 52K dataset from the Alpaca model\n  - Instruction Category: Multi\n\n- **LaMini-LM  2023-4 | All | EN | CI \u0026 MC | [Paper](https://arxiv.org/pdf/2304.14402.pdf) | [Github](https://github.com/mbzuai-nlp/LaMini-LM) | [Dataset](https://huggingface.co/datasets/MBZUAI/LaMini-instruction)**\n  - Publisher: Monash University et al.\n  - Size: 2585615 instances\n  - License: CC-BY-NC-4.0\n  - Source: Generated by ChatGPT with synthetic and existing prompts\n  - Instruction Category: Multi\n\n- **LogiCoT  2023-5 | All | EN \u0026 ZH | CI \u0026 MC | [Paper](https://arxiv.org/pdf/2305.12147.pdf) | [Github](https://github.com/csitfun/logicot) | [Dataset](https://huggingface.co/datasets/csitfun/LogiCoT)**\n  - Publisher: Westlake University et al.\n  - Size: 604840 instances\n  - License: CC-BY-NC-ND-4.0\n  - Source: Expand the datasets using GPT-4\n  - Instruction Category: Reasoning\n\n- **LongForm  2023-4 | All | EN | CI \u0026 MC | [Paper](https://arxiv.org/pdf/2304.08460.pdf) | [Github](https://github.com/akoksal/LongForm) | [Dataset](https://huggingface.co/datasets/akoksal/LongForm)**\n  - Publisher: LMU Munich et al.\n  - Size: 27739 instances\n  - License: MIT\n  - Source: Select documents from existing corpora and generating prompts for the documents using LLMs\n  - Instruction Category: Multi\n\n- **Luotuo-QA-B  2023-5 | All | EN \u0026 ZH | CI \u0026 MC | [Github](https://github.com/LC1332/Luotuo-QA) | [Dataset](https://huggingface.co/datasets/Logic123456789/Luotuo-QA-B)**\n  - Publisher: Luotuo\n  - Size: 157320 instances\n  - License: Apache-2.0 \u0026 CC0\n  - Source: Use LLMs to generate Q\u0026A pairs on CSL, arXiv, and CNN-DM datasets\n  - Instruction Category: Multi\n\n- **OpenOrca  2023-6 | All | Multi | CI \u0026 MC | [Paper](https://arxiv.org/pdf/2306.02707.pdf) | [Dataset](https://huggingface.co/datasets/Open-Orca/OpenOrca)**\n  - Publisher: Microsoft Researc\n  - Size: 4233923 instances\n  - License: MIT\n  - Source: Expand upon the Flan 2022 dataset using GPT-3.5-Turbo and GPT-4\n  - Instruction Category: Multi\n\n- **Wizard_evol_instruct_zh  2023-5 | All | ZH | CI \u0026 MC | [Github](https://github.com/LC1332/Chinese-alpaca-lora) | [Dataset](https://huggingface.co/datasets/silk-road/Wizard-LM-Chinese-instruct-evol)**\n  - Publisher: Central China Normal University et al.\n  - Size: 70K instances\n  - License: CC-BY-4.0\n  - Source: Generated by GPT with Wizard_evol_instruct prompts translated into Chinese \n  - Instruction Category: Multi\n\n- **Lithuanian-QA-v1 2024-8 | All | LT | CI \u0026 MC | [Paper](https://www.arxiv.org/abs/2408.12963) | [Dataset](https://huggingface.co/datasets/neurotechnology/lithuanian-qa-v1)**\n  - Publisher: Neurotechnology\n  - Size: 13848 instances\n  - License: CC-BY-4.0\n  - Source: Use ChatGPT to generate Q\u0026A pairs on Wikipedia corpus\n  - Instruction Category: Multi\n\n- **LongWriter-6K 2024-8 | All | EN \u0026 ZH | CI \u0026 MC | [Paper](https://arxiv.org/abs/2408.07055) | [Github](https://github.com/THUDM/LongWriter) | [Dataset](https://huggingface.co/datasets/THUDM/LongWriter-6k)**\n  - Publisher: Tsinghua University et al.\n  - Size: 6000 instances\n  - License: Apache-2.0\n  - Source: Generated by GPT-4o with open-source datasets’ prompts\n  - Instruction Category: Multi\n\n- **SlimOrca 2023-10 | All | EN | CI \u0026 MC | [Dataset](https://huggingface.co/datasets/Open-Orca/SlimOrca)**\n  - Publisher: Open-Orca\n  - Size: 517,982 instances\n  - License: MIT\n  - Source: Generated by GPT-4 and from OpenOrca data\n  - Instruction Category: Multi\n\n#### HG \u0026 CI \u0026 MC\n\n- **COIG  2023-4 | All | ZH | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/pdf/2304.07987.pdf) | [Github](https://github.com/FlagOpen/FlagInstruct) | [Dataset](https://huggingface.co/datasets/BAAI/COIG)**\n  - Publisher: BAAI\n  - Size: 191191 instances\n  - License: Apache-2.0\n  - Source: Translated instructions, Leetcode, Chinese exams, etc.\n  - Instruction Category: Multi\n\n- **HC3  2023-1 | All | EN \u0026 ZH | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/pdf/2301.07597.pdf) | [Github](https://github.com/Hello-SimpleAI/chatgpt-comparison-detection) | [Dataset1](https://huggingface.co/datasets/Hello-SimpleAI/HC3) | [Dataset2](https://huggingface.co/datasets/Hello-SimpleAI/HC3-Chinese)**\n  - Publisher: SimpleAI\n  - Size: 37175 instances\n  - License: CC-BY-SA-4.0\n  - Source: Human-Q\u0026A pairs and ChatGPT-Q\u0026A pairs from Q\u0026A platforms, encyclopedias, etc.\n  - Instruction Category: Multi\n\n- **Phoenix-sft-data-v1  2023-5 | All | Multi | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/pdf/2304.10453.pdf) | [Github](https://github.com/FreedomIntelligence/LLMZoo) | [Dataset](https://huggingface.co/datasets/FreedomIntelligence/phoenix-sft-data-v1)**\n  - Publisher: The Chinese University of Hong Kong et al.\n  - Size: 464510 instances\n  - License: CC-BY-4.0\n  - Source: Collected multi-lingual instructions, post-translated multi-lingual instructions, self-generated user-centered multi-lingual instructions\n  - Instruction Category: Multi\n\n- **TigerBot_sft_en  2023-5 | Partial | EN | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/abs/2312.08688) | [Github](https://github.com/TigerResearch/TigerBot) | [Dataset](https://huggingface.co/datasets/TigerResearch/sft_en)**\n  - Publisher: TigerBot\n  - Size: 677117 instances\n  - License: Apache-2.0\n  - Source: Self-instruct, human-labeling, open-source data cleaning\n  - Instruction Category: Multi\n\n- **TigerBot_sft_zh  2023-5 | Partial | ZH | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/abs/2312.08688) | [Github](https://github.com/TigerResearch/TigerBot) | [Dataset](https://huggingface.co/datasets/TigerResearch/sft_zh)**\n  - Publisher: TigerBot\n  - Size: 530705 instances\n  - License: Apache-2.0\n  - Source: Self-instruct, human-labeling, open-source data cleaning\n  - Instruction Category: Multi\n\n- **Aya Collection  2024-2 | All | Multi (114) | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/abs/2402.06619) | [Dataset](https://hf.co/datasets/CohereForAI/aya_collection) | [Website](https://aya.for.ai)**\n  - Publisher:  Cohere For AI Community et al.\n  - Size: 513M instances\n  - License: Apache-2.0\n  - Source: Templated data, Translated data and Aya Dataset\n  - Instruction Category: Multi\n\n- **REInstruct 2024-8 | Not | EN | HG \u0026 CI \u0026 MC | [Paper](https://www.arxiv.org/abs/2408.10663) | [Github](https://github.com/cs32963/REInstruct)**\n  - Publisher: Chinese Information Processing Laboratory et al.\n  - Size: 35K instances\n  - License: -\n  - Source: Automatically constructing instruction data from the C4 corpus using a small amount of manually annotated seed instruction data\n  - Instruction Category: Multi\n\n### Domain-specific Instruction Fine-tuning Datasets\nThe domain-specific instruction fine-tuning datasets are constructed for a particular domain by formulating instructions that encapsulate knowledge and task types closely related to that domain. \n\n**Dataset information format：**\n\n```\n- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website\n  - Publisher:\n  - Size:\n  - License:\n  - Source:\n  - Instruction Category:\n  - Domain:\n```\n\n#### Medical \u003ca id=\"medical02\"\u003e\u003c/a\u003e\n\n- **ChatDoctor  2023-3 | All | EN | HG \u0026 MC | [Paper](https://arxiv.org/ftp/arxiv/papers/2303/2303.14070.pdf) | [Github](https://github.com/Kent0n-Li/ChatDoctor) | [Dataset](https://github.com/Kent0n-Li/ChatDoctor)**\n  - Publisher: University of Texas Southwestern Medical Center et al.\n  - Size: 115K instances\n  - License: Apache-2.0\n  - Source: Real conversations between doctors and patients \u0026 Generated by ChatGPT\n  - Instruction Category: Multi\n  - Domain: Medical\n\n- **ChatMed_Consult_Dataset  2023-5 | All | ZH | MC | [Github](https://github.com/michael-wzhu/ChatMed) | [Dataset](https://huggingface.co/datasets/michaelwzhu/ChatMed_Consult_Dataset)**\n  - Publisher: michael-wzhu\n  - Size: 549326 instances\n  - License: CC-BY-NC-4.0\n  - Source: Generated by GPT-3.5-Turbo\n  - Instruction Category: Multi\n  - Domain: Medical\n\n- **CMtMedQA  2023-8 | All | ZH | HG | [Paper](https://arxiv.org/pdf/2308.03549.pdf) | [Github](https://github.com/SupritYoung/Zhongjing) | [Dataset](https://huggingface.co/datasets/Suprit/CMtMedQA)**\n  - Publisher: Zhengzhou University\n  - Size: 68023 instances\n  - License: MIT\n  - Source: Real conversations between doctors and patients\n  - Instruction Category: Multi\n  - Domain: Medical\n\n- **DISC-Med-SFT  2023-8 | All | ZH | HG \u0026 CI | [Paper](https://arxiv.org/pdf/2308.14346.pdf) | [Github](https://github.com/FudanDISC/DISC-MedLLM) | [Dataset](https://huggingface.co/datasets/Flmc/DISC-Med-SFT) | [Website](https://med.fudan-disc.com)**\n  - Publisher: Fudan University et al.\n  - Size: 464898 instances\n  - License: Apache-2.0\n  - Source: Open source datasets \u0026 Manually selected data\n  - Instruction Category: Multi\n  - Domain: Medical\n\n- **HuatuoGPT-sft-data-v1  2023-5 | All | ZH | HG \u0026 MC | [Paper](https://arxiv.org/pdf/2305.15075.pdf) | [Github](https://github.com/FreedomIntelligence/HuatuoGPT) | [Dataset](https://huggingface.co/datasets/FreedomIntelligence/HuatuoGPT-sft-data-v1)**\n  - Publisher: The Chinese University of Hong Kong et al.\n  - Size: 226042 instances\n  - License: Apache-2.0\n  - Source: Real conversations between doctors and patients \u0026 Generated by ChatGPT\n  - Instruction Category: Multi\n  - Domain: Medical\n\n- **Huatuo-26M  2023-5 | Partial | ZH | CI | [Paper](https://arxiv.org/pdf/2305.01526.pdf) | [Github](https://github.com/FreedomIntelligence/Huatuo-26M)**\n  - Publisher: The Chinese University of Hong Kong et al.\n  - Size: 26504088 instances\n  - License: Apache-2.0\n  - Source: Collection and improvement of various datasets\n  - Instruction Category: Multi\n  - Domain: Medical\n\n- **MedDialog  2020-4 | All | EN \u0026 ZH | HG | [Paper](https://aclanthology.org/2020.emnlp-main.743.pdf) | [Github](https://github.com/UCSD-AI4H/Medical-Dialogue-System)**\n  - Publisher: UC San Diego\n  - Size: 3.66M instances\n  - License: -\n  - Source: Real conversations between doctors and patients\n  - Instruction Category: Multi\n  - Domain: Medical\n\n- **Medical Meadow  2023-4 | All | EN | HG \u0026 CI | [Paper](https://arxiv.org/pdf/2304.08247.pdf) | [Github](https://github.com/kbressem/medAlpaca) | [Dataset](https://huggingface.co/medalpaca)**\n  - Publisher: University Hospital Aachen et al.\n  - Size: 160076 instances\n  - License: GPL-3.0\n  - Source: Crawl data from the Internet \u0026 Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n  - Domain: Medical\n\n- **Medical-sft  2023-5 | All | EN \u0026 ZH | CI | [Github](https://github.com/shibing624/MedicalGPT) | [Dataset](https://huggingface.co/datasets/shibing624/medical)**\n  - Publisher: Ming Xu\n  - Size: 2.07M instances\n  - License: Apache-2.0\n  - Source: Collection and improvement of various NLP datasets\n  - Instruction Category: Multi\n  - Domain: Medical\n\n- **QiZhenGPT-sft-20k  2023-5 | Partial | ZH | CI | [Github](https://github.com/CMKRG/QiZhenGPT) | [Dataset](https://github.com/CMKRG/QiZhenGPT/blob/main/data/train/sft-20k.json)**\n  - Publisher: Zhejiang University\n  - Size: 20K instances\n  - License: GPL-3.0\n  - Source: Collection and improvement of various datasets\n  - Instruction Category: Multi\n  - Domain: Medical\n\n- **ShenNong_TCM_Dataset  2023-6 | All | ZH | MC | [Github](https://github.com/michael-wzhu/ShenNong-TCM-LLM) | [Dataset](https://huggingface.co/datasets/michaelwzhu/ShenNong_TCM_Dataset)**\n  - Publisher: michael-wzhu\n  - Size: 112565 instances\n  - License: Apache-2.0\n  - Source: Generated by ChatGPT\n  - Instruction Category: Multi\n  - Domain: Medical\n\n\n#### Code \u003ca id=\"code02\"\u003e\u003c/a\u003e\n\n- **Code_Alpaca_20K  2023-3 | All | EN \u0026 PL | MC | [Github](https://github.com/sahil280114/codealpaca) | [Dataset](https://github.com/sahil280114/codealpaca)**\n  - Publisher: Sahil Chaudhary\n  - Size: 20K instances\n  - License: Apache-2.0\n  - Source: Generated by Text-Davinci-003\n  - Instruction Category: Code\n  - Domain: Code\n\n- **CodeContest  2022-3 | All | EN \u0026 PL | CI | [Paper](https://arxiv.org/pdf/2203.07814.pdf) | [Github](https://github.com/google-deepmind/code_contests)**\n  - Publisher: DeepMind\n  - Size: 13610 instances\n  - License: Apache-2.0\n  - Source: Collection and improvement of various datasets\n  - Instruction Category: Code\n  - Domain: Code\n\n- **CommitPackFT  2023-8 | All | EN \u0026 PL (277) | HG | [Paper](https://arxiv.org/pdf/2308.07124.pdf) | [Github](https://github.com/bigcode-project/octopack) | [Dataset](https://huggingface.co/datasets/bigcode/commitpackft)**\n  - Publisher: Bigcode\n  - Size: 702062 instances\n  - License: MIT\n  - Source: GitHub Action dump\n  - Instruction Category: Code\n  - Domain: Code\n\n- **ToolAlpaca  2023-6 | All | EN \u0026 PL | HG \u0026 MC | [Paper](https://arxiv.org/pdf/2306.05301.pdf) | [Github](https://github.com/tangqiaoyu/ToolAlpaca)**\n  - Publisher: Chinese Information Processing Laboratory et al.\n  - Size: 3928 instances\n  - License: Apache-2.0\n  - Source: Manually filter APIs \u0026 Generated by ChatGPT\n  - Instruction Category: Code\n  - Domain: Code\n\n- **ToolBench  2023-7 | All | EN \u0026 PL | HG \u0026 MC | [Paper](https://arxiv.org/pdf/2307.16789v2.pdf) | [Github](https://github.com/OpenBMB/ToolBench)**\n  - Publisher: Tsinghua University et al.\n  - Size: 126486 instances\n  - License: Apache-2.0\n  - Source: Manually filter APIs \u0026 Generated by ChatGPT\n  - Instruction Category: Code\n  - Domain: Code\n\n- **Magicoder-OSS-Instruct-75K 2023-12 | All | EN | CI \u0026 MC | [Paper](https://arxiv.org/abs/2312.02120) | [Github](https://github.com/ise-uiuc/magicoder) | [Dataset](https://huggingface.co/datasets/ise-uiuc/Magicoder-OSS-Instruct-75K)** \n  - Publisher: University of Illinois at Urbana-Champaign et al.\n  - Size: 75197 instances\n  - License: MIT\n  - Source: Generated by prompting LLMs with seed code snippets from open-source repositories to create diverse coding problems and solutions\n  - Instruction Category: Code\n  - Domain: Code\n\n#### Legal\n\n- **DISC-Law-SFT  2023-9 | Partial | ZH | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/pdf/2309.11325.pdf) | [Github](https://github.com/FudanDISC/DISC-LawLLM) | [Website](https://law.fudan-disc.com)**\n  - Publisher: Fudan University et al.\n  - Size: 403K instances\n  - License: Apache-2.0\n  - Source: Open source datasets \u0026 Legal-related Text Content \u0026 Generated by GPT-3.5-Turbo\n  - Instruction Category: Multi\n  - Domain: Law\n\n- **HanFei 1.0  2023-5 | All | ZH | - | [Github](https://github.com/siat-nlp/HanFei) | [Dataset](https://github.com/siat-nlp/HanFei)**\n  - Publisher: Chinese Academy of Sciences et al.\n  - Size: 255K instances\n  - License: Apache-2.0\n  - Source: Filter legal-related data according to rules\n  - Instruction Category: Multi\n  - Domain: Law\n\n- **LawGPT_zh  2023-5 | Partial | ZH | CI \u0026 MC | [Github](https://github.com/LiuHC0428/LAW-GPT) | [Dataset](https://github.com/LiuHC0428/LAW-GPT#数据集构建)**\n  - Publisher: Shanghai Jiao Tong University\n  - Size: 200K instances\n  - License: -\n  - Source: Real conversations \u0026 Generated by ChatGPT\n  - Instruction Category: Multi\n  - Domain: Law\n\n- **Lawyer LLaMA_sft  2023-5 | Partial | ZH | CI \u0026 MC | [Paper](https://arxiv.org/pdf/2305.15062.pdf) | [Github](https://github.com/AndrewZhe/lawyer-llama) | [Dataset](https://github.com/AndrewZhe/lawyer-llama/tree/main/data)**\n  - Publisher: Peking Universit\n  - Size: 21476 instances\n  - License: Apache-2.0\n  - Source: Generated by ChatGPT with other datasets’ prompts\n  - Instruction Category: Multi\n  - Domain: Law\n\n#### Math \u003ca id=\"math01\"\u003e\u003c/a\u003e\n\n- **BELLE_School_Math  2023-5 | All | ZH | MC | [Github](https://github.com/LianjiaTech/BELLE/tree/main/data/10M) | [Dataset](https://huggingface.co/datasets/BelleGroup/school_math_0.25M)**\n  - Publisher: BELLE\n  - Size: 248481 instances\n  - License: GPL-3.0\n  - Source: Generated by ChatGPT\n  - Instruction Category: Math\n  - Domain: Math\n\n- **Goat  2023-5 | All | EN | HG | [Paper](https://arxiv.org/pdf/2305.14201.pdf) | [Github](https://github.com/liutiedong/goat) | [Dataset](https://huggingface.co/datasets/tiedong/goat)**\n  - Publisher: National University of Singapore\n  - Size: 1746300 instances\n  - License: Apache-2.0\n  - Source: Artificially synthesized data\n  - Instruction Category: Math\n  - Domain: Math\n\n- **MWP  2021-9 | All | EN \u0026 ZH | CI | [Paper](https://browse.arxiv.org/pdf/2109.00799.pdf) | [Github](https://github.com/LYH-YF/MWPToolkit) | [Dataset](https://huggingface.co/datasets/Macropodus/MWP-Instruct)**\n  - Publisher: Xihua University et al.\n  - Size: 251598 instances\n  - License: MIT\n  - Source: Collection and improvement of various datasets\n  - Instruction Category: Math\n  - Domain: Math\n\n- **OpenMathInstruct-1  2024-2 | All | EN | CI \u0026 MC | [Paper](https://arxiv.org/abs/2402.10176) | [Github](https://github.com/Kipok/NeMo-Skills) | [Dataset](https://huggingface.co/datasets/nvidia/OpenMathInstruct-1)**\n  - Publisher: NVIDIA\n  - Size: 1.8M instances\n  - License: NVIDIA License\n  - Source: GSM8K and MATH datasets (original questions); Generated using Mixtral-8×7B model\n  - Instruction Category: Math\n  - Domain: Math\n\n- **OrcaMathWordProblems 2024-2 | All | EN | CI \u0026 MC | [Paper](https://arxiv.org/abs/2402.14830) | [Dataset](https://huggingface.co/datasets/microsoft/orca-math-word-problems-200k)**\n  - Publisher: Microsoft Research\n  - Size: 200035 instances\n  - License: MIT\n  - Source: Generated by GPT-4 Turbo\n  - Instruction Category: Math\n  - Domain: Math\n\n- **MathInstruct 2023-9 | All | EN | CI \u0026 MC | [Paper](https://arxiv.org/abs/2309.05653) | [Github](https://github.com/TIGER-AI-Lab/MAmmoTH) | [Dataset](https://huggingface.co/datasets/TIGER-Lab/MathInstruct) | [Website](https://tiger-ai-lab.github.io/MAmmoTH/)**\n  - Publisher: University of Waterloo et al.\n  - Size: 262039 instances\n  - License: MIT\n  - Source: GSM8K, MATH, etc. and use GPT-4 to generate CoT and PoT reasoning processes\n  - Instruction Category: Math\n  - Domain: Math\n\n- **MetaMathQA 2023-9 | All | EN | CI \u0026 MC | [Paper](https://arxiv.org/abs/2309.12284) | [Github](https://github.com/meta-math/MetaMath) | [Dataset](https://huggingface.co/datasets/meta-math/MetaMathQA) | [Website](https://meta-math.github.io/)**\n  - Publisher: University of Cambridge et al.\n  - Size: 395000 instances\n  - License: MIT\n  - Source: Augmented from the GSM8K and MATH datasets using LLMs\n  - Instruction Category: Math\n  - Domain: Math\n\n#### Education\n\n- **Child_chat_data  2023-8 | All | ZH | HG \u0026 MC | [Github](https://github.com/HIT-SCIR-SC/QiaoBan)**\n  - Publisher: Harbin Institute of Technology et al.\n  - Size: 5000 instances\n  - License: -\n  - Source: Real conversations \u0026 Generated by GPT-3.5-Turbo\n  - Instruction Category: Multi\n  - Domain: Education\n\n- **Educhat-sft-002-data-osm  2023-7 | All | EN \u0026 ZH | CI | [Paper](https://arxiv.org/pdf/2308.02773.pdf) | [Github](https://github.com/icalk-nlp/EduChat) | [Dataset](https://huggingface.co/datasets/ecnu-icalk/educhat-sft-002-data-osm)**\n  - Publisher: East China Normal University et al.\n  - Size: 4279419 instances\n  - License: CC-BY-NC-4.0\n  - Source: Collection and improvement of various datasets\n  - Instruction Category: Multi\n  - Domain: Education\n\n- **TaoLi_data  2023-X | All | ZH | HG \u0026 CI | [Github](https://github.com/blcuicall/taoli) | [Dataset](https://github.com/blcuicall/taoli)**\n  - Publisher: Beijing Language and Culture University et al.\n  - Size: 88080 instances\n  - License: Apache-2.0\n  - Source: Collection and improvement of various datasets \u0026 Manually extract dictionary data\n  - Instruction Category: Multi\n  - Domain: Education\n\n\n#### Other \u003ca id=\"other02\"\u003e\u003c/a\u003e\n\n- **DISC-Fin-SFT  2023-10 | Partial | ZH | HG \u0026 CI \u0026 MC | [Paper](http://arxiv.org/abs/2310.15205) | [Github](https://github.com/FudanDISC/DISC-FinLLM) | [Website](https://fin.fudan-disc.com)**\n  - Publisher: Fudan University et al.\n  - Size: 246K instances\n  - License: Apache-2.0\n  - Source: Open source datasets \u0026 Manually collect financial data \u0026 ChatGPT assistance\n  - Instruction Category: Multi\n  - Domain: Financial\n\n- **AlphaFin  2024-3 | All | EN \u0026 ZH | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/abs/2403.12582) | [Github](https://github.com/AlphaFin-proj/AlphaFin) | [Dataset](https://huggingface.co/datasets/AlphaFin/AlphaFin-dataset-v1)**\n  - Publisher: South China University of Technology et al.\n  - Size: 167362 instances\n  - License: Apache-2.0\n  - Source: Traditional research datasets, real-time financial data, handwritten CoT data\n  - Instruction Category: Multi\n  - Domain: Financial\n\n- **GeoSignal  2023-6 | Partial | EN | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/pdf/2306.05064.pdf) | [Github](https://github.com/davendw49/k2) | [Dataset](https://huggingface.co/datasets/daven3/geosignal)**\n  - Publisher: Shanghai Jiao Tong University et al.\n  - Size: 22627272 instances\n  - License: Apache-2.0\n  - Source: Open source datasets \u0026 Geoscience-related Text Content \u0026 Generated by GPT-4\n  - Instruction Category: Multi\n  - Domain: Geoscience\n\n- **MeChat  2023-4 | All | ZH | CI \u0026 MC | [Paper](https://arxiv.org/pdf/2305.00450.pdf) | [Github](https://github.com/qiuhuachuan/smile) | [Dataset](https://github.com/qiuhuachuan/smile/tree/main/data)**\n  - Publisher: Zhejiang University et al.\n  - Size: 56K instances\n  - License: CC0-1.0\n  - Source: Based on PsyQA dataset with the proposed SMILE method\n  - Instruction Category: Multi\n  - Domain: Mental Health\n\n- **Mol-Instructions  2023-6 | All | EN | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/pdf/2306.08018.pdf) | [Github](https://github.com/zjunlp/Mol-Instructions) | [Dataset](https://huggingface.co/datasets/zjunlp/Mol-Instructions)**\n  - Publisher: Zhejiang University et al.\n  - Size: 2043586 instances\n  - License: CC-BY-4.0\n  - Source: Molecule-oriented, Protein-oriented, Biomolecular text instructions\n  - Instruction Category: Multi\n  - Domain: Biology\n\n- **Owl-Instruction  2023-9 | All | EN \u0026 ZH | HG \u0026 MC | [Paper](https://arxiv.org/pdf/2309.09298.pdf) | [Github](https://github.com/HC-Guo/Owl)**\n  - Publisher: Beihang University et al.\n  - Size: 17858 instances\n  - License: -\n  - Source: Generated by GPT-4 \u0026 Manual verification\n  - Instruction Category: Multi\n  - Domain: IT\n\n- **PROSOCIALDIALOG  2022-5 | All | EN | HG \u0026 MC | [Paper](https://arxiv.org/pdf/2205.12688.pdf) | [Dataset](https://huggingface.co/datasets/allenai/prosocial-dialog)**\n  - Publisher: Allenai\n  - Size: 165681 instances\n  - License: CC-BY-4.0\n  - Source: Generated by humans with GPT-3 created prompts\n  - Instruction Category: Social Norms\n  - Domain: Social Norms\n\n- **TransGPT-sft  2023-7 | All | ZH | HG | [Github](https://github.com/DUOMO/TransGPT) | [Dataset](https://huggingface.co/datasets/DUOMO-Lab/TransGPT-sft)**\n  - Publisher: Beijing Jiaotong University\n  - Size: 58057 instances\n  - License: Apache-2.0\n  - Source: Manually collect traffic-related data\n  - Instruction Category: Multi\n  - Domain: Transportation\n\n\n## Preference Datasets\nPreference datasets are collections of instructions that provide preference evaluations for multiple responses to the same instruction input.\n\n### Preference Evaluation Methods\nThe preference evaluation methods for preference datasets can be categorized into voting, sorting, scoring, and other methods. **Datasets are classified based on preference evaluation methods.**\n\n**Dataset information format：**\n\n```\n- Dataset name  Release Time | Public or Not | Language | Construction Method | Paper | Github | Dataset | Website\n  - Publisher:\n  - Size:\n  - License:\n  - Domain:\n  - Instruction Category: \n  - Preference Evaluation Method: \n  - Source: \n```\n\n#### Vote\n\n- **Chatbot_arena_conversations  2023-6 | All | Multi | HG \u0026 MC | [Paper](https://browse.arxiv.org/pdf/2306.05685.pdf) | [Dataset](https://huggingface.co/datasets/lmsys/chatbot_arena_conversations)**\n  - Publisher: UC Berkeley et al.\n  - Size: 33000 instances\n  - License: CC-BY-4.0 \u0026 CC-BY-NC-4.0\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: VO-H\n  - Source: Generated by twenty LLMs \u0026 Manual judgment\n\n- **hh-rlhf  2022-4 | All | EN | HG \u0026 MC | [Paper1](https://browse.arxiv.org/pdf/2204.05862.pdf) | [Paper2](https://browse.arxiv.org/pdf/2209.07858.pdf) | [Github](https://github.com/anthropics/hh-rlhf) | [Dataset](https://huggingface.co/datasets/Anthropic/hh-rlhf)**\n  - Publisher: Anthropic\n  - Size: 169352 instances\n  - License: MIT\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: VO-H\n  - Source: Generated by LLMs \u0026 Manual judgment\n\n- **MT-Bench_human_judgments  2023-6 | All | EN | HG \u0026 MC | [Paper](https://browse.arxiv.org/pdf/2306.05685.pdf) | [Github](https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge) | [Dataset](https://huggingface.co/datasets/lmsys/mt_bench_human_judgments) | [Website](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard)**\n  - Publisher: UC Berkeley et al. \n  - Size: 3.3K instances\n  - License: CC-BY-4.0\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: VO-H\n  - Source: Generated by LLMs \u0026 Manual judgment\n\n- **PKU-SafeRLHF  2023-7 | Partial | EN | HG \u0026 CI \u0026 MC | [Paper](https://browse.arxiv.org/pdf/2307.04657.pdf) | [Github](https://github.com/PKU-Alignment/safe-rlhf) | [Dataset](https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF)**\n  - Publisher: Peking University\n  - Size: 361903 instances\n  - License: CC-BY-NC-4.0\n  - Domain: Social Norms\n  - Instruction Category: Social Norms\n  - Preference Evaluation Method: VO-H\n  - Source: Generated by LLMs \u0026 Manual judgment\n\n- **SHP  2021-10 | All | EN | HG | [Paper](https://browse.arxiv.org/pdf/2110.08420.pdf) | [Github](https://github.com/kawine/dataset_difficulty) | [Dataset](https://huggingface.co/datasets/stanfordnlp/SHP)**\n  - Publisher: Stanford \n  - Size: 385563 instances\n  - License: -\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: VO-H\n  - Source: Reddit data \u0026 Manual judgment\n\n- **Zhihu_rlhf_3k  2023-4 | All | ZH | HG | [Dataset](https://huggingface.co/datasets/liyucheng/zhihu_rlhf_3k)**\n  - Publisher: Liyucheng\n  - Size: 3460 instances\n  - License: CC-BY-2.0\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: VO-H\n  - Source: Zhihu data \u0026 Manual judgment\n\n- **Summarize_from_Feedback  2020-9 | All | EN | HG \u0026 CI | [Paper](https://arxiv.org/pdf/2009.01325.pdf) | [Dataset](https://huggingface.co/datasets/openai/summarize_from_feedback)**\n  - Publisher: OpenAI\n  - Size: 193841 instances\n  - License: -\n  - Domain: News\n  - Instruction Category: Multi\n  - Preference Evaluation Method: VO-H \u0026 SC-H\n  - Source: Open source datasets \u0026 Manual judgment and scoring\n\n- **CValues  2023-7 | All | ZH | MC | [Paper](https://arxiv.org/pdf/2307.09705.pdf) | [Github](https://github.com/X-PLUG/CValues) | [Dataset](https://www.modelscope.cn/datasets/damo/CValues-Comparison/summary)**\n  - Publisher: Alibaba Group\n  - Size: 145K instances\n  - License: Apache-2.0\n  - Domain: Social Norms\n  - Instruction Category: Social Norms\n  - Preference Evaluation Method: VO-M\n  - Source: Generated by LLMs \u0026 Evaluation by the reward model\n\n- **huozi_rlhf_data 2024-2 | All | ZH | HG \u0026 MC | [Github](https://github.com/HIT-SCIR/huozi) | [Dataset](https://github.com/HIT-SCIR/huozi/blob/main/data/huozi-rlhf/huozi_rlhf_data.csv)**\n  - Publisher: Huozi-Team\n  - Size: 16918 instances\n  - License: Apache-2.0\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: VO-H\n  - Source: Generated by Huozi model \u0026 Manual judgment\n\n#### Sort\n\n- **OASST1_pairwise_rlhf_reward  2023-5 | All | Multi | CI | [Dataset](https://huggingface.co/datasets/tasksource/oasst1_pairwise_rlhf_reward)**\n  - Publisher: Tasksource\n  - Size: 18918 instances\n  - License: Apache-2.0\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: SO-H\n  - Source: OASST1 datasets \u0026 Manual sorting\n\n\n#### Score\n\n- **Stack-Exchange-Preferences  2021-12 | All | EN | HG | [Paper](https://arxiv.org/pdf/2112.00861.pdf) | [Dataset](https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences)**\n  - Publisher: Anthropic\n  - Size: 10807695 instances\n  - License: CC-BY-SA-4.0\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: SC-H\n  - Source: Stackexchange data \u0026 Manual scoring\n\n- **WebGPT  2021-12 | All | EN | HG \u0026 CI | [Paper](https://arxiv.org/pdf/2112.09332.pdf) | [Dataset](https://huggingface.co/datasets/openai/webgpt_comparisons)**\n  - Publisher: OpenAI\n  - Size: 19578 instances\n  - License: -\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: SC-H\n  - Source: Open source datasets \u0026 Manual scoring\n\n- **Alpaca_comparison_data  2023-3 | All | EN | MC | [Github](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM#data-release)**\n  - Publisher: Stanford Alpaca\n  - Size: 51K instances\n  - License: Apache-2.0\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: SC-M\n  - Source: Generated by three LLMs \u0026 GPT-4 scoring\n\n- **Stable_Alignment  2023-5 | All | EN | MC | [Paper](https://arxiv.org/pdf/2305.16960.pdf) | [Github](https://github.com/agi-templar/Stable-Alignment)**\n  - Publisher: Dartmouth College et al.\n  - Size: 169K instances\n  - License: Apache-2.0\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: SC-M\n  - Source: Generated by LLMs \u0026 Model scoring\n\n- **UltraFeedback  2023-10 | All | EN | CI \u0026 MC | [Paper](https://arxiv.org/abs/2310.01377) | [Github](https://github.com/OpenBMB/UltraFeedback) | [Dataset](https://huggingface.co/datasets/openbmb/UltraFeedback)**\n  - Publisher: Tsinghua University et al.\n  - Size: 63967 instances\n  - License: MIT\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: SC-M\n  - Source: Generated by seventeen LLMs \u0026 Model scoring\n\n- **OpenHermesPreferences 2024-3 | All | EN | CI \u0026 MC | [Dataset](https://huggingface.co/datasets/argilla/OpenHermesPreferences)**\n  - Publisher: Argilla et al.\n  - Size: 989490 instances\n  - License: -\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: SO-M\n  - Source: OpenHermes-2.5 dataset \u0026 Model sorting\n\n- **HelpSteer 2023-11 | All | EN | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/abs/2311.09528) | [Dataset](https://huggingface.co/datasets/nvidia/HelpSteer)**\n  - Publisher: NVIDIA\n  - Size: 37120 instances\n  - License: CC-BY-4.0\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: SC-H\n  - Source: Generated by LLMs \u0026 Manual judgment\n\n- **HelpSteer2 2024-6 | All | EN | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/abs/2406.08673) | [Github](https://github.com/NVIDIA/NeMo-Aligner) | [Dataset](https://huggingface.co/datasets/nvidia/HelpSteer2)**\n  - Publisher: NVIDIA\n  - Size: 21362 instances\n  - License: CC-BY-4.0\n  - Domain: General\n  - Instruction Category: Multi\n  - Preference Evaluation Method: SC-H\n  - Source: Generated by LLMs \u0026 Manual judgment\n\n#### Other \u003ca id=\"other03\"\u003e\u003c/a\u003e\n\n- **Medical-rlhf  2023-5 | All | ZH | CI \u0026 MC | [Github](https://github.com/shibing624/MedicalGPT) | [Dataset](https://huggingface.co/datasets/shibing624/medical)**\n  - Publisher: Ming Xu\n  - Size: 4K instances\n  - License: Apache-2.0\n  - Domain: Medical\n  - Instruction Category: Multi\n  - Preference Evaluation Method: Other\n  - Source: Response_chosen comes from the doctor's response \u0026 Response_rejected comes from the model's response\n\n- **PRM800K  2023-5 | All | EN | HG \u0026 CI \u0026 MC | [Paper](https://arxiv.org/pdf/23","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmmlzn%2FAwesome-LLMs-Datasets","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flmmlzn%2FAwesome-LLMs-Datasets","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmmlzn%2FAwesome-LLMs-Datasets/lists"}