{"id":13641846,"url":"https://github.com/hello-simpleai/chatgpt-comparison-detection","last_synced_at":"2025-04-20T11:32:37.763Z","repository":{"id":65264773,"uuid":"586171281","full_name":"Hello-SimpleAI/chatgpt-comparison-detection","owner":"Hello-SimpleAI","description":"Human ChatGPT Comparison Corpus (HC3), Detectors, and more! 🔥","archived":false,"fork":false,"pushed_at":"2023-12-01T16:03:51.000Z","size":55,"stargazers_count":1254,"open_issues_count":27,"forks_count":120,"subscribers_count":26,"default_branch":"main","last_synced_at":"2024-10-27T21:53:04.019Z","etag":null,"topics":["ai","chatbot","chatgpt","dataset","deep-learning","gpt-3","gpt2","gpt3","machine-learning","ml","nlp","openai","python","text-classification"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2301.07597","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Hello-SimpleAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-01-07T07:21:24.000Z","updated_at":"2024-10-12T14:03:59.000Z","dependencies_parsed_at":"2024-01-18T23:52:26.482Z","dependency_job_id":null,"html_url":"https://github.com/Hello-SimpleAI/chatgpt-comparison-detection","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hello-SimpleAI%2Fchatgpt-comparison-detection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hello-SimpleAI%2Fchatgpt-comparison-detection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hello-SimpleAI%2Fchatgpt-comparison-detection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Hello-SimpleAI%2Fchatgpt-comparison-detection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Hello-SimpleAI","download_url":"https://codeload.github.com/Hello-SimpleAI/chatgpt-comparison-detection/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223827759,"owners_count":17209838,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","chatbot","chatgpt","dataset","deep-learning","gpt-3","gpt2","gpt3","machine-learning","ml","nlp","openai","python","text-classification"],"created_at":"2024-08-02T01:01:24.965Z","updated_at":"2024-11-09T12:31:18.882Z","avatar_url":"https://github.com/Hello-SimpleAI.png","language":"Python","funding_links":[],"categories":["Chatbots"],"sub_categories":[],"readme":"# ChatGPT-Comparison-Detection Project 🔬\n\n![](https://img.shields.io/badge/Languages-%20English%2C%20Chinese-brightgreen) \n![](https://img.shields.io/badge/ChatGPT-Corpus%2C%20Detector-blue)\n\nOfficial repository of paper [\"How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection\"](https://arxiv.org/abs/2301.07597). Please star, watch, and fork our repo for the active updates!\n\nSee also→([📢 Feedback Space for Detectors](https://github.com/Hello-SimpleAI/chatgpt-comparison-detection/discussions/2) please feel free to leave your feedback here! 请留下您宝贵的意见！)\n\n\n\n\u003cimg width=\"600\" alt=\"image\" src=\"https://user-images.githubusercontent.com/37113676/212355768-5ef7a26a-7cc5-4c38-91dc-2ee249ec49d5.png\"\u003e\n\n---\n### Human ChatGPT Comparison Corpus (HC3) / 人类-ChatGPT 问答对比语料集\nYes, we propose the first **Human vs. ChatGPT** comparison corpus, named **HC3**.\n\n我们提出了第一个 **Human vs. ChatGPT** 对比语料, 叫做 **HC3**.\n\n\u003cimg width=\"520\" alt=\"image\" src=\"https://user-images.githubusercontent.com/37113676/213218672-e92b7036-a602-48c8-b70d-50ee1673bac8.png\"\u003e\n\nThe first version of the HC3 datasets are now available on 🤗 Huggingface Datasets:\n- [HC3-English](https://huggingface.co/datasets/Hello-SimpleAI/HC3)\n- [HC3-Chinese](https://huggingface.co/datasets/Hello-SimpleAI/HC3-Chinese)\n\n\n在中文社区，HC3 数据集也已在 ModelScope 上可用:\n- [HC3-English](https://www.modelscope.cn/datasets/simpleai/HC3)\n- [HC3-Chinese](https://www.modelscope.cn/datasets/simpleai/HC3-Chinese)\n\n\n\u003e Train/Test splits \u0026 filtered versions of the paper, ref to Google Drive links in [HC3/README.md](HC3/README.md).\n\n### Dataset Copyright\n\nIf the source datasets used in this corpus has a specific license which is stricter than CC-BY-SA, our products follow the same.\nIf not, they follow CC-BY-SA license.\n\n| English Split       | Source | Source License | Note |\n|----------|-------------|--------|-------------|\n| reddit_eli5 | [ELI5](https://github.com/facebookresearch/ELI5)   | BSD License    |     |\n| open_qa  | [WikiQA](https://www.microsoft.com/en-us/download/details.aspx?id=52419)  | [PWC Custom](https://paperswithcode.com/datasets/license)   |      |\n| wiki_csai   | Wikipedia | CC-BY-SA |   | [Wiki FAQ](https://en.wikipedia.org/wiki/Wikipedia:FAQ/Copyright) |\n| medicine    | [Medical Dialog](https://github.com/UCSD-AI4H/Medical-Dialogue-System) | Unknown|  [Asking](https://github.com/UCSD-AI4H/Medical-Dialogue-System/issues/10)|\n| finance     | [FiQA](https://paperswithcode.com/dataset/fiqa-1) | Unknown |  Asking by 📧  |\n\n| Chinese Split       | Source | Source License  | Note |\n|----------|-------------|-----------|-------------|\n| open_qa  | [WebTextQA \u0026 BaikeQA](https://github.com/brightmart/nlp_chinese_corpus) | MIT license |  |  |\n| baike     | Baidu Baike  | None   |    |   |\n| nlpcc_dbqa  | [NLPCC-DBQA](https://github.com/msra-nlc/ChineseDBQA) | Unknown |   [Asking](https://github.com/UCSD-AI4H/Medical-Dialogue-System/issues/10) |\n| medicine    | [Chinese Medical Dialogue](https://tianchi.aliyun.com/dataset/90163) |  CC-BY-NC 4.0 | \n| finance     | [FinanceZhidao](https://www.heywhale.com/mw/dataset/5e9588f8e7ec38002d0331b1/content) | CC-BY 4.0 |  |\n| psychology  | [On Baidu AI Studio](https://aistudio.baidu.com/aistudio/datasetdetail/38489) | CC0  | |\n|law          | [LegalQA](https://github.com/siatnlp/LegalQA) | Unknown | [Asking](https://github.com/siatnlp/LegalQA/issues/2) |\n\n\n---\n\n### ChatGPT detectors / 内容检测器\n![image](https://user-images.githubusercontent.com/37113676/211677236-d7c028f5-b9a5-4d88-baee-8b86dc942ff7.png)\n(Hosted on 🤗 Hugging Face Spaces)\n\n\nWe provide three kinds of detectors, all in Bilingual / 我们提供了三个版本的检测器，且都支持中英文:\n- [QA version / 问答版](https://huggingface.co/spaces/Hello-SimpleAI/chatgpt-detector-qa): detect whether an **answer** is generated by ChatGPT for certain **question**, using PLM-based classifiers / 判断某个**问题的回答**是否由ChatGPT生成，使用基于PTM的分类器来开发;\n- [Sinlge-text version / 独立文本版](https://huggingface.co/spaces/Hello-SimpleAI/chatgpt-detector-single): detect whether a piece of text is ChatGPT generated, using PLM-based classifiers / 判断**单条文本**是否由ChatGPT生成，使用基于PTM的分类器来开发;\n- [Linguistic version / 语言学版](https://huggingface.co/spaces/Hello-SimpleAI/chatgpt-detector-ling): detect whether a piece of text is ChatGPT generated, using linguistic features / 判断**单条文本**是否由ChatGPT生成，使用基于语言学特征的模型来开发;\n\n\n在 modelscope 中文社区平台，三个版本的检测器也都可用:\n- [QA version / 问答版](https://www.modelscope.cn/studios/simpleai/chatgpt-detector-qa)\n- [Sinlge-text version / 独立文本版](https://www.modelscope.cn/studios/simpleai/chatgpt-detector-single)\n- [Linguistic version / 语言学版](https://www.modelscope.cn/studios/simpleai/chatgpt-detector-ling)\n\n\nThe model weights are all available at 🤗 Hugging Face Models:\n\n| Model Checkpoints              | Comment      |\n|-----------------------|------------|\n|[chatgpt-detector-roberta](https://huggingface.co/Hello-SimpleAI/chatgpt-detector-roberta)|To detect a single piece of text|\n|[chatgpt-qa-detector-roberta](https://huggingface.co/Hello-SimpleAI/chatgpt-qa-detector-roberta)|To detect a question-answer pair|\n|[chatgpt-detector-roberta-chinese](https://huggingface.co/Hello-SimpleAI/chatgpt-detector-roberta-chinese)|检测单条文本，中文版|\n|[chatgpt-qa-detector-roberta-chinese](https://huggingface.co/Hello-SimpleAI/chatgpt-qa-detector-roberta-chinese)|检测一对QA文本，中文版|\n\nThe English models are based on [roberta-base](https://huggingface.co/roberta-base).\nThe Chinese models are based on [hfl/chinese-roberta-wwm-ext](https://huggingface.co/hfl/chinese-roberta-wwm-ext).\n\n\n---\n\n### Important Dates / 重要节点:\n\n| Events                | Dates      |\n|-----------------------|------------|\n| Project Launch / 项目启动        | 2022-12-09 ✅ |\n| Comparison Data Collection / 对比数据收集        | 2022-12-11 to Now 🏎️|\n| Release ChatGPT Detector (Demo) / 检测器 Demo 发布 | 2023-01-11 ✅|\n| Models Release / 模型开源 | 2023-01-18 ✅|\n| Comparison Corpus Release / 语料集开源 | 2023-01-18 ✅|\n| Research Paper / 研究论文发布 | 2023-01-19 ✅|\n|...|...|\n\n\n\n---\n\n### Citation\n\nCheckout this paper [arxiv: 2301.07597](https://arxiv.org/abs/2301.07597)\n\n```\n@article{guo-etal-2023-hc3,\n    title = \"How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection\",\n    author = \"Guo, Biyang  and\n      Zhang, Xin  and\n      Wang, Ziyuan  and\n      Jiang, Minqi  and\n      Nie, Jinran  and\n      Ding, Yuxuan  and\n      Yue, Jianwei  and\n      Wu, Yupeng\",\n    journal={arXiv preprint arxiv:2301.07597}\n    year = \"2023\",\n}\n```\n\n\n\n---\n### Our Story... / 背景故事\n\nOn December 9, 2022, which is 10 days after the launch of [ChatGPT](https://openai.com/blog/chatgpt/), we started this project, for two purposes: \n1. To create some **open-source models** for efficiently detecting ChatGPT-generated content; \n2. To collect a valuable **human-ChatGPT comparison Q\u0026A corpus**, to facilitate releated research.\n\n2022 年 12 月 9 日，也就是 [ChatGPT](https://openai.com/blog/chatgpt/) 推出的第 10 天，我们开始了这个项目，为了两个目的：\n1. 做出一些**开源**模型工具来高效检测 ChatGPT 生成的内容；\n2. 收集一批有价值的**人类和 ChatGPT 对比**的中英双语问答语料，来助力相关学术研究。\n\nWelcome to follow our project! We have released a preview of our ChatGPT detectors, and the **models, dataset will be open-sourced** in about a week. We look forward to receiving feedback from the community to help improve the models and make contributions to **open** academic research together:)\u003cbr\u003e\n欢迎关注我们项目，我们目前已经发布ChatGPT检测器预览版，并将于约**一周内发布开源模型、数据集**。期待得到广大群众的反馈，来帮助我们改进模型，为**开放**的学术研究一起做贡献！\n\n### About Us / 关于我们\n\nWe are a group of insignificant researchers (in the shadow of ChatGPT) hoping to do some significant work for the community. The team for this projects consists of PhD students and engineers from 6 universities/companies.\u003cbr\u003e\n我们是一群（在 ChatGPT 的阴影下）渺小的研究人员，但希望为社区做一些有意义的事。这个项目的团队由来自6所大学/公司的博士生和工程师组成。\n\n|   |   |   |   |\n|:-:|:-:|:-:|:-:|\n| [Biyang Guo](https://github.com/beyondguo) | [Minqi Jiang](https://github.com/Minqi824) | [Ziyuan Wang](https://github.com/SUFEHeisenberg) | [Xin Zhang](https://github.com/izhx) |\n|\u003cimg src=\"https://avatars.githubusercontent.com/u/37113676?s=64\u0026v=4\" alt=\"\" width=\"40\"/\u003e|\u003cimg src=\"https://avatars.githubusercontent.com/u/39890732?s=64\u0026v=4\" alt=\"\" width=\"40\"/\u003e|\u003cimg src=\"https://avatars.githubusercontent.com/u/44188955?s=64\u0026v=4\" alt=\"\" width=\"40\"/\u003e|\u003cimg src=\"https://avatars.githubusercontent.com/u/26690193?s=64\u0026v=4\" alt=\"\" width=\"40\"/\u003e|\n| [Jinran Nie](https://github.com/NJRBarry) | [Yuxuan Ding](https://github.com/yxding95) | [Jianwei Yue](https://github.com/TurquoiseA) | [Yupeng Wu](https://github.com/realRoc) |\n|\u003cimg src=\"https://avatars.githubusercontent.com/u/27188419?s=64\u0026v=4\" alt=\"\" width=\"40\"/\u003e|\u003cimg src=\"https://avatars.githubusercontent.com/u/16249556?s=70\u0026v=4\" alt=\"\" width=\"40\"/\u003e|  \u003cimg src=\"https://avatars.githubusercontent.com/u/23006855?s=64\u0026v=4\" alt=\"\" width=\"40\"/\u003e | \u003cimg src=\"https://avatars.githubusercontent.com/u/44936809?s=64\u0026v=4\" alt=\"\" width=\"40\"/\u003e  |\n\n\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhello-simpleai%2Fchatgpt-comparison-detection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhello-simpleai%2Fchatgpt-comparison-detection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhello-simpleai%2Fchatgpt-comparison-detection/lists"}