{"id":13754072,"url":"https://github.com/YuchuanTian/AIGC_text_detector","last_synced_at":"2025-05-09T22:30:48.062Z","repository":{"id":172014699,"uuid":"648472339","full_name":"YuchuanTian/AIGC_text_detector","owner":"YuchuanTian","description":"The official codes of our work on AIGC detection: \"Multiscale Positive-Unlabeled Detection of AI-Generated Texts\" (ICLR'24 Spotlight)","archived":false,"fork":false,"pushed_at":"2023-12-28T16:32:08.000Z","size":253,"stargazers_count":88,"open_issues_count":2,"forks_count":13,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-08-03T09:06:46.408Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2305.18149","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/YuchuanTian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-02T03:49:26.000Z","updated_at":"2024-08-02T08:22:22.000Z","dependencies_parsed_at":"2023-12-21T16:19:06.212Z","dependency_job_id":"89cc5a0f-cfff-4ded-9802-4634f052b63c","html_url":"https://github.com/YuchuanTian/AIGC_text_detector","commit_stats":null,"previous_names":["yuchuantian/aigc_text_detector"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuchuanTian%2FAIGC_text_detector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuchuanTian%2FAIGC_text_detector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuchuanTian%2FAIGC_text_detector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/YuchuanTian%2FAIGC_text_detector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/YuchuanTian","download_url":"https://codeload.github.com/YuchuanTian/AIGC_text_detector/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224884614,"owners_count":17386121,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T09:01:38.479Z","updated_at":"2025-05-09T22:30:48.055Z","avatar_url":"https://github.com/YuchuanTian.png","language":"Python","funding_links":[],"categories":["A01_文本生成_文本对话"],"sub_categories":["大语言对话模型及数据"],"readme":"# Multiscale Positive-Unlabeled Detection of AI-Generated Texts\r\n\r\n*Yuchuan Tian, Hanting Chen, Xutao Wang, Zheyuan Bai, Qinghua Zhang, Ruifeng Li, Chao Xu, Yunhe Wang*\r\n\r\nThe official codes of our paper \"Multiscale Positive-Unlabeled Detection of AI-Generated Texts\".\r\n\r\n[[arXiv]](https://arxiv.org/abs/2305.18149)\r\n\r\nPaper Link: https://arxiv.org/pdf/2305.18149.pdf\r\n\r\n## News\r\n\r\n**3/25/2025**: We release a demo (with both English and Chinese) on HuggingFace: [DEMO](https://huggingface.co/spaces/yuchuantian/AIGC_text_detector)\r\n\r\n**3/6/2025**: We will update a brand-new detector version to align with the latest LLMs. Please keep tuned!\r\n\r\n\r\n*BibTex* formatted citation:\r\n\r\n```\r\n@misc{tian2023multiscale,\r\n      title={Multiscale Positive-Unlabeled Detection of AI-Generated Texts}, \r\n      author={Yuchuan Tian and Hanting Chen and Xutao Wang and Zheyuan Bai and Qinghua Zhang and Ruifeng Li and Chao Xu and Yunhe Wang},\r\n      year={2023},\r\n      eprint={2305.18149},\r\n      archivePrefix={arXiv},\r\n      primaryClass={cs.CL}\r\n}\r\n```\r\n\r\n## Detector Models\r\n\r\nWe have open-sourced detector models in the paper as follows.\r\n\r\nLinks for Detectors: [Google Drive](https://drive.google.com/drive/folders/1Q_78qoaAuO8HOtt-SawimiXhli6y0Uii?usp=drive_link)  [Baidu Disk (PIN:1234)](https://pan.baidu.com/s/11hOpOxImAh1ZfDy9F5jC1Q)\r\n\r\nWe have also uploaded detector models to HuggingFace, where easy-to-use **DEMOs** and online **APIs** are provided.\r\n\r\n| Variants                                                     | HC3-Full-En        | HC3-Sent-En        |\r\n| ------------------------------------------------------------ | ------------------ | ------------------ |\r\n| seed0                                                        | 98.68              | 82.84              |\r\n| seed1 [HuggingFace: en v1](https://huggingface.co/yuchuantian/AIGC_detector_env1) | 98.56              | 87.06              |\r\n| seed2                                                        | 97.97              | 86.02              |\r\n| **Avg.**                                                     | **98.40$\\pm$0.31** | **85.31$\\pm$1.80** |\r\n\r\n## Stronger Detectors\r\n\r\nWe have also open-sourced detector models with strengthened training strategies. Specifically, we develop a strong Chinese detector ```AIGC_detector_zhv2```, which demonstrates similar performance to SOTA closed-source Chinese detectors on various texts, including news articles, poetry, essays, etc. The **DEMOs** and **APIs** are available on HuggingFace.\r\n\r\n| Detector                  | Google Drive                                                 | Baidu Disk                                                   | HuggingFace Link                                             |\r\n| ------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |\r\n| English, version 2 (env2) | [Google Drive](https://drive.google.com/drive/folders/11ROLjxopgx44YT9RS8RmchdxR6Yi-CJk?usp=drive_link) | [Baidu Disk (PIN:1234)](https://pan.baidu.com/s/11CQaWzsT7a-IgceOBpmb7g) | [en v2](https://huggingface.co/yuchuantian/AIGC_detector_env2) |\r\n| Chinese, version 2 (zhv2) | [Google Drive](https://drive.google.com/drive/folders/1-a7n-T9Z1_EIWbvip2eC0ssx5rih8pQI?usp=drive_link) | [Baidu Disk (PIN:1234)](https://pan.baidu.com/s/1VPGYtswC1GJXESWzne4RPA) | [zh v2](https://huggingface.co/yuchuantian/AIGC_detector_zhv2) |\r\n\r\n## About the Dataset\r\n\r\nHere we provide the official link for the HC3 dataset: [Dataset Link](https://github.com/Hello-SimpleAI/chatgpt-comparison-detection/blob/main/HC3/README.md). We also provide identical dataset copies on [Google Drive](https://drive.google.com/drive/folders/10GsKfCWe_BHLdGnfhSV-_k0-PH31_oWn?usp=drive_link) and [Baidu Disk (PIN:1234)](https://pan.baidu.com/s/1OUJbPYbC2ZUAt50MFDdHrQ) for your ease of use. We acknowledge the marvelous work by HC3 authors.\r\n\r\n#### Data Preprocessing\r\n\r\nIn Appendix B of [our paper](https://arxiv.org/pdf/2305.18149.pdf), we proposed the removal of redundant spaces in human texts of the HC3-English dataset. We have provided a helper function ```en_cleaning``` in ```corpus_cleaning_kit.py``` that takes a sentence string as input and returns a preprocessed sentence without redundant spaces.\r\n\r\nHere we provide a cleaned version of HC3-English. In this version, all answers are cleaned (*i. e.* redundant spaces are removed). However, please use the original version of HC3 for all experiments in our paper, as we have embedded the cleaning procedures in the training \u0026 validation scripts.\r\n\r\n**CLEANED** HC3-English Link:     [Google Drive](https://drive.google.com/drive/folders/11m9w7blNjUR2VE5N5AU7aOmj9YZhOyLy?usp=drive_link)    [Baidu Disk (PIN:1234)](https://pan.baidu.com/s/1kKSiyj1Nv2me6mODZd0Y4A)\r\n\r\n##  Preparation\r\n\r\n- Install requirement packages:\r\n\r\n```shell\r\npip install -r requirements.txt\r\n```\r\n\r\n- Download datasets to directory: ```./data``` \r\n\r\n- Download nltk package punct (This step could be done by ```nltk``` api: ```nltk.download('punkt')```)\r\n\r\n- Download pretrained models (This step could be automatically done by ```transformers```)\r\n\r\n\r\nBefore running, the directory should contain the following files:\r\n\r\n```\r\n├── data\r\n│   ├── unfilter_full\r\n│   │   ├── en_test.csv\r\n│   │   └── en_train.csv\r\n│   └── unfilter_sent\r\n│       ├── en_test.csv\r\n│       └── en_train.csv\r\n├── README.md\r\n├── corpus_cleaning_kit.py\r\n├── dataset.py\r\n├── multiscale_kit.py\r\n├── option.py\r\n├── pu_loss_mod.py\r\n├── prior_kit.py\r\n├── requirements.txt\r\n├── train.py\r\n└── utils.py\r\n```\r\n\r\n## Training\r\n\r\nThe script for training is ```train.py```.\r\n\r\n#### RoBERTa on HC3-English\r\n\r\nCommands for seed=0,1,2:\r\n\r\n```shell\r\nCUDA_VISIBLE_DEVICES=0 python train.py --batch-size 32 --max-sequence-length 512 --train-data-file unfilter_full/en_train.csv --val-data-file unfilter_full/en_test.csv --model-name roberta-base --local-data data --lamb 0.4 --prior 0.2 --pu_type dual_softmax_dyn_dtrun --len_thres 55 --aug_min_length 1 --max-epochs 1 --weight-decay 0 --mode original_single --aug_mode sentence_deletion-0.25 --clean 1 --val_file1 unfilter_sent/en_test.csv --quick_val 1 --learning-rate 5e-05 --seed 0\r\n\r\nCUDA_VISIBLE_DEVICES=0 python train.py --batch-size 32 --max-sequence-length 512 --train-data-file unfilter_full/en_train.csv --val-data-file unfilter_full/en_test.csv --model-name roberta-base --local-data data --lamb 0.4 --prior 0.2 --pu_type dual_softmax_dyn_dtrun --len_thres 55 --aug_min_length 1 --max-epochs 1 --weight-decay 0 --mode original_single --aug_mode sentence_deletion-0.25 --clean 1 --val_file1 unfilter_sent/en_test.csv --quick_val 1 --learning-rate 5e-05 --seed 1\r\n\r\nCUDA_VISIBLE_DEVICES=0 python train.py --batch-size 32 --max-sequence-length 512 --train-data-file unfilter_full/en_train.csv --val-data-file unfilter_full/en_test.csv --model-name roberta-base --local-data data --lamb 0.4 --prior 0.2 --pu_type dual_softmax_dyn_dtrun --len_thres 55 --aug_min_length 1 --max-epochs 1 --weight-decay 0 --mode original_single --aug_mode sentence_deletion-0.25 --clean 1 --val_file1 unfilter_sent/en_test.csv --quick_val 1 --learning-rate 5e-05 --seed 2\r\n\r\n```\r\n\r\n#### BERT on HC3-English\r\n\r\nCommands for seed=0,1,2:\r\n\r\n```shell\r\nCUDA_VISIBLE_DEVICES=0 python train.py --batch-size 32 --max-sequence-length 512 --train-data-file unfilter_full/en_train.csv --val-data-file unfilter_full/en_test.csv --model-name bert-base-cased --local-data data --lamb 0.5 --prior 0.3 --pu_type dual_softmax_dyn_dtrun --len_thres 60 --aug_min_length 1 --max-epochs 1 --weight-decay 0 --mode original_single --aug_mode sentence_deletion-0.25 --clean 1 --val_file1 unfilter_sent/en_test.csv --quick_val 1 --learning-rate 5e-05 --seed 0\r\n\r\n\r\nCUDA_VISIBLE_DEVICES=0 python train.py --batch-size 32 --max-sequence-length 512 --train-data-file unfilter_full/en_train.csv --val-data-file unfilter_full/en_test.csv --model-name bert-base-cased --local-data data --lamb 0.5 --prior 0.3 --pu_type dual_softmax_dyn_dtrun --len_thres 60 --aug_min_length 1 --max-epochs 1 --weight-decay 0 --mode original_single --aug_mode sentence_deletion-0.25 --clean 1 --val_file1 unfilter_sent/en_test.csv --quick_val 1 --learning-rate 5e-05 --seed 1\r\n\r\n\r\nCUDA_VISIBLE_DEVICES=0 python train.py --batch-size 32 --max-sequence-length 512 --train-data-file unfilter_full/en_train.csv --val-data-file unfilter_full/en_test.csv --model-name bert-base-cased --local-data data --lamb 0.5 --prior 0.3 --pu_type dual_softmax_dyn_dtrun --len_thres 60 --aug_min_length 1 --max-epochs 1 --weight-decay 0 --mode original_single --aug_mode sentence_deletion-0.25 --clean 1 --val_file1 unfilter_sent/en_test.csv --quick_val 1 --learning-rate 5e-05 --seed 2\r\n\r\n```\r\n\r\n## Acknowledgement\r\n\r\nOur code refers to the following GitHub repo:\r\n\r\nhttps://github.com/openai/gpt-2-output-dataset\r\n\r\nWe sincerely thank their authors for open-sourcing.\r\n\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYuchuanTian%2FAIGC_text_detector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FYuchuanTian%2FAIGC_text_detector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYuchuanTian%2FAIGC_text_detector/lists"}