{"id":20779442,"url":"https://github.com/freedomintelligence/grammargpt","last_synced_at":"2025-08-22T06:32:19.291Z","repository":{"id":183596487,"uuid":"670427340","full_name":"FreedomIntelligence/GrammarGPT","owner":"FreedomIntelligence","description":"The code and data for GrammarGPT.","archived":false,"fork":false,"pushed_at":"2023-10-10T04:51:13.000Z","size":833,"stargazers_count":165,"open_issues_count":3,"forks_count":9,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-12-13T09:36:45.536Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FreedomIntelligence.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-07-25T03:16:34.000Z","updated_at":"2024-11-25T02:28:52.000Z","dependencies_parsed_at":"2023-07-25T04:36:40.620Z","dependency_job_id":null,"html_url":"https://github.com/FreedomIntelligence/GrammarGPT","commit_stats":null,"previous_names":["freedomintelligence/grammargpt"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FGrammarGPT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FGrammarGPT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FGrammarGPT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FreedomIntelligence%2FGrammarGPT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FreedomIntelligence","download_url":"https://codeload.github.com/FreedomIntelligence/GrammarGPT/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230568586,"owners_count":18246377,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T13:27:56.532Z","updated_at":"2024-12-20T10:08:08.818Z","avatar_url":"https://github.com/FreedomIntelligence.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning\n\n# ✨ Latest News\n- [07/31/2023]: Release the [model weights](https://huggingface.co/FreedomIntelligence/GrammarGPT).\n- [07/26/2023]: Release the [tech report](https://arxiv.org/abs/2307.13923).\n\n\n# ⚡ Introduction\n\nWelcome to the repository of GrammarGPT.\n\nThe implementation repository for [NLPCC 2023 Sharedtask1](http://tcci.ccf.org.cn/conference/2023/taskdata.php), which achieves third place. \n\nHere is a list of what has been released:\n* The 1k [data](./pseudo_data/instruction.json) for training, 65% of which are generated by ChatGPT, and the rest are manually annotated.\n* The code for training and inferencing.\n* You can find more details about the data and model on our [technical report](https://arxiv.org/abs/2307.13923).\n\n# 💭 Overview\nWe introduced GrammarGPT, an open-source LLM, to preliminary explore its potential for native Chinese grammatical error correction. The core recipe of GrammarGPT is to leverage the hybrid dataset of ChatGPT-generated and human-annotated. For grammatical errors with clues, we proposed a heuristic method to guide ChatGPT to generate ungrammatical sentences by providing those clues. For grammatical errors without clues, we collected ungrammatical sentences from publicly available websites and manually corrected them. In addition, we employed an error-invariant augmentation method to enhance the ability of the model to correct native Chinese grammatical errors.\n![](./assets/GrammarGPT.png)\n\n# 📚 Construction of Hybrid Dataset\n-\nThis table shows the six main types of grammatical errors made by native Chinese speakers, which can be divided into two types, e.g., with (w/) and without (w/o) clues. We can find that the incorrect sentences are fluent and in line with the habits of native Chinese. However, they do not conform to Chinese grammar, which is more difficult to correct. We utilized both ChatGPT-generated data and human-annotated data for dealing with grammatical errors with and without clues, respectively. \n\n![](./assets/ErrorTypes.jpg)\n\n## ChatGPT-generated Data\nGrammatical errors with clues are easy to detect and correct by recognizing the specific clues. For example, **more than** and **about** are used together leading to **redundant component**, **The cause** and **caused by** are used together leading to **structural confusion**, and **prompting** and **pace** are used together leading to **improper collocation**. Conversely, we can construct ungrammatical sentences by inserting these cues into grammatical sentences. We can instruct ChatGPT to generate the ungrammatical sentences that meet our requirements by providing these [Clues](https://wenku.baidu.com/view/1ce351635727a5e9846a610e?aggId=e4e228d30166f5335a8102d276a20029bc646366\u0026fr=catalogMain_text_ernie_recall_v1%3Awk_recommend_main_graph\u0026_wkts_=1686039387317\u0026bdQuery=%E5%86%97%E4%BD%99%E7%97%85%E5%8F%A5%E7%BB%83%E4%B9%A0)\n![](ChatGPT.png)collected from public websites. \n\n![](./assets/ChatGPT-Generated.png)\n## Human-annotated Data\nFor those ungrammatical errors,we collected data from public websites [1](https://wenku.baidu.com/view/1ce351635727a5e9846a610e?aggId=e4e228d30166f5335a8102d276a20029bc646366\u0026fr=catalogMain_text_ernie_recall_v1%3Awk_recommend_main_graph\u0026_wkts_=1686039387317\u0026bdQuery=%E5%86%97%E4%BD%99%E7%97%85%E5%8F%A5%E7%BB%83%E4%B9%A0) [2](https://baijiahao.baidu.com/s?id=1675817725570818147\u0026wfr=spider\u0026for=pc) [3](https://easylearn.baidu.com/edu-page/tiangong/exercisedetail?id=174470eef8c75fbfc77db25d\u0026from=search-duoti_pc-xiti_Detail_pc) [4](http://bj.xdf.cn/zhongkao/chuer/zhidao/134300.html) [5](http://bj.xdf.cn/zhongkao/chuer/zhidao/134299.html) [6](https://www.yueyeche.com.cn/zhjx/202207/19911.html) [7](https://mp.weixin.qq.com/s?__biz=MzI0NzE5NDI2MA==\u0026mid=2652204429\u0026idx=2\u0026sn=6db3a396e1f1da2a56185917e8459d71\u0026chksm=f2527a76c525f3600808e041222a6a78a49817314ad69603ab48129d31492a60b6920c8ac736\u0026scene=27) [8](https://mp.weixin.qq.com/s?__biz=MzUzMDQ2MTM4OQ==\u0026mid=2247557713\u0026idx=4\u0026sn=50caf0d739fd625a277e0d88fd97e1e8\u0026chksm=fa52c5f3cd254ce57609af3da2a21e6fd0c7cdbb45d6a41cb3168c0e7e57b23b825508433d6e\u0026scene=27) [9](https://wenku.baidu.com/view/5c9798cd961ea76e58fafab069dc5022aaea46f2.html?fr=aladdin664466\u0026ind=3\u0026_wkts_=1686039743632\u0026bdQuery=%E5%8F%A5%E5%BC%8F%E6%9D%82%E7%B3%85) [10](https://zhuanlan.zhihu.com/p/479275444) [11](https://www.zszzs.com/wendang/qitafanwen/54091.html) [12](https://mp.weixin.qq.com/s?__biz=MzU4NTc3MzkwMw==\u0026mid=2247500319\u0026idx=3\u0026sn=6ba362341e8f5543a8bb815e3a1657bd\u0026chksm=fd87e43fcaf06d29a7486e45fa98215710987154fe9fcd58df33a4abf676699be2d44c293646\u0026scene=27) [13](https://baijiahao.baidu.com/s?id=1742587369710610978\u0026wfr=spider\u0026for=pc) [14](https://mp.weixin.qq.com/s/DQnlXE_bKrSmTUVqTesqIg) [15](https://baijiahao.baidu.com/s?id=1617092703098480309\u0026wfr=spider\u0026for=pc) [16](https://www.renrendoc.com/paper/208183328.html) and manaually annotated them.\n\n## Error-invariant Augmentation\nNative Chinese grammatical errors are often subtle and infrequently found in the position of named entities. Therefore, we adopt a strategy of substituting the named entities in the parallel data with similar ones([Synonyms](https://github.com/chatopera/Synonyms)).\n![](./assets/Augmentation.png)\n\n# 🚀 Training\n```\npython finetuning.py\n```\n# 🧐 Inferencing\n```\npython generate.py\n```\n\n# 😀 Acknowledgement\n\nWe are aware that our works are inspired by the following works, including but not limited to\n\n- Bloom: https://huggingface.co/bigscience/bloom\n- Self-instruct: https://github.com/yizhongw/self-instruct\n- LLMZoo: https://github.com/FreedomIntelligence/LLMZoo\n  \nWithout these, nothing could happen in this repository.\n\n\n# Citation\n```\n@inproceedings{fan2023grammargpt,\n  title={GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning},\n  author={Fan, Yaxin and Jiang, Feng and Li, Peifeng and Li, Haizhou},\n  booktitle={CCF International Conference on Natural Language Processing and Chinese Computing},\n  pages={69--80},\n  year={2023},\n  organization={Springer}\n}\n```\nWe are from the School of Data Science, the Chinese University of Hong Kong, Shenzhen (CUHKSZ), and the Shenzhen Research Institute of Big Data (SRIBD).\n\nThe first author is a visiting student from Soochow University, and we welcome aspiring individuals to join our group and contribute to the new era of LLM.\n\n\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://api.star-history.com/svg?repos=FreedomIntelligence/GrammarGPT\u0026type=Date\u0026theme=dark\" /\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://api.star-history.com/svg?repos=FreedomIntelligence/GrammarGPT\u0026type=Date\" /\u003e\n  \u003cimg alt=\"Star History Chart\" src=\"https://api.star-history.com/svg?repos=FreedomIntelligence/GrammarGPT\u0026type=Date\" /\u003e\n\u003c/picture\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreedomintelligence%2Fgrammargpt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffreedomintelligence%2Fgrammargpt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreedomintelligence%2Fgrammargpt/lists"}