{"id":27344437,"url":"https://github.com/jxtse/GEC-Metrics-DSGram","last_synced_at":"2025-04-12T17:06:21.139Z","repository":{"id":243058012,"uuid":"806984994","full_name":"jxtse/GEC-Metrics-DSGram","owner":"jxtse","description":"Metrics for Grammatical Error Correction models that is closer to human feedback, proposed a novel dynamic weighting evaluation method 一种新颖的语法纠错模型评价无参考指标，采用大语言模型生成动态权重的评价方法","archived":false,"fork":false,"pushed_at":"2024-12-22T06:48:36.000Z","size":2323,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-12-22T07:28:18.493Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jxtse.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-28T09:17:10.000Z","updated_at":"2024-12-22T06:48:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"4e0d4766-94ed-4ad1-bba2-d961858ca4d0","html_url":"https://github.com/jxtse/GEC-Metrics-DSGram","commit_stats":null,"previous_names":["jxtse/gec_metrics_llm","jxtse/gec-metrics-dsgram"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxtse%2FGEC-Metrics-DSGram","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxtse%2FGEC-Metrics-DSGram/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxtse%2FGEC-Metrics-DSGram/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxtse%2FGEC-Metrics-DSGram/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jxtse","download_url":"https://codeload.github.com/jxtse/GEC-Metrics-DSGram/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248602306,"owners_count":21131615,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-12T17:02:13.269Z","updated_at":"2025-04-12T17:06:21.131Z","avatar_url":"https://github.com/jxtse.png","language":"Python","funding_links":[],"categories":["A01_文本生成_文本对话"],"sub_categories":["大语言对话模型及数据"],"readme":"# DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction\n\n## Introduction\n\n**DSGram** is a novel evaluation framework designed to enhance the performance evaluation of Grammatical Error Correction (GEC) models, especially in the era of large language models (LLMs). Traditional reference-based evaluation metrics often fall short due to the inherent discrepancies between model-generated corrections and provided gold references. DSGram addresses this issue by introducing a dynamic weighting mechanism that integrates Semantic Coherence, Edit Level, and Fluency.\n\nThis repository contains the code and data associated with the paper: **\"DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction in the Era of Large Language Models\"** by Jinxiang Xie, Yilin Li, Xunjian Yin, and Xiaojun Wan.\n\n![图片](https://github.com/jxtse/GEC_Metrics_LLM/blob/main/images/FlowChart.png)\n\n## Repository Structure\n\n- `Dataset/`: Contains the datasets used for evaluation, including human-annotated and LLM-simulated sentences.\n  - `results/`: Directory to store the evaluation results.   \n- `DSGram/`: Source code for implementing the DSGram evaluation framework.\n\n![图片](https://github.com/jxtse/GEC_Metrics_LLM/blob/main/images/Example.png)\n\n## Citation\n\nIf you use DSGram in your research, please cite our paper:\n```\n@misc{xie2024dsgramdynamicweightingsubmetrics,\n      title={DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction in the Era of Large Language Models}, \n      author={Jinxiang Xie and Yilin Li and Xunjian Yin and Xiaojun Wan},\n      year={2024},\n      eprint={2412.12832},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL},\n      url={https://arxiv.org/abs/2412.12832}, \n}\n```\n\n# Tech Used\n ![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge\u0026logo=python\u0026logoColor=ffdd54) ![Azure](https://img.shields.io/badge/azure-%230072C6.svg?style=for-the-badge\u0026logo=azure-devops\u0026logoColor=white)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjxtse%2FGEC-Metrics-DSGram","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjxtse%2FGEC-Metrics-DSGram","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjxtse%2FGEC-Metrics-DSGram/lists"}