{"id":13535113,"url":"https://github.com/luhua-rain/MRC_Competition_Dureader","last_synced_at":"2025-04-02T00:32:30.500Z","repository":{"id":40608201,"uuid":"189688627","full_name":"luhua-rain/MRC_Competition_Dureader","owner":"luhua-rain","description":"机器阅读理解 冠军/亚军代码及中文预训练MRC模型","archived":false,"fork":false,"pushed_at":"2022-11-19T20:32:49.000Z","size":32511,"stargazers_count":733,"open_issues_count":39,"forks_count":150,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-02-02T03:32:00.782Z","etag":null,"topics":["bert","dureader","mrc","pytorch","qa","squad"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/luhua-rain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-06-01T04:04:53.000Z","updated_at":"2025-01-19T19:09:00.000Z","dependencies_parsed_at":"2023-01-21T08:04:53.483Z","dependency_job_id":null,"html_url":"https://github.com/luhua-rain/MRC_Competition_Dureader","commit_stats":null,"previous_names":["basketballandlearn/dureader-bert","basketballandlearn/mrc_competition_dureader"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luhua-rain%2FMRC_Competition_Dureader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luhua-rain%2FMRC_Competition_Dureader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luhua-rain%2FMRC_Competition_Dureader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luhua-rain%2FMRC_Competition_Dureader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/luhua-rain","download_url":"https://codeload.github.com/luhua-rain/MRC_Competition_Dureader/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246735344,"owners_count":20825220,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","dureader","mrc","pytorch","qa","squad"],"created_at":"2024-08-01T08:00:49.848Z","updated_at":"2025-04-02T00:32:25.486Z","avatar_url":"https://github.com/luhua-rain.png","language":"Python","funding_links":[],"categories":["BERT QA \u0026 RC task:"],"sub_categories":[],"readme":"## 机器阅读理解预训练模型及代码开源\n\n\n*********************** **更新** ***********************\n* 5/21：开源**基于大规模MRC数据再训练**的模型（包括`roberta-wwm-large`、`macbert-large`）\n* 5/18：开源比赛代码\n\n\n## Contents\n  - [基于大规模MRC数据再训练的模型](#基于大规模MRC数据再训练)\n  - [仓库介绍](#仓库介绍)\n  - [运行流程](#运行流程)\n  - [小小提示](#小小提示)\n\n\n## 基于大规模MRC数据再训练\n\n此库发布的再训练模型，在 阅读理解/分类 等任务上均有大幅提高\u003cbr/\u003e\n（已有多位小伙伴在 Dureader、法研杯、医疗问答 等多个比赛中取得**top5**的好成绩😁）\n\n|                模型/数据集                 |  Dureader-2021  |  tencentmedical |\n| ------------------------------------------|--------------- | --------------- |\n|                                           |    F1-score    |    Accuracy     |\n|                                           |  dev / A榜     |     test-1      |\n| macbert-large (哈工大预训练语言模型)         | 65.49 / 64.27  |     82.5        |\n| roberta-wwm-ext-large (哈工大预训练语言模型) | 65.49 / 64.27  |     82.5        |\n| macbert-large (ours)                      | 70.45 / **68.13**|   **83.4**    |\n| roberta-wwm-ext-large (ours)              | 68.91 / 66.91   |    83.1        |\n\n\n* **数据来源**\n  * 网上收集的大量中文MRC数据\n  （其中包括公开的MRC数据集以及自己爬取的网页数据等，\n  囊括了医疗、教育、娱乐、百科、军事、法律、等领域。）\n\n* **数据构造**\n  * 清洗\n    * 舍弃：context\u003e1024的舍弃、question\u003e64的舍弃、网页标签占比超过30%的舍弃。\n    * 重新标注：若answer\u003e64且不完全出现在文档中，则采用模糊匹配: 计算所有片段与answer的相似度(F1值)，取相似度最高的且高于阈值（0.8）\n  * 数据标注\n    * 收集的数据有一部分是不包含的位置标签的，仅仅是(问题-文章-答案)的三元组形式。\n      所以，对于只有答案而没有位置标签的数据通过正则匹配进行位置标注：\u003cbr/\u003e\n      ① 若答案片段多次出现在文章中，选择上下文与问题最相似的答案片段作为标准答案（使用F1值计算相似度，答案片段的上文48和下文48个字符作为上下文）；\u003cbr/\u003e\n      ② 若答案片段只出现一次，则默认该答案为标准答案。\n    * 采用滑动窗口将长文档切分为多个重叠的子文档，故一个文档可能会生成多个有答案的子文档。\n  * 无答案数据构造\n    * 在跨领域数据上训练可以增加数据的领域多样性，进而提高模型的泛化能力，而负样本的引入恰好能使得模型编码尽可能多的数据，加强模型对难样本的识别能力：\u003cbr/\u003e\n      ① 对于每一个问题，随机从数据中捞取context，并保留对应的title作为负样本;（50%）\u003cbr/\u003e\n      ② 对于每一个问题，将其正样本中答案出现的句子删除，以此作为负样本；（20%）\u003cbr/\u003e\n      ③ 对于每一个问题，使用BM25算法召回得分最高的前十个文档，然后根据得分采样出一个context作为负样本，\n      对于非实体类答案，剔除得分最高的context（30%）\n* **用途**  \n  * 此mrc模型可直接用于`open domain`，[点击体验](https://huggingface.co/luhua/chinese_pretrain_mrc_roberta_wwm_ext_large)\n  * 将此模型放到下游 MRC/分类 任务微调可比直接使用预训练语言模型提高`2个点`/`1个点`以上\n* **合作**\n  * 相关训练数据以及使用更多数据训练的模型/一起打比赛 可邮箱联系(luhua98@foxmail.com)~ \n  \n```\n----- 使用方法 -----\nfrom transformers import AutoTokenizer, AutoModelForQuestionAnswering\n\nmodel_name = \"chinese_pretrain_mrc_roberta_wwm_ext_large\" # \"chinese_pretrain_mrc_macbert_large\"\n\n# Use in Transformers\ntokenizer = AutoTokenizer.from_pretrained(f\"luhua/{model_name}\")\nmodel = AutoModelForQuestionAnswering.from_pretrained(f\"luhua/{model_name}\")\n\n# Use locally（通过 https://huggingface.co/luhua 下载模型及配置文件）\ntokenizer = BertTokenizer.from_pretrained(f'./{model_name}')\nmodel = AutoModelForQuestionAnswering.from_pretrained(f'./{model_name}')\n```\n\n## 仓库介绍\n* **目的**\n  * **开源了基于MRC数据再训练的模型**，在MRC任务下微调，效果大幅优于使用预训练的语言模型，其次，旨在提供一个效果不错的`强基线`\n  * 有些[mrc比赛](#比赛)由于\"年代久远\"整理不过来（`others`文件夹），但方案和代码都有，对比着看就看懂了\n* **优化**\n  * 代码基于Hugginface的squad代码。之前自己开发，版本多且许多细节没有考虑，便转移到squad代码上迭代。但其实现的类缺乏对中文的支持，推理结果有一些影响，**修改之后 此库能较好的支持中文，抽取的答案精度也尽可能不受影响**\n  \n\n## 运行流程\n\n脚本参数解释\n\n* `--lm`: 要加载的模型的文件夹名称\n* `--do_train`: 开启训练\n* `--evaluate_during_training`: 开启训练时的验证\n* `--do_test`:  开启预测\n* `--version_2_with_negative`: 开启适配于数据中有`无答案数据`（如：squad2.0、dureader2021）\n* `--threads`: 数据处理所使用的线程数（可以通过os.cpu_count()查看机器支持的线程数）\n  \n##### 一、数据 \u0026 模型：\n* 将train、dev、test等数据放在datasets文件夹下(样例数据已给出，符合格式即可)\n* 通过 export lm=xxx 指定模型目录\n\n##### 二、一键运行\n```python \nsh train_bert.sh  # sh test_bert.sh\n```\n\n##### 三、无答案问题\n* 如果包含无答案类型数据（如：squad2.0、dureader2021），加入--version_2_with_negative就行\n* 将数据替换为Dureader2021_checklist的数据, 加入--version_2_with_negative即可\n\n\n## 小小提示：\n* 代码上传前已经跑通。文件不多，所以如果碰到报错之类的信息，可能是代码路径不对、缺少安装包等问题，一步步解决，可以提issue\n* 环境\n  ```\n  pip install transformers==2.10.0 \n  ```\n* 代码基于transformers 2.10.0版本，但是预训练模型可以使用其他版本加载。转换为tf可使用[转换](https://github.com/huggingface/transformers/blob/master/src/transformers/models/bert/convert_bert_pytorch_checkpoint_to_original_tf.py)\n* 预训练相关参数 [参考](https://github.com/basketballandlearn/MRC_Competition_Dureader/issues/33)\n\n\n## 感谢\n[zhangxiaoyu](https://github.com/Decalogue) [zhongjialun](https://github.com/slaxes) [huanghui](https://github.com/huanghuidmml)  [nanfulai](https://github.com/nanfulai)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluhua-rain%2FMRC_Competition_Dureader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fluhua-rain%2FMRC_Competition_Dureader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluhua-rain%2FMRC_Competition_Dureader/lists"}